| Enrich_Convert_Excel_to_Pandas_Dataframe |
9 (release) |
- |
initial_tools |
This program takes a file under the parameter name "input_file".
It treats that file as an Excel file and converts it into a pickled Pandas dataframe file.
The three programs [[program:Enrich_Convert_Excel_to_Pandas_Dataframe]], [[program:Enrich_Convert_CSV_to_Excel]] and [[program:Enrich_Convert_Pandas_Dataframe_to_CSV]] are circular and are called by corresponding rules.
This conversion is very simple:
It does not attempt to clean anything and makes naive assumptions about file formats.
If the Excel file contains multiple worksheets, this will create one file per worksheet.
(A note for new developers joining LOD: This Program is not very good. It was written because something is better than nothing. You are invited to write better alternatives if you encounter any errors. In particular, an interactive HTML representation that lets you select the area of interest would probably be best.)
|
| PresentBasicInfoOnTableForPandas |
44 (release) |
- |
floriandietz@elody.com |
For a [[symbol:task_present_table_for_pandas]], checks all [[symbol:require_present_table_main_info]], [[symbol:require_present_table_column_info]] and [[symbol:require_present_table_geographical_location]] on it and creates an Option for each of these that will create a Message to the user describing the file/the column and mark the require_ Tag with !provide.
Arguments:
theTask : a [[symbol:task_present_table_for_pandas]]
theFile : the file object to work on
Notes about the geographical visualization:
-This map visualization is FREE, but Google charges for each address lookup, so we don't allow more than a few addresses at once and only a limited number of times a day, across all users. We will enable visualization for more data once we have implemented a payment method. If you don't want to wait that long, or don't want to pay money, feel free to upload an alternative to this visualization. Maybe you can make something that is free to use using OpenStreetMaps?
-If... |
| InteractGetUserFeedback |
1 (release) |
- |
floriandietz@elody.com |
This Program handles a [[symbol:task_get_user_feedback]].
Arguments:
theTask : A [[symbol:task_get_user_feedback]] Tag.
|
| VisualizeTimeseriesForPandas |
7 (release) |
- |
floriandietz@elody.com |
Creates a graph of a timeseries for a pandas file that has been analyzed by [[symbol:task_data_cleansing_and_analysis_for_pandas]].
This Program is designed to visualize timeseries for whole days, not for dates with time components. The visualization will look bad if there are time components. It also can not display dates or timedeltas on the y-axis, only numbers. If you want to improve this program so it can support these additional datatypes, feel free to do so!
If this program is called with only the file but without a choice of parameters, it will create an Option that asks the user to choose parameters, then calls this program again.
If this program is called with parameters, it does two things:
-It creates an Option that will auto-execute and show the visualization.
-It creates a pandas file containing a dataframe with two columns, which are the content of the graph. If you have a better way to visualize the graph, you can deactivate the Option and just use this... |
| DemoDataExplorationUsingExampleFile |
30 (release) |
- |
floriandietz@elody.com |
This Program is run by [[rule:Demo-data-exploration-using-example-file]].
It generates two example files to test data exploration with. Both files have the same content. The first is a pickled pandas DataFrame, the second is a CSV file.
|
| BasicDataCleansingAndAnalysisForPandas |
57 (release) |
- |
floriandietz@elody.com |
This program is called by [[rule:Do-basics-for-task-data-cleansing-and-analysis-for-pandas]] to perform basics data cleansing and analysis on [[symbol:task_data_cleansing_and_analysis_for_pandas]].
This program also acts as the central control unit to solve [[symbol:task_data_cleansing_and_analysis_for_pandas]].
---
Note: This program is part of the first real, practical example that was developed for Elody. It is deliberately overengineered to test boundaries. It is inefficient, but easily extensible by other contributors.
---
This program executes multiple times. Every time it runs, it adds a number of Options and [[symbol:option_to_modify_table]] Tags (or none) and applies any changes from previously defined [[symbol:option_to_modify_table]] Tags that have been accepted in the meantime. If no changes were defined or applied and this isn't the first time the program executes, it will mark the [[symbol:require_open_ended_work]] with !provide.
(This means that this program runs... |
| AdvancedDataCleansingAndAnalysisForPandas |
20 (release) |
- |
floriandietz@elody.com |
This Program works on [[symbol:task_data_cleansing_and_analysis_for_pandas]] to supplement [[program:BasicDataCleansingAndAnalysisForPandas]].
Features:
-Convert strings columns with numerical data that end on a percentage symbol to a float.
-Check numerical columns for outliers, using IQR. The user is presented with an Option to remove or replace outliers.
-Recognize possible misspellings in strings that could be categorical. This is for identifying mistakes in categorical columns, not for natural language processing. The user is presented with an Option for each possibly misspelt word and its most likely correction. The identification of possible misspellings follows a simple algorithm that takes uses Levenshtein distance and compares frequently occurring words to rarely occurring ones.
-Recognize a pair of columns as a pair of geographic coordinates usable with [[symbol:info_geographic_coordinate_column_pair]] if all numbers have a valid range and type and the column names... |
| Demo_a_simple_timeseries_prediction |
9 (release) |
- |
tutorial_developer_demo |
This Program is part of the tutorial.
Receives parameters from the option generated by the Rule [[rule:Demo-a-simple-timeseries-prediction]].
These parameters contain a timeseries that is to be predicted.
Performs a prediction on this timeseries.
Depending on what the user chose in the parameters, the timeseries is either displayed using a simple inbuilt graphing library, or that task is delegated to Elody's other programs.
If the timeseries entered by the user couldn't be parsed, a new Option is generated to ask them to enter it again.
|
| Enrich_Convert_CSV_to_Pandas_Dataframe |
3 (release) |
- |
initial_tools |
This program takes a file under the parameter name "input_file".
It treats that file as a CSV file and converts it into a pickled Pandas Dataframe file.
This program was added after the following programs, to speed things up a bit because it turned out that the conversion CSV->Pandas needed to be fast and so couldn't take two steps:
-[[program:Enrich_Convert_Excel_to_Pandas_Dataframe]]
-[[program:Enrich_Convert_CSV_to_Excel]]
-[[program:Enrich_Convert_Pandas_Dataframe_to_CSV]]
This conversion uses standard Pandas functions for parsing a CSV file and performs no modifications whatsoever, loading the file in the most naive way. In particular, headers are not recognized as headers and are left as part of the DataFrame.
(A note for new developers joining LOD: This Program is not very good. It was written because something is better than nothing. You are invited to write better alternatives if you encounter any errors.)
|
| Enrich_Convert_Pandas_Dataframe_to_CSV |
7 (release) |
- |
initial_tools |
This program takes a file under the parameter name "input_file".
It treats that file as a pickled Pandas dataframe file and converts it into a CSV file.
The three programs [[program:Enrich_Convert_Excel_to_Pandas_Dataframe]], [[program:Enrich_Convert_CSV_to_Excel]] and [[program:Enrich_Convert_Pandas_Dataframe_to_CSV]] are circular and are called by corresponding rules.
This conversion is very simple:
It does not attempt to clean anything and makes naive assumptions about file formats. It also can't handle quoted strings or dates.
The header and index are dropped. This is simply to ensure that this Program is reversible by using [[program:Enrich_Convert_CSV_to_Excel]] and [[program:Enrich_Convert_Excel_to_Pandas_Dataframe]].
(A note for new developers joining LOD: This Program is not very good. It was written because something is better than nothing. You are invited to write better alternatives if you encounter any errors.)
|
