| option_to_modify_table |
NO
|
floriandietz@elody.com
|
This Tag is used by [[program:BasicDataCleansingAndAnalysisForPandas]] to describe the changes that an Option is going to make to a [[symbol:modifiable_file]] if it is executed.
Other Programs are free to use this Tag as well, in the same way, to coordinate with each other.
Whenever a Program considers whether or not to create an Option to make a change, it first checks if an earlier option already exists with an option_to_modify_table Tag on it that has the same weight, comment and miscellaneous arguments. If such an Option exists, the new Option is not created as that would be redundant.
Mark a Tag of this kind with [[symbol:accepted]] to indicate whether or not the modification it describes should be performed. To reject an option_to_modify_table, simply deactivate its corresponding Option without marking it as [[symbol:accepted]].
Once the modification described by this Tag has been performed, mark it with [[symbol:handled]].
If you decide to use this mechanism as well, be careful about the way you define the weight, comment and miscellaneous arguments of this Tag, as this affects whether or not a possible change will be offered as an Option a second time. In general, any fact about the table that influences the decision whether or not to make the change should be represented in the weight/comment/miscellaneous arguments. This way, the Option is made a second time exactly if the circumstances changed enough that it's worth asking again.
Arguments of this Tag:
0 : An Option.
1 : A [[symbol:modifiable_file]] Tag.
2+ : miscellaneous arguments. Note that [[program:BasicDataCleansingAndAnalysisForPandas]] compares these by reference, not value. |
| column_nulls_count |
NO
|
floriandietz@elody.com
|
The weight of this Tag counts the number of null values of a column.
Arguments of this Tag:
0 : A [[symbol:column]] Tag. |
| column_info_types |
NO
|
floriandietz@elody.com
|
The comment of this Tag describes the datatype of a column.
Valid values are:
-primitive values like int32, int64, float32,f loat64, boolean, string, datetime, ...
-categorical: Different values represent different categories.
-numeric: Values can be ordered.
-discrete: Values are numeric and there is a finite number of values between any two values.
-continuous: Values are numeric and there can be an infinite number of values between two values.
-primary_key: A different value for every entry and no null entries.
The purpose of this Tag is to inform other Rules and Programs so that they can make intelligent decision. Therefore, being useful is more important than being correct. An integer indicator variable that only has the value 0 and 1, for example, is technically numeric and discrete, but it may make more sense to mark it as categorical instead. The same is true for IDs and for other kinds of numeric values that fall into a suspiciously small number of clusters of different values.
If multiple datatypes apply, separate them with commas. Examples:
-The name of the customer: "string,categorical"
-the ID of the item the customer bought: "int64,int,categorical"
-The date and time on which an item was bought: "datetime,continuous,numeric"
-The number of items the customer bought: "int64,int,discrete,numeric"
-The price the customer paid: "float64,float,continuous,numeric"
-The discount of the item: "float64,float,continuous,numeric,categorical"
Note the last example: If the discount is a number, but there are only a fixed number of different discount values, it can be useful both as a numeric value and as a categorical value, depending on the algorithm you want to use.
Arguments of this Tag:
0 : A [[symbol:column]] Tag. |
| column_info_name |
NO
|
floriandietz@elody.com
|
The comment of this Tag describes the name of a column. It is null if the table is unnamed.
Arguments of this Tag:
0 : A [[symbol:column]] Tag. |
| column_info_index |
NO
|
floriandietz@elody.com
|
The weight of this Tag describes the index of a column within a table.
(Note that this is a separate Tag because it is possible to shift the index of a column, for instance by deleting another column in the same table. When that happens, you don't want to have to redefine the whole column and recreate all of its Tags. Instead, you can just update only this Tag and leave the rest as it is.)
Arguments of this Tag:
0 : A [[symbol:column]] Tag. |
| column |
NO
|
floriandietz@elody.com
|
Defines a column of a table. Other Tags describing the column can be attached to it.
Arguments of this Tag:
0 : Some object that represents a table, e.g. a file or a [[symbol:modifiable_file]]. |
| table_info_row_count |
NO
|
floriandietz@elody.com
|
The weight of this Tag describes the number of entries of a table.
Arguments of this Tag:
0 : Some object that represents a table, e.g. a file or a [[symbol:modifiable_file]]. |
| table_info_name |
NO
|
floriandietz@elody.com
|
The comment of this Tag describes the name of a table. It is null if the table is unnamed.
Arguments of this Tag:
0 : Some object that represents a table, e.g. a file or a [[symbol:modifiable_file]]. |
| current_file |
NO
|
floriandietz@elody.com
|
This Tag is created to indicate that a [[symbol:modifiable_file]] has a new version of the file available.
Arguments of this Tag:
0 : The [[symbol:modifiable_file]] Tag.
1 : The file. |
| modifiable_file |
NO
|
floriandietz@elody.com
|
Used to indicate a file that can be updated / replaced with newer versions. Works together with [[symbol:current_file]]. Don't forget to always create an initial [[symbol:current_file]] as well.
Tags describing the content of the file can be attached to this Tag. Since the arguments of a Rule or Option will by default always look for the last object that matches the filter, updating/overwriting these descriptive Tags is easy: You can in most cases simply attach new Tags to this Tag every time the file changes, without having to !nullify any of the old ones.
Arguments of this Tag:
0 : The original file.
1 : Any object that indicates the purpose of all modifications of the file. For example, a [[symbol:task_data_cleansing_and_analysis_for_pandas]] Tag. |
