Bots From Extension: minio
Minio
This extension provides 34 bots.
Bot @minio:add-bounded-dataset
Bot Position In Pipeline: Source Sink
Add schema bounded dataset
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the dataset | |
| schema_name* | Text | Name of the schema to bind this dataset |
Bot @minio:add-schema
Bot Position In Pipeline: Source Sink
Add json schema
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| schema_file* | Text | File path or URL of the json schema file | |
| name* | Text | Name of the json schema |
Bot *minio:bookmark-list
Bot Position In Pipeline: Source
List of saved bookmarks
This bot expects a Full CFXQL.
Bot applies the Query on the data that is already loaded from previous bot or from a source.
Bot @minio:content-to-object
Bot Position In Pipeline: Sink
Convert data from a column into objects
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_content_column* | Text | Name of the column in input that contains the data | |
| output_column* | Text | Column name where object names will be inserted | |
| output_folder* | Text | Folder name where objects will be stored |
Bot @minio:file-to-object
Bot Position In Pipeline: Sink
Convert files from a column into objects
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_filename_column* | Text | Name of the column in input that contains the filenames | |
| output_column* | Text | Column name where object names will be inserted | |
| output_folder* | Text | Folder name where objects will be stored |
Bot @minio:get-from-location
Bot Position In Pipeline: Source Sink
Retrieve dataset from a specified location
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| path* | Text | Path to object in Minio bucket. | |
| empty_as_null | Text | yes | While reading the saved datasets, treat empty strings as Nulls. 'yes' or 'no' |
| bucket_name* | Text | Minio bucket name |
Bot @minio:list-all-schemas
Bot Position In Pipeline: Source Sink
List all json schemas
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
Bot @minio:list-from-location
Bot Position In Pipeline: Source
List datasets in a specified location
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| location | Text | Location in Minio bucket. By default, all folders in bucket will be listed | |
| bucket_name* | Text | Minio bucket name |
Bot @minio:load-bookmark
Bot Position In Pipeline: Source Sink
Load a previously saved bookmark
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the bookmark to load | |
| default | Text | Name of the bookmark to load |
Bot @minio:load-ml-model
Bot Position In Pipeline: Source Sink
Load the ML Model with 'name' and 'model_type'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the ML Model to load | |
| model_type* | Text | ML Model type. Must be one of 'clustering', 'regression', 'classification' |
Bot @minio:load-template
Bot Position In Pipeline: Source Sink
Load the formatting template with 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the template to load |
Bot *minio:ml-model-list
Bot Position In Pipeline: Source
List of saved ML Models
This bot expects a Full CFXQL.
Bot applies the Query on the data that is already loaded from previous bot or from a source.
Bot @minio:object-add
Bot Position In Pipeline: Source Sink
Add object to a folder
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Object name | |
| folder* | Text | Folder name on the object storage | |
| input_file* | Text | File from which object will be added | |
| description | Text | Description | |
| overwrite | Text | yes | If file already exists, overwrite without prompting |
Bot @minio:object-delete
Bot Position In Pipeline: Source Sink
Delete object from a folder
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Object name | |
| folder* | Text | Folder name on the object storage |
Bot @minio:object-delete-list
Bot Position In Pipeline: Sink
Delete list of objects
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_object_column* | Text | Column with object names |
Bot @minio:object-get
Bot Position In Pipeline: Source Sink
Get Object from a folder
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Object name | |
| folder* | Text | Folder name on the object storage | |
| save_to_file | Text | Save the downloaded object to specified file | |
| save_to_dir | Text | Save the downloaded object to specified directory |
Bot @minio:object-list
Bot Position In Pipeline: Source
List objects for a folder
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| folder | Text | Folder name on the object storage |
Bot @minio:object-to-content
Bot Position In Pipeline: Sink
Convert object pointers from a column into content
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_object_column* | Text | Name of the column in input that contains the object name | |
| output_column* | Text | Column name where content will be inserted |
Bot @minio:object-to-file
Bot Position In Pipeline: Sink
Convert object pointers from a column into file
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_object_column* | Text | Name of the column in input that contains the objects | |
| output_column* | Text | Column name where filenames need to be inserted |
Bot @minio:object-to-inline-img
Bot Position In Pipeline: Sink
Convert object pointers from a column into inline HTML img tags
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_object_column* | Text | Name of the column in input that contains the JPEG or PNG Image | |
| output_column* | Text | Column name where HTML img tag code need to be inserted |
Bot @minio:recall
Bot Position In Pipeline: Source Sink
Recall (load) a previously saved dataset
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name | Text | Name of the dataset to recall.(name is mandatory, if tag parameter is not given) | |
| cache | Text | no | Cache the result for future recalls. 'yes' or 'no' |
| cache_refresh_seconds | Text | 120 | Refresh the cache (if new update available) after specified seconds |
| empty_as_null | Text | yes | While reading the saved datasets, treat empty strings as Nulls. 'yes' or 'no' |
| return_empty | Text | no | Return an empty dataframe if an error occurs loading the dataset |
| empty_df_columns | Text | Comma separated list of columns for empty dataframe | |
| ignore_dtypes | Text | no | Ignore column data types during loading |
| make_copy | Text | yes | Specifies whether to make copy of dataset for temp datasets. 'yes' or 'no' |
| tag | Text | Name of the tag, return the first occurrence of the dataset having the given tag |
Bot @minio:recall-chunked
Bot Position In Pipeline: Source
Recall (load) a previously saved dataset as a data stream. Loads num_rows in each chunk.
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the dataset to recall | |
| num_rows* | Text | Number of rows to fetch in each chunk | |
| empty_as_null | Text | yes | While reading the saved datasets, treat empty strings as Nulls. 'yes' or 'no' |
| return_empty | Text | no | Return an empty dataframe if an error occurs loading the dataset |
| empty_df_columns | Text | Comma separated list of columns for empty dataframe | |
| ignore_dtypes | Text | no | Ignore column data types during loading |
Bot @minio:save
Bot Position In Pipeline: Sink
Save the dataset with 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the dataset to save | |
| format | Text | csv | Data format on the object storage. Ignored for the temporary datasets. Must be one of 'csv' or 'parquet' |
| publish | Text | Deprecated. Name of the tag to publish in cfxDimensions platform. Can be used only with Dimensions configuration. |
|
| append | Text | no | If set to 'yes', appends the input dataset as a chunk to the existing dataset if any. Valid values are 'yes', 'no' |
| return_appended_dataset | Text | no | If set to 'yes' and append is 'yes', returns the full appended dataset. Valid values are 'yes', 'no' |
| tag | Text | Name of the tag, appends the given tag name in the metadata |
Bot @minio:save-bookmark
Bot Position In Pipeline: Sink
Save the bookmark with 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the bookmark to save | |
| value_column* | Text | Name of value column | |
| value_type | Text | timestamp | Value type (timestamp, numeric, text) |
| ts_format | Text | If value_type is timestamp, format. Valid units are s, ms, ns or null for string format. | |
| value_func | Text | max | Value functions are first, last, min, max |
Bot @minio:save-ml-model
Bot Position In Pipeline: Source Sink
Save the ML Model with 'name' and 'model_type'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the ML Model to save | |
| model_type* | Text | ML Model type. Must be one of 'clustering', 'regression', 'classification' | |
| description | Text | Template content column | |
| model_data_path* | Text | ML Model file path |
This bot also accepts wildcard parameters. Any additional name = 'value' parameters are passed to the bot.
Bot @minio:save-template
Bot Position In Pipeline: Sink
Save the formatting template with 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the template to save | |
| description_col | Text | description | Template description column |
| content_col | Text | content | Template content column |
| content_type_col | Text | content_type | Template content type column |
Bot @minio:save-to-location
Bot Position In Pipeline: Sink
Save the dataset to a specified location
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name | Text | Name of the dataset to save. If not provided, will be extracted from location | |
| format | Text | Data format on the object storage. Supported formats are csv, paraquet, gzip, json and zip. If not provided, will be extracted from location |
|
| location | Text | Location in Minio bucket to save the object. | |
| ignore_index | Text | no | Ignore index columns while saving file to location. Possible values are 'yes' or 'no'. (Applicable only for csv format) |
| bucket_name* | Text | Minio bucket name |
Bot *minio:savedlist
Bot Position In Pipeline: Source
List of saved datasets
This bot expects a Full CFXQL.
Bot applies the Query on the data that is already loaded from previous bot or from a source.
Bot @minio:stack-generate
Bot Position In Pipeline: Source Sink
Generate stack from input topology dataset
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_stack_name* | Text | Name of input stack json to generate stack from | |
| app_type | Text | OIA | Name of application to publish stack to |
| output_stack_name* | Text | Name of application to publish stack to | |
| room_id* | Text | Location to publish stack to | |
| incident_id* | Text | Comma separated Incident id under which stack will be referenced | |
| incident_summary* | Text | Comma separated summary for incidents | |
| fqdn* | Text | FQDN | |
| url* | Text | URL | |
| ip_address* | Text | IP Address |
Bot *minio:stack-list
Bot Position In Pipeline: Source
List of application Stacks
This bot expects a Full CFXQL.
Bot applies the Query on the data that is already loaded from previous bot or from a source.
Bot @minio:stack-load
Bot Position In Pipeline: Source Sink
Load application Stack specified by 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the Stack to load |
Bot @minio:stack-save
Bot Position In Pipeline: Source Sink
Save application Stack specified by 'name'
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| name* | Text | Name of the Stack to save | |
| description | Text | Stack description | |
| stack_data_column | Text | data | Column name where stack definition is stored in dataframe |
| additional_nodes_rules | Text | Name of dataset with rules to create and attach new node to matching existing nodes. |
Bot *minio:template-list
Bot Position In Pipeline: Source
List of saved formatting templates
This bot expects a Full CFXQL.
Bot applies the Query on the data that is already loaded from previous bot or from a source.
Bot @minio:validate-data
Bot Position In Pipeline: Sink
Check integrity of data data using 'schema' that has been uploaded previously
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| schema_name* | Text | Schema to verify the data against. | |
| failfast | Text | yes | Specify 'yes'(default) to abort quickly on first error or 'no' to keep validating records |
| action | Text | none | Action to take if validation fails. Must be one of 'none' (default), 'fail','skip-block', 'skip-pipeline' |