Bots From Extension: file
Read & Write From Files
This extension provides 14 bots.
Bot @files:appendfile
Bot Position In Pipeline: Sink
Save dataframe to file using filename, if file exists, append the data. Supports only CSV format.
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Filename or URL for the output data. Only CSV files are supported for append. |
Bot @files:archivefile
Bot Position In Pipeline: Sink
Archive dataframe to file using filename and folder format specified. File is saved as compressed (gzipped) json.
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| archive_folder* | Text | Root folder for creating archive sub-directories and files. For e.g. - /Archive | |
| file_format | Text | /%Y/%m/%d/data.json | Defines the folder structure and filename to use. Supports Jinja style variables in file format. E.g. - /syslog/%Y/%m/%d/%H/%M/{{CATEQUIP}}.json |
| bucket_minutes | Text | 5 | Frequency of rolling over to a new bucket/folder |
| timestamp_column | Text | Timestamop column name in the source data. This field is used to extract the timestamp value and create folder sctructure based on that value. If not specified, current time is used |
|
| compress | Text | no | Should the archieved file be compressed? If you want files to be compressed, set it to 'yes'. |
| append | Text | no | If set to 'yes', data is appended to existing file |
| fix_path_elements | Text | yes | Fix the path |
| overwrite | Text | no | Set it to 'yes' to overwrite the archieve file if it exists |
Bot @files:datasets-to-xlsx
Bot Position In Pipeline: Source
Export specified datasets to an Excel file, each dataset as a different sheet. Returns summary of export.
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Excel filename for the exported data | |
| datasets* | Text | Regular expression listing of datasets to be included in export |
Bot @files:ec-entity-analysis
Bot Position In Pipeline: Source
Processes EC zip file and Builds relationships between the entities
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_zipfile_path* | Text | Path to EdgeCollector ZIP file | |
| output_path* | Text | Path to store output files | |
| device_ops_view | Text | False | Enable to create device-ops-view json |
| device_view | Text | False | Enable to create device-view json |
| poe_summary | Text | False | Enable to create poe_summary json |
Bot @files:ec-get-collected-data
Bot Position In Pipeline: Source
Parse collected JSON data from a CloudFabrix EdgeCollector ZIP Filepath
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Path to EdgeCollector ZIP filename | |
| ip_pattern | Text | .* | IP Address pattern for which data should be loaded |
| ip_list | Text | Comma searated IP Addresses for which data should be loaded | |
| json_path | Text | JSON path to a specific object in each input JSON file (seperated by .), example: 'base.data' |
Bot @files:ec-get-discovery-results
Bot Position In Pipeline: Sink
Parse collected JSON data from a CloudFabrix EdgeCollector Agent results
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| input_column | Text | DISCOVERY_ZIP_FILE | Column Name in input dataframe which is usually output of EdgeCollector Agent discovery results |
| ip_pattern | Text | .* | IP Address pattern for which data should be loaded |
| ip_list | Text | Comma searated IP Addresses for which data should be loaded | |
| json_path | Text | JSON path to a specific object in each input JSON file (seperated by .), example: 'base.data' | |
| minio_file_location | Text | Location of stored Edgecollector Agent discovery file in minio | |
| read_failed_devices | Text | no | Specify 'yes' to include failed devices or 'no' to load only successful devices. By default loads successfull devices only |
Bot @files:ec-get-raw-data
Bot Position In Pipeline: Source
Get collected raw text from a CloudFabrix EdgeCollector ZIP File
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Path to EdgeCollector ZIP filename | |
| ip_pattern | Text | .* | IP Address pattern for which data should be loaded |
| raw_filename_pattern | Text | .* | Regular expression pattern to filter for specific filenames |
Bot @files:export-content-to-zip-file
Bot Position In Pipeline: Sink
Export each row as a file into a ZIP file
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| output_zipfile* | Text | Output ZIP filename | |
| filename_column | Text | filename | Column with filename to be added to ZIP, one per each row |
| content_column | Text | content | File content column |
| encoding_column | Text | encoding | Column name with encoding type. Encoding type can be 'text' or 'binary'. If column not found, uses 'text' encoding |
Bot @files:get-zip-fileinfo
Bot Position In Pipeline: Sink
List information about each file within the provided ZIP File
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Path to ZIP file | |
| is_minio_file | Text | yes | 'yes' if the zip file is located in minio. Valid values are 'yes' or 'no' |
Bot @files:load-content-from-file
Bot Position In Pipeline: Sink
Load content from file specified by 'filename' column into 'content' column
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename_column | Text | filename | Column with filename to be loaded per each row |
| content_column | Text | content | File content column where loaded data will be placed |
| encoding_column | Text | encoding | Column name with encoding type. Encoding type can be 'text' or 'binary'. If column found, uses 'text' encoding |
Bot @files:loadfile
Bot Position In Pipeline: Source
Load dataframe from file using filename, filetype, sheet and line_read parameters
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Filename or URL for the input data | |
| filetype | Text | File type is either csv/json/xlsx. If not specified, it will be derived from extension. | |
| sheet | Text | Specify sheet name if file type is xlsx. If not specified, first sheet will be returned. | |
| line_read | Text | no | Only applicable for JSON. By default, the whole file is loaded as JSON. If you want to load as a json object per line, set it to 'yes'. |
Bot @files:rda-get-discovery-results
Bot Position In Pipeline: Sink
Parse collected data from a CloudFabrix RDA ZIP Filepath
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Path to ZIP file | |
| type | Text | RDA collection type included within the meta.json | |
| descriptors | Text | Comma separated RDA collection descriptors included within the meta.json. Eg: cucm-servers,cucm-servers | |
| is_minio_file | Text | yes | 'yes' if the zip file is located in minio. Valid values are 'yes' or 'no' |
Bot @files:save-content-to-file
Bot Position In Pipeline: Sink
Save content from 'content' column into file specified by 'filename' column
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename_column | Text | filename | Column with filename to be saved per each row |
| content_column | Text | content | File content column |
| encoding_column | Text | encoding | Column name with encoding type. Encoding type can be 'text' or 'binary'. If column not found, uses 'text' encoding |
| minio_location | Text | Location to save the file in minio |
Bot @files:savefile
Bot Position In Pipeline: Sink
Save dataframe to file using filename, filetype and sheet parameters
This bot expects a Restricted CFXQL.
Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot
| Parameter Name | Type | Default Value | Description |
|---|---|---|---|
| filename* | Text | Filename for the output data | |
| filetype | Text | auto | File type is either csv/json/xlsx. If not specified, it will be derived from extension. |
| sheet | Text | Sheet 1 | Specify sheet name if file type is xlsx. If not specified, sheet name will be 'Sheet 1' |