Skip to content

Bots From Extension: elasticsearch_v2

Elasticsearch - Read, Write and Append data to Elasticsearch / Opensearch Indices

This extension provides 8 bots.





Bot @esv2:append-index

Bot Position In Pipeline: Sink

Appends the record(s) to an existing or a new index on Elasticsearch

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
index* Text Index to update
continue_on_failure Text no Continue on Failure. Provide yes/no
skip_empty_fields Text no Skip empty fields in the row before sending. Provide yes/no

Example Pipelines Using this Bot







Bot @esv2:delete-from-index-by-id

Bot Position In Pipeline: Sink

Delete one or more records from Elasticsearch index

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
index* Text Specify Elastichsearch index name
id_col* Text Specify the input column that has one or more ids of Elasticsearch index records.
concurrent_delete Text 50 Specify number of records to be deleted concurrently







Bot #esv2:delete-from-index-by-query

Bot Position In Pipeline: Sink

Delete one or more records from Elasticsearch index.

This bot expects Full CFXQL.

Bot translates the Query to native query of the Data source supported by this extension.

This is a data sink which expects the following input parameters to be passed via input dataframe.

Input Dataframe Column Name Type Default Value Description
index* Text Specify Elastichsearch index name

Example usage:

1
2
3
4
@dm:empty
    --> @dm:addrow index = 'index-id'
    --> #esv2:delete-from-index-by-query host_ip in
            ['xx.xx.xx.xx', 'yy.yy.yy.yy']







Bot @esv2:index-meta

Bot Position In Pipeline: Source

Reads the metadata from Elasticsearch index

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
index* Text Index to get metadata for







Bot *esv2:list-indices

Bot Position In Pipeline: Source

Lists all available indices on Elasticsearch

This bot expects a Full CFXQL.

Bot applies the Query on the data that is already loaded from previous bot or from a source.







Bot #esv2:read-index

Bot Position In Pipeline: Sink

Reads the data from Elasticsearch indice(s).

This bot expects Full CFXQL.

Bot translates the Query to native query of the Data source supported by this extension.

This is a data sink which expects the following input parameters to be passed via input dataframe.

Input Dataframe Column Name Type Default Value Description
index* Text Specify Elastichsearch index name
limit Text 1000 To limit the number of rows. Set 0 to remove the limit

Example usage:

1
2
3
@dm:empty
    --> @dm:addrow index = 'index-id' and limit=100
    --> #esv2:read-index host_ip in ['xx.xx.xx.xx', 'yy.yy.yy.yy']







Bot @esv2:read-index-histogram

Bot Position In Pipeline: Source Sink

Reads the Histogram data from Elasticsearch indice(s)

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
index* Text Index to read
hist_interval* Text Histogram Interval.
hist_field* Text Histogram field.
sort_fields Text comma separated fields to sort by.
sort_dir Text asc Sort Direction asc/desc.







Bot @esv2:update-index

Bot Position In Pipeline: Sink

Updates the record(s) to an existing or a new index on Elasticsearch

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
index* Text Index to update
ids* Text comma separated columns to match against.
skip_empty_fields Text no Skip empty fields in the row before sending. Provide yes/no