Bots From Extension: elasticsearch_v2

Elasticsearch - Read, Write and Append data to Elasticsearch / Opensearch Indices

This extension provides 11 bots.

Bot @esv2:append-index

Bot Position In Pipeline: Sink

Appends the record(s) to an existing or a new index on Elasticsearch

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name	Type	Default Value	Description
index*	Text		Index to update
continue_on_failure	Text	no	Continue on Failure. Provide yes/no
skip_empty_fields	Text	no	Skip empty fields in the row before sending. Provide yes/no

Example Pipelines Using this Bot

Bot *esv2:cluster-health

Bot Position In Pipeline: Source

Lists health stats for cluster in Elasticsearch

This bot expects a Full CFXQL.

Bot applies the Query on the data that is already loaded from previous bot or from a source.

Bot *esv2:cluster-node-stats

Bot Position In Pipeline: Source

Lists all cluster nodes stats in Elasticsearch

This bot expects a Full CFXQL.

Bot applies the Query on the data that is already loaded from previous bot or from a source.

Bot @esv2:delete-from-index-by-id

Bot Position In Pipeline: Sink

Delete one or more records from Elasticsearch index

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name	Type	Default Value	Description
index*	Text		Specify Elastichsearch index name
id_col*	Text		Specify the input column that has one or more ids of Elasticsearch index records.
concurrent_delete	Text	50	Specify number of records to be deleted concurrently

Bot #esv2:delete-from-index-by-query

Bot Position In Pipeline: Sink

Delete one or more records from Elasticsearch index.

This bot expects Full CFXQL.

Bot translates the Query to native query of the Data source supported by this extension.

This is a data sink which expects the following input parameters to be passed via input dataframe.

Input Dataframe Column Name	Type	Default Value	Description
`index`*	Text		Specify Elastichsearch index name

Example usage:

@dm:empty
    --> @dm:addrow index = 'index-id'
    --> #esv2:delete-from-index-by-query host_ip in
            ['xx.xx.xx.xx', 'yy.yy.yy.yy']

Bot @esv2:index-meta

Bot Position In Pipeline: Source

Reads the metadata from Elasticsearch index

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name	Type	Default Value	Description
index*	Text		Index to get metadata for

Bot *esv2:list-indices

Bot Position In Pipeline: Source

Lists all available indices on Elasticsearch

This bot expects a Full CFXQL.

Bot applies the Query on the data that is already loaded from previous bot or from a source.

Bot *esv2:list-shards

Bot Position In Pipeline: Source

Lists all shards for indices in Elasticsearch

This bot expects a Full CFXQL.

Bot applies the Query on the data that is already loaded from previous bot or from a source.

Bot #esv2:read-index

Bot Position In Pipeline: Sink

Reads the data from Elasticsearch indice(s). Add suffix .keyword for exact matches on keyword columns.

This bot expects Full CFXQL.

Bot translates the Query to native query of the Data source supported by this extension.

This is a data sink which expects the following input parameters to be passed via input dataframe.

Input Dataframe Column Name	Type	Default Value	Description
`index`*	Text		Specify Elastichsearch index name
`limit`	Text	1000	To limit the number of rows. Set 0 to remove the limit

Example usage:

@dm:empty
    --> @dm:addrow index = 'index-id' and limit=100
    --> #esv2:read-index host_ip in ['xx.xx.xx.xx', 'yy.yy.yy.yy']

Bot @esv2:read-index-histogram

Bot Position In Pipeline: Source Sink

Reads the Histogram data from Elasticsearch indice(s)

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name	Type	Default Value	Description
index*	Text		Index to read
hist_interval*	Text		Histogram Interval.
hist_field*	Text		Histogram field.
sort_fields	Text		comma separated fields to sort by.
sort_dir	Text	asc	Sort Direction asc/desc.

Bot @esv2:update-index

Bot Position In Pipeline: Sink

Updates the record(s) to an existing or a new index on Elasticsearch

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name	Type	Default Value	Description
index*	Text		Index to update
ids*	Text		comma separated columns to match against.
skip_empty_fields	Text	no	Skip empty fields in the row before sending. Provide yes/no