Skip to content

Bots From Extension: rn

RDA Networking & Streaming

This extension provides 9 bots.





Bot @rn:read-stream

Bot Position In Pipeline: Source

Read one more rows from a specified stream in a chunked manner. No acknowledgment is expected.

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text RDA Network stream name
batch_count Text 100 Wait for batch_count messages or batch_wait_time seconds for each batch
batch_wait_time Text 1.0 Wait for batch_count messages or batch_wait_time seconds for each batch
group* Text Consumer group name. If specified, must be a valid group name without any spaces.

Example Pipelines Using this Bot







Bot @rn:read-stream-with-ack

Bot Position In Pipeline: Source

Read one message at a time from the specified stream and send ack to publisher

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text RDA Network stream name
group* Text Consumer group name. If specified, must be a valid group name without any spaces.







Bot @rn:service-invoke

Bot Position In Pipeline: Sink

Pipeline as a Microservice (PaaM) request invoker. Make a service request with input dataframe and output the data returnedd by the service. This bot can be used to make a service request call to any microservice that is implemented by @rn:service-receive-request

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text Initiate a call to this service and wait for the response
timeout Text 10.0 Timeout in seconds. Default is 10s.
action Text fail Acion to perform if the requst fails. Must be one of continue, fail, skip-block, skip-pipeline.
If 'continue', bot send original input as output.







Bot @rn:service-receive-request

Bot Position In Pipeline: Source

Pipeline as a Microservice (PaaM) request receeiver. This bot can be used convert any pipeline into a microservice which will receive to requests and send response back to the client.

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text Name for the service which this pipeline implements using request-response paradigm.
desription Text Optional description of the service.
min_rows Text 0 Minimum number of rows in input. Default is 0. If specified it must >= 0.
max_rows Text 0 Maximum number of rows in input. If specified it must >= 0. Default is no limit.







Bot @rn:service-respond-request

Bot Position In Pipeline: Sink

Pipeline as a Microservice (PaaM) request responder. This bot used conjuction with @rn:service-receive-request

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
flags Text Optional flags. Currently no specific flags are expected.







Bot @rn:write-macaw-notifcation-asset-db

Bot Position In Pipeline: Source Sink

Send Macaw Notification to stream ASSET_DB_UPDATED to notify that a dataset has been updated

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
dataset* Text Input Dataset Name to be published to Asset DB
tag* Text Dataset Tag Name for Asset DB
model* Text Dataset Model Name for Asset DB
submitted_by Text rda_pipeline Optional. Name of the data publisher
tag_prefix Text Optional. Tag prefix to be used in Asset DB
customerName Text Optional. Customer Name used in Asset DB
bucketName Text Optional.if not provided the bucketName will be picked from dataset-location bot
objectName Text Optional.if not provided the objectName will be picked from dataset-location bot







Bot @rn:write-stats-to-stream

Bot Position In Pipeline: Sink

Write input dataframe stats (row count, byte count) to specified stream.

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text RDA network stream name
groupby Text Optional comma separated list of columns. If not specified, publishes total row count and byte_count.
If Specified, computes group level row count and byte count.

This bot also accepts wildcard parameters. Any additional name = 'value' parameters are passed to the bot.

Example Pipelines Using this Bot







Bot @rn:write-stream

Bot Position In Pipeline: Sink

Write input dataframe as messages to a specified stream. Each row is encoded as a JSON object. No acknowledgment is expected.

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text RDA network stream name
compression Text auto Compress data while publising. Possible values are 'auto', 'yes' or 'no'. 'auto' would enable
compression if size meets a specific criteria

Example Pipelines Using this Bot







Bot @rn:write-stream-with-ack

Bot Position In Pipeline: Sink

Write input dataframe as messages to a specified stream. Each row is encoded as a JSON object. Ensures that subscriber has acknowledged the message.

This bot expects a Restricted CFXQL.

Each parameter may be specified using '=' operator and AND logical operation
Following are the parameters expected for this Bot

Parameter Name Type Default Value Description
name* Text RDA network stream name
timeout Text 10.0 Timeout in seconds for ACK from the subscriber
retries Text 3 Number of retries