Artifacts to Control Data in RDA Fabric

Artifact Type	Description
Formatting Templates	RDAF uses Jinja2 Templating Engine to format data.
Credentials	Credentials to various systems / integrations are stored securely in RDA Fabric Vault
Bookmarks	RDA Fabric uses Bookmarks as a way to remember how far each data stream has been read.
Pipelines	Pipelines are low code lines of text to manage data.
Service Blueprints	Service Blueprints help manage lifecycle of one more pipelines, and related artifacts, collectively as a single service.

1. Formatting Templates

RDAF uses Jinja2 Templating Engine to format data to make it suitable for REST Endpoints, HTML Reports sent via email, and many other formats as needed by various integration endpoints.

Storage Location

RDAF Object Storage

Related Bots

Related RDA Client CLI Commands

rdac fmt-template: -h

Example Output

Formatting Templates management commands

Following are valid sub-commands for fmt-template:
list                      List Formatting Templates
get                       Get Formatting Template
delete                    Delete Formatting Template

Managing through RDA Portal

In RDA Portal, Click Home Menu -> Configuration -> Rda Administration -> Formatting Templates

Managing through RDA Studio

Both RDA Studio & RDA Fabric share this artifact
In RDA Studio, Select Task 'Formatting Templates: Edit'

Examples

Example Formatting Templates

2. Credentials

Credentials to various systems / integrations are stored securely in RDA Fabric Vault, encrypted using Fernet that uses 256 bit key. RDAF also uses a unique salt for each encryption.

Storage Location

In RDA Fabric / RDA Portal: RDAF Object Storage
In Studio: Stores locally inside container or on mounted volume.
RDA Fabric & Studio DO NOT share credentials. Each one must be configured separately.

Related RDA Client CLI Commands

rdac secret -h

Example Output

Credentials (Secrets) management commands

Following are valid sub-commands for secret:
  types                     List of all available secret types
  add                       Add a new secret to the vault
  delete                    Delete a secret from the vault
  list                      List names and types of all secrets in vault

See RDA CLI Guide for installation instructions

Managing through RDA Portal

In RDA Portal, Click Home Menu -> Configuration -> Rda Integrations -> Credentials

Managing through RDA Studio

RDA Studio & RDA Fabric DO NOT share this artifact
In RDA Studio, Select Task 'Administration: Manage Bot Sources'

3. Bookmarks

RDA Fabric uses Bookmarks as a way to remember how far each data stream has been read. It ensures that future data polls / pulls continue with rest of the data after the bookmarked point. Bookmarks typically stores timestamps or document IDs. Bookmarks are only managed via bots.

Storage Location

RDAF Object Storage

Related Bots

4. Pipelines

In RDA, Pipelines are low code lines of text to manage data. Each pipeline is usually built using a series of Bots and Parameters to control the behavior of that bot.

Storage Location * In RDA Fabric / RDA Portal: RDAF Object Storage * In Studio: Stores locally inside docker container or on a mounted volume. * RDA Fabric & Studio DO NOT share pipelines. Using RDA Studio, Pipelines can be published to push pipelines to RDA Fabric.

Related Bots

Related RDA Client CLI Commands

  pipeline-delete           Delete pipeline by name and version
  pipeline-get              Get pipeline by name and version
  pipeline-get-versions     Get versions for the pipeline
  pipeline-list             List published pipelines
  pipeline-publish          Publish the pipeline on a worker pod
  pipeline-published-run    Run a published pipeline on a worker pod
  pipeline-convert-to-json  Convert all pipelines in folder from yaml to json
  run                       Run a pipeline on a worker pod
  run-get-output            Run a pipeline on a worker pod, wait for completion and get output
  watch-logs                Start watching logs produced by the pipelines
  watch-traces              Start watching traces produced by the pipelines

See RDA CLI Guide for installation instructions

Managing through RDA Portal

Go to Home Menu -> Click Configuration -> Rda Administration -> Click Pipelines -> Published Pipelines
Typically pipelines are developed, debugged & verified in RDA Studio and then published to RDA Fabric for execution.
Following is an example view of RDA Pipeline Builder.

Managing through RDA Studio

RDA Studio & RDA Fabric DO NOT share this artifact
In RDA Studio, Select Task 'Pipelines: Add a New Pipeline' or 'Pipelines: Edit a Pipeline'

Examples

Example Pipelines

5. Service Blueprints

Service Blueprints help manage life cycle of one more pipelines, and related artifacts, collectively as a single service.

Service Blueprints are created in YAML format.

Storage Location

In RDA Fabric / RDA Portal: RDAF Object Storage
Not supported in RDA Studio.

Related RDA Client CLI Commands

    deployment-activity       List recent deployment activities
    deployment-add            Add a new Deployment to the repository. 
                              Deployment specification must be in valid YML format
    deployment-audit-report   Display Audit report for a given deployment ID
    deployment-delete         Delete an existing deployment from repository
    deployment-dependencies   List all artifact dependencies used by the deployment
    deployment-disable        Disable an existing deployment if it is not already disabled
    deployment-enable         Enable an existing deployment if it is not already enabled
    deployment-map            Print service map information in JSON format for the given deployment
    deployment-status         Display status of all deployments
    deployment-svcs-status    List current status of all service pipelines in a deployment

See RDA CLI Guide for installation instructions

Note: The term deployment in CLI commands refers to Service Blueprints.

Managing through RDA Portal

Go to Home Menu -> Click Configuration -> Rda Administration -> Service Blueprints -> Add
cfxCloud offers a set of pre-built Service Blueprints. In Home screen, Click Add + action next to Services, to import pre-built blueprints.

Managing through RDA Studio

Not supported

Examples

See Beginner's Guide: Deploying service blueprint in RDA Portal for an example.