Skip to content

Artifacts to Control Data in RDA Fabric

Artifact Type Description
Formatting Templates RDAF uses Jinja2 Templating Engine to format data.
Credentials Credentials to various systems / integrations are stored securely in RDA Fabric Vault
Bookmarks RDA Fabric uses Bookmarks as a way to remember how far each data stream has been read.
Pipelines Pipelines are low code lines of text to manage data.
Service Blueprints Service Blueprints help manage lifecycle of one more pipelines, and related artifacts, collectively as a single service.

1. Formatting Templates

RDAF uses Jinja2 Templating Engine to format data to make it suitable for REST Endpoints, HTML Reports sent via email, and many other formats as needed by various integration endpoints.

Storage Location

RDAF Object Storage

Related Bots

Related RDA Client CLI Commands

    fmt-template-delete       Delete Formatting Template
    fmt-template-get          Get Formatting Template
    fmt-template-list         List Formatting Templates

Managing through RDA Portal

  • In RDA Portal, Click on left menu Configuration
  • Expand Artifacts
  • Click on 'View Details' next to Formatting Templates

Managing through RDA Studio

  • Both RDA Studio & RDA Fabric share this artifact
  • In RDA Studio, Select Task 'Formatting Templates: Edit'

Portal

Examples

2. Credentials

Credentials to various systems / integrations are stored securely in RDA Fabric Vault, encrypted using Fernet that uses 256 bit key. RDAF also uses a unique salt for each encryption.

Storage Location

  • In RDA Fabric / RDA Portal: RDAF Object Storage
  • In Studio: Stores locally inside container or on mounted volume.
  • RDA Fabric & Studio DO NOT share credentials. Each one must be configured separately.

Related RDA Client CLI Commands

    secret-add                Add a new secret to the vault
    secret-list               List names and types of all secrets in vault
    secret-types              List of all available secret types
    check-credentials         Perform credential check for one or more sources on a worker pod

See RDA CLI Guide for installation instructions

Managing through RDA Portal

  • In RDA Portal, Click on left menu Configuration
  • Expand Artifacts
  • Click on 'View Details' next to Credentials

Managing through RDA Studio

  • RDA Studio & RDA Fabric DO NOT share this artifact
  • In RDA Studio, Select Task 'Administration: Manage Bot Sources'

Portal

3. Bookmarks

RDA Fabric uses Bookmarks as a way to remember how far each data stream has been read. It ensures that future data polls / pulls continue with rest of the data after the bookmarked point. Bookmarks typically stores timestamps or document IDs. Bookmarks are only managed via bots.

Storage Location

RDAF Object Storage

Related Bots

4. Pipelines

In RDA, Pipelines are low code lines of text to manage data. Each pipeline is usually built using a series of Bots and Parameters to control the behavior of that bot.

Storage Location * In RDA Fabric / RDA Portal: RDAF Object Storage * In Studio: Stores locally inside docker container or on a mounted volume. * RDA Fabric & Studio DO NOT share pipelines. Using RDA Studio, Pipelines can be published to push pipelines to RDA Fabric.

Related Bots

Related RDA Client CLI Commands

      pipeline-delete           Delete pipeline by name and version
      pipeline-get              Get pipeline by name and version
      pipeline-get-versions     Get versions for the pipeline
      pipeline-list             List published pipelines
      pipeline-publish          Publish the pipeline on a worker pod
      pipeline-published-run    Run a published pipeline on a worker pod
      run                       Run a pipeline on a worker pod
      run-get-output            Run a pipeline on a worker pod, wait for completion and get output
      watch-logs                Start watching logs produced by the pipelines
      watch-traces              Start watching traces produced by the pipelines

See RDA CLI Guide for installation instructions

Managing through RDA Portal

  • In RDA Portal, Click on left menu Configuration
  • Expand Artifacts
  • Click on 'View Details' next to Pipelines
  • Typically pipelines are developed, debugged & verified in RDA Studio and then published to RDA Fabric for execution.
  • Following is an example view of Upcoming RDA Pipeline Builder. Check this page for updates Portal

Managing through RDA Studio

  • RDA Studio & RDA Fabric DO NOT share this artifact
  • In RDA Studio, Select Task 'Pipelines: Add a New Pipeline' or 'Pipelines: Edit a Pipeline'

Portal Portal

Examples

5. Service Blueprints

Service Blueprints help manage life cycle of one more pipelines, and related artifacts, collectively as a single service.

Service Blueprints are created in YAML format.

Storage Location

  • In RDA Fabric / RDA Portal: RDAF Object Storage
  • Not supported in RDA Studio.

Related RDA Client CLI Commands

    deployment-activity       List recent deployment activities
    deployment-add            Add a new Deployment to the repository. 
                              Deployment specification must be in valid YML format
    deployment-audit-report   Display Audit report for a given deployment ID
    deployment-delete         Delete an existing deployment from repository
    deployment-dependencies   List all artifact dependencies used by the deployment
    deployment-disable        Disable an existing deployment if it is not already disabled
    deployment-enable         Enable an existing deployment if it is not already enabled
    deployment-map            Print service map information in JSON format for the given deployment
    deployment-status         Display status of all deployments
    deployment-svcs-status    List current status of all service pipelines in a deployment

See RDA CLI Guide for installation instructions

Note: The term deployment in CLI commands refers to Service Blueprints.

Managing through RDA Portal

  • In RDA Portal, Click on left menu Configuration
  • Expand Artifacts

  • Click on 'View Details' next to Service Blueprints

  • cfxCloud offers a set of pre-built Service Blueprints. In Home screen, Click Add + action next to Services, to import pre-built blueprints.

Managing through RDA Studio

  • Not supported

Examples

See Beginner's Guide: Deploying service blueprint in RDA Portal for an example.