Skip to content

Creating RDA Dashboards

RDA Dashboards can be used to visualize any data at rest in RDA Fabric. Two primary options for data at rest are Datasets and Persistent Streams

1. Dashboard Management

1.1 Grid Layout

RDA Dashboards use a grid to display specified widgets. Grid width is always 12. column parameter of the layout can be between 0 (left most()) to 11 (right most). Grid Height is indefinite. row parameter of layout can start 0 for first row at these top.

Chart

RDA Dashboard uses an automated algorithm to layout the specified widgets in a dashboard.

1.2 YAML Syntax

name: my-dashboard
label: My Dashboard
description: Example dashboard
enabled: true
dashboard_style: tabbed
dashboard_filters:
    time_filter: true
    columns_filter:
     -  id: sys_created_on
        label: Created On
        type: DATETIME
     -  id: assigned_to
        label: Assigned To
        type: TEXT
dashboard_sections:
 -  title: Service Now
    show_filter: true
    widgets:
     -  title: Sample Data
        widget_type: tabular
        min_width: 8
        max_width: 12
        timestamp: sys_created_on
        columns:
            assigned_to: Assigned To
            sys_created_on: Created On
            severity: Severity
        dataset: servicenow_data

Parameter description for RDA Dashboard:

Parameter Name
Mandatory Description
name yes Unique name or ID for the Dashboard. May contain letters, digits and hyphen only.
label yes Label for the dashboard. This is visible to users of the dashboard
description yes Descrption for the dashboard. This is also visible to the users
enabled Specifies if the dashboard is enabled. If not enabled, Dashboard would be not visible to users. Valid values are true or false. Default is false
dashboard_style Dashboards ca be tabbed or sectioned. If set to auto, tabs will be shown only if more than one section is specified under dashboard_sections. Default is auto
dashboard_filters This object defines if and how filtering should be enabled for dashboard. See Filtering section for more details
dashboard_sections yes This is list of objects which specify list of sections, and widgets in each section.

1.3 Adding Dashboard through RDA Portal

Steps

  1. Login to RDA Portal and Click on Configuration from left side menu.
  2. Click on Artifact and then View Details next to Dashboards
  3. Click on Add YAML
  4. Copy paste the required Dashboard YAML and Save.

If there are no errors, and if the Dashboard is enabled, it should be visible under Dashboards menu from left side navigation.

1.3 Adding Dashboard through RDA CLI

Steps:

  1. Make sure you have already rdac.py installed and configured as explained here
  2. Save Dashboard YAML into current directory, for example my_dashboard.yml
  3. Run command rdac.py dashboard add --file ./my_dashboard.yml
  4. If it is successful, you verify that it is properly added using rdac.py dashboard list

2. Widget Types

Following widgets can be used inside RDA Service Blueprints or standalone Dashboards.

2.1. Counter Chart

Chart

#Counter Chart

Counter chart has three variations. Simple Counter, Expression Counter and Group Counter

2.1.1 Simple Counter

Following snippet should be placed under Dashboard section (widgets)

Counter Chart Definition
dashboard_sections:
 -  title: Custom Gateway Metrics 2
    widget_type: custom_counter
    formatter: DescriptiveCountFormatter
    stream: rda_system_worker_trace_summary
    ts_column: timestamp
    duration_hours: 96
    column: num_bot_executions
    agg: sum
    type: int
    widget_id: 02be1e20

Parameter description for custom_counter widget type:

Parameter Name
Mandatory Description
title yes Title for the widget
formatter Optional formatter of numerical data. See Supported Formatters
stream
or
dataset
yes Either dataset name or stream name must be specified. If both specified, dataset takes precedence. Dataset must be a valid dataset in RDAF. stream must refer to a valid Persistent Stream in RDAF.
ts_column yes Name of the timestamp column in data
column yes Name of the column in the data that contains numerical data to perform aggregations
agg Support aggregations are sum,min, max, mean. Default is sum
type Data type. Valid values are int or float
duration_hours This parameter specifies how many hours in the past should the data be fetched. If time_filter is set to true, time filter will override this value.
extra_filter Optional filter. This filter can be used further filter the data besides time based filtering. Must be in CFXQL format.

2.1.2 Expression Counter Chart

Appearance wise, Expression Counter Chart looks same as Simple Counter Chart.

This chart allows for a complex calculation of one more variables derived from the data.

dashboard_sections:
 -  title: Example
    time_filter: true
    widgets:
     -  widget_type: expression_counter
        title: Percent Incidents for Infra Teams
        expression: (infra_tickets/total_tickets) * 100
        height: 2 
        min_width: 4
        max_width: 6
        unit: '%'
        segments:
         -  variable: total_tickets
            stream: demo-data
            ts_column: timestamp
            duration_hours: 96
            column: count
            agg: sum

Parameter description for expression_counter_reports section:

Parameter Name
Mandatory Description
title yes Title for the widget
expression yes Python like expression to compute the final result to be shown in counter chart.
unit Units to be displayed next to the value
segments yes Segments is a list of objects. Each segment defines a variable to be computed.

Each segment may use following parameters:

Parameter Name
Mandatory Description
variable yes Name of the variable to be computed. This variable name may be used in expression.
stream
or
dataset
yes Either dataset name or stream name must be specified. If both specified, dataset takes precedence. Dataset must be a valid dataset in RDAF. stream must refer to a valid Persistent Stream in RDAF.
ts_column yes Name of the timestamp column in data
column yes Name of the column in the data that contains numerical data to perform aggregations
agg Support aggregations are sum,min, max, mean. Default is sum
type Data type. Valid values are int or float
duration_hours This parameter specifies how many hours in the past should the data be fetched. If time_filter is set to true, time filter will override this value.
extra_filter Optional filter. This filter can be used further filter the data besides time based filtering. Must be in CFXQL format.

2.1.3 Group Counter Chart

Chart

Group Counter Chart

Example YAML for this widget:

Group Counter Definition
dashboard_sections:
    -   title: Events Processed
        widget_type: counter
        min_width: 6
        max_width: 12
        formatter: DescriptiveCountFormatter
        stream: demo-log-intel-stats
        ts_column: timestamp
        style:
            color-map:
                ingestion: '#388e3c'
                archived: '#8e24aa'
                splunk: '#d32f2f'
        limit: 5
        duration_hours: 24
        sparkline:
            interval: 4h
        group_by:
            - type
        column: count
        agg: sum
        type: int
        widget_id: da66c98b

Parameter description for counter section:

Parameter Name
Mandatory Description
title yes Title for the widget
formatter Optional formatter of numerical data. See Supported Formatters
stream
or
dataset
yes Either dataset name or stream name must be specified. If both specified, dataset takes precedence. Dataset must be a valid dataset in RDAF. stream must refer to a valid Persistent Stream in RDAF.
ts_column yes Name of the timestamp column in data
column yes Name of the column in the data that contains numerical data to perform aggregations
agg Support aggregations are sum,min, max, mean. Default is sum
type Data type. Valid values are int or float
duration_hours This parameter specifies how many hours in the past should the data be fetched. If time_filter is set to true, time filter will override this value.
extra_filter Optional filter. This filter can be used further filter the data besides time based filtering. Must be in CFXQL format.
sparkline Optional visualization configuration for the grouped counts with default interval as 1D, fill as True and smooth also as True

2.2. Pie Chart

Chart

Pie Chart

Example YAML for this widget:

dashboard_sections:
 -  title: Ocean
    widgets:
     -  title: EC2 Instances by Type
        widget_type: pie_chart
        agg: value_count
        column: InstanceId
        dataset: cfx-aws-ec2-instances
        extra_filter: '`State.Name` is "running"'
        formatter: CountFormatter
        group_by: InstanceType
        type: int

2.3. Tabular Report

Chart

Tabular Report

Example YAML for this widget:

dashboard_sections:
 -  title: Ocean
    widgets:
     -  title: EC2 Instances
        widget_type: tabular
        min_width: 8
        max_width: 12
        columns:
            InstanceId: Instance ID
            InstanceType: Instance Type
            Placement.AvailabilityZone: Availability Zone
            PublicDnsName: Public DNS
            PublicIpAddress: Public IP
            State.Name: State
        dataset: cfx-aws-ec2-instances
        query: '`State.Name` is "running"'

2.4. Pivot Report

Chart

Pivot Report

Example YAML for this widget:

dashboard_sections:
 -  title: Example
    widgets:
    -   title: Pivot Traces By Destination
        widget_type: pivot
        stream: rda_microservice_traces
        extra_filter: request_type != 'are-you-there'
        ts_column: timestamp
        duration_hours: 24
        index_cols:
            -   name: source
                label: source_label
            -   name: request_type
                label: request_type_label
        value_cols: duration
        value_col_sort: ascending
        grp_columns:
            -   name: destination
                label: destination_label
            -   name: subject_type
                label: subject_type_label
        agg_funcs: sum
        post_pivot_filter: duration != 0
        widget_id: 3d7466a3

2.5. Time Series Chart

Chart

Time Series Chart

Example YAML for this widget:

dashboard_sections:
 -  title: Example
    widgets:
     -  widget_type: timeseries
        min_width: 6
        max_width: 12
        height: 8
        chartProperties:
            xAxisLabel: Timestamp
            yAxisLabel: Count
        stream: demo-log-intel-stats
        ts_column: timestamp
        duration_hours: 24
        interval: 15Min
        group_by:
         -  type
        series_spec:
         -  column: count
            label: Number of Messages
            agg: sum
            type: int

2.5.1 Time Series Chart From Multiiple Sources

Chart

Time Series Chart From Multiple Sources

Example YAML for this widget:

dashboard_sections:
 -  title: Ocean
    widgets:
        title: Data Processing Summary 3
        widget_type: timeseries_multisource
        min_width: 6
        max_width: 12
        height: 8
        xAxisLabel: Timestamp
        yAxisLabel: Count
        sources:
            -   source_label: src1
                stream: rda_synthetic_metrics
                ts_column: timestamp
                duration_hours: 24
                interval: 15Min
                extra_filter: metric_name is 'db_slow_queries'
                group_by:
                    - metric_name
                series_spec:
                    -   column: value
                        label: db_slow_queries
                        agg: mean
                        type: int
            -   source_label: src2
                stream: demo-log-intel-stats
                ts_column: timestamp
                duration_hours: 24
                interval: 15Min
                hide_lines: true
                group_by:
                    - type
                series_spec:
                    -   column: count
                        label: Number of Messages
                        agg: sum
                        type: int

2.6. Shaded Chart

Chart

Shaded Chart

Example YAML for this widget:

dashboard_sections:
    -   title: Analytics
        show_filter: true
        widgets:
            -   widget_type: shaded_chart
                title: Shaded Reports
                stream: sample-data-for-regression
                ts_column: timestamp
                baseline_column: baseline
                anomalies_column: anomalies
                predicted_column: predicted
                predicted_anomalies: predicted_anomalies
                upperBound_column: upperBound
                lowerBound_column: lowerBound
                duration_hours: 5000
                synchronized-group: 0
                markers-def:
                    -   message: Now
                        color: '#E53935'
                        timestamp: 1647814186
                    -   message: Tomorrow
                        color: '#E53935'
                        timestamp: 1648937386
                    -   message: Current
                        color: '#E53935'
                        timestamp: 1658355595
                show-markers: true
                downsample: true
                downsample-to-percent: 10
                downsample-limit-rows: 500

2.7. Data Flow Chart

Chart

Data Flow Chart

Example YAML for this widget:

dashboard_sections:
 -  title: Example
    widgets:
     -  widget_type: dataflow
        title: Data Processed by Pipeline
        chartProperties:
            layoutDirection: tb
            userZoomingEnabled: true
            hasToolbar: false
        output:
            formatter: ByteFormatter(1024)
            stream: rda_system_worker_trace_summary
            ts_column: timestamp
            duration_hours: 48
            group_by:
             -  pipeline_name
            column: peak_df_size_kb
            label: Data Size
            agg: sum
            type: float
        input:
            formatter: DescriptiveCountFormatter
            stream: rda_system_worker_trace_summary
            ts_column: timestamp
            duration_hours: 48
            group_by:
             -  pipeline_name
            column: num_bot_executions
            label: Bot Executions
            agg: sum
            type: int

2.8. Bar Chart

Chart

Bar Chart with Vertical Orientation

Example YAML for this widget:

dashboard_sections:
    - title: By Request Type
      widget_type: bar_chart
      stream: rda_microservice_traces
      extra_filter: request_type != 'are-you-there'
      ts_column: timestamp
      chartProperties:
          yAxisLabel: null
          xAxisLabel: Count
          stacked: true
          legendLocation: none
          orientation: vertical
       duration_hours: 24
       group_by:
        - request_type
       column: duration
       agg: value_count
       type: int
       widget_id: f4786acb

If the desired orientation is horizontal bar chart, specify orientation: horizontal under chartProperties

Chart

Bar Chart with Horizontal Orientation

2.9. Multi Bar Chart

Chart

Multi Bar Chart

Example YAML for this widget:

dashboard_sections:
    -   title: Example
        widgets:
            -   title: Traces By Destination
                widget_type: multi_bar_chart
                stream: rda_microservice_traces
                extra_filter: request_type != 'are-you-there'
                ts_column: timestamp
                chartProperties:
                    yAxisLabel: Duration
                    xAxisLabel: Services
                    stacked: true
                    legendLocation: right
                    orientation: vertical
                duration_hours: 24
                group_by:
                    - request_type
                    - destination
                column: duration
                agg: value_count
                type: int

If the desired orientation is horizontal bar chart, specify orientation: horizontal under chartProperties

Chart

Multi Bar Chart with Horizontal Orientation

2.10. Label Widget

Chart

Label Widget with Multiple Variables

Label widgets can be very simple, just to show a HTML Style text:

1
2
3
4
5
6
7
8
dashboard_sections:
 -  title: Example
    widgets:
     -  widget_type: label
        label: '<center><h2>My Report Label</h2></center>'
        min_width: 6
        max_width: 12
        height: 1

Or, label widgets can be used to display a complex message with data derived from a dataset or a Persistent Stream:

dashboard_sections:
 -  title: Example
    widgets:
     -  widget_type: label
        label: '<h3><font style="color: #ffffff;"><table border=0>
            <tr><td width="50%" align="left" rowspan=2><b>Ingested</b></td><td>{{ "{:,}".format(total | int) }}</td></tr>
            <tr><td>{{ "{:,.2f}".format(total_bytes/(1024*1024*1024)) }} GB</td></tr>
            <tr><td height="20px" colspan=2></td></tr>
            <tr><td width="50%" align="left" rowspan=2><b>Archived</b></td><td>{{ "{:,}".format(archived | int) }} ({{ "{:.1f}".format(archived*100.0/total) }}%)</td></tr>
            <tr><td>{{ "{:,.2f}".format(archived_bytes/(1024*1024*1024)) }} GB</td></tr>
            <tr><td height="20px" colspan=2></td></tr>
            <tr><td width="50%" align="left" rowspan=2><b>Routed to <br/>Splunk</b></td><td>{{ "{:,}".format(splunk | int) }} ({{ "{:.1f}".format(splunk*100.0/total) }}%)</td></tr>
            <tr><td>{{ "{:,.2f}".format(splunk_bytes/(1024*1024*1024)) }} GB</td></tr>
            <tr><td height="20px" colspan=2></td></tr>
            <tr><td width="50%" align="left" rowspan=1>Cost Savings*</td><td> {{ "${:,.0f}".format( (total_bytes-splunk_bytes)*1500/(1024*1024*1024)) }} </td></tr>
            <tr><td height="20px" colspan=2></td></tr>
            <tr><td colspan=2><h5><font color="#cccccc;">* Estimate using Annualized Enterprise Contract</font></h5></td></tr>
            </table></font></h3 >'
        min_width: 4
        max_width: 4
        height: 4
        style:
        backgroundColor: '#1976d2'
        color: '#ffffff'
        segments:
         -  variable: archived
            agg: sum
            type: int
            stream: demo-log-intel-stats
            ts_column: timestamp
            extra_filter: type == 'archived'
            duration_hours: 250
            column: count
         -  variable: archived_bytes
            agg: sum
            type: int
            stream: demo-log-intel-stats
            ts_column: timestamp
            extra_filter: type == 'archived'
            duration_hours: 250
            column: byte_count
         -  variable: splunk
            agg: sum
            type: int
            stream: demo-log-intel-stats
            ts_column: timestamp
            extra_filter: type == 'splunk'
            duration_hours: 250
            column: count
         -  variable: splunk_bytes
            agg: sum
            type: int
            stream: demo-log-intel-stats
            ts_column: timestamp
            extra_filter: type == 'splunk'
            duration_hours: 250
            column: byte_count
         -  variable: total
            extra_filter: type == 'ingestion'
            stream: demo-log-intel-stats
            ts_column: timestamp
            duration_hours: 250
            column: count
            agg: sum
            type: int
         -  variable: total_bytes
            extra_filter: type == 'ingestion'
            stream: demo-log-intel-stats
            ts_column: timestamp
            duration_hours: 250
            column: byte_count
            agg: sum
            type: int

2.11. Image Widget

Image widgets can show any Image that is accessible through a URL (such as logo)

Example YAML definition for Image Widget

1
2
3
4
5
6
7
8
dashboard_sections:
 -  title: Ocean
    widgets:
     -  widget_type: image
        max_width: 6
        min_width: 4
        height: 6
        imageUrl: https://images.unsplash.com/photo-1657883509333-0b1d2ee75853?ixlib=rb-1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=704&q=80

3. More Information

3.1. Supported Formatters

Formatter Name
Description
CountFormatter Formats integer values as K for thousands, M for Millions. If the input value is already a counted as thousands, optional value multiplier can be specified. Example CountFormatter(1000)
DescriptiveCountFormatter Same as CountFormatter but uses descriptive words like Thousand and Million
ByteFormatter Formats data into Kilobytes, Megabytes or Gigabytes. Optional multipler can specified if the input data is already in Kilobytes ByteFormatter(1024)