.. include:: /../getting_started/vars.rst
.. _section-components:
Components
**********
*Connext Observability Framework* consists of three RTI components:
- *RTI Monitoring Library 2.0* enables you to instrument a |Connext|
application to forward telemetry data. The library also accepts
remote commands to change the set of forwarded telemetry data at
runtime.
- *RTI Observability Collector Service* scalably collects telemetry
data from multiple |Connext| applications and stores this data in a
third-party observability backend. This component can also be configured
to forward telemetry data to an OpenTelemetry Collector to allow integration
with other third-party observability backends.
- *RTI Observability Dashboards* enable you to visualize and alert based
on the |Connext| application metrics, as well as display |Connext| log
messages.
*Observability Framework* requires third-party components for
storing and visualizing telemetry data. This release provides native
integration with Prometheus for metrics storage, Grafana Loki for logs storage,
and Grafana for visualization. Integration with other third-party
components is also possible when using `OpenTelemetry
`_ and the `OpenTelemetry Collector
`_.
*RTI Observability Dashboards* are provided as a set of Grafana
dashboards to be deployed on a Grafana server. These dashboards only work with
the Prometheus and Grafana Loki backends. Future releases could support other
backends.
:numref:`FigureFramework` shows a simple representation of how
*Observability Framework* components work together.
.. figure:: static/rti_observability_framework.png
:alt: The RTI Observability Framework
:name: FigureFramework
:align: center
:figwidth: 100%
Observability Framework Components
.. _section-library:
Monitoring Library 2.0
======================
*Monitoring Library 2.0* includes the following key features:
- Collection and forwarding of |Connext| metrics and logs. Logs for DDS Security
events will be supported in future releases.
- Configuration using a new :link_connext_dds_pro_um:`MONITORING QosPolicy (DDS Extension) <#users_manual/MONITORING_QosPolicy.htm>`.
The QoS policy can be set programmatically or via XML.
- Runtime changes to the set of generated and forwarded telemetry data using
remote commands from *Observability Collector Service*. In this release, the
library only allows you to change the forwarding and generation log verbosity
levels for a |Connext| application.
- Ability to enable and disable use of *Monitoring Library 2.0* at
runtime by changing the Monitoring QoS policy.
- Lower overhead as compared to using the
:link_connext_dds_pro_um:`RTI Monitoring Library<#users_manual/PartMonitoringLib.htm>`.
.. _section-collector-service:
Observability Collector Service
===============================
*Observability Collector Service* scalably collects telemetry data
forwarded by *Monitoring Library 2.0* in a |Connext| application.
*Collector Service* is distributed as a Docker™ image and can work in two
modes:
- **Storage**: *Collector Service* sends the telemetry data for storage to
third-party observability backends. This release provides native
integration with Prometheus for metrics and Grafana Loki for logs.
Integration with other third-party components is also possible using
OpenTelemetry and the OpenTelemetry Collector.
- **Forwarder**: *Collector Service* forwards the telemetry data from a
|Connext| application to other collector instances. This mode is not
supported in the current release.
*Observability Collector Service* includes the following key features:
- Collecting and filtering telemetry data forwarded by |Connext| applications
(using *Monitoring Library 2.0*) or other collectors. This release does
not provide filtering capabilities.
- Sending telemetry data for storage to Prometheus for metrics and Grafana Loki
for logs.
- Ability to send telemetry data to an OpenTelemetry Collector using the
OpenTelemetry protocol (OTLP). This feature enables integration with
third-party observability backends other than Prometheus and Grafana Loki.
- Remote command forwarding from *Observability Dashboards* to the
|Connext| applications and other resources to which the commands are
directed. This release only allows forwarding commands that change
the logging verbosity of |Connext| applications. Future releases will
support additional commands.
Storage Components
------------------
*Observability Collector Service* includes native integration with Prometheus
and Grafana Loki to store metrics and logs, respectively.
.. figure:: static/native_integration.png
:alt: Native Integration
:name: FigureNative
:align: center
:figwidth: 100%
Native Integration
This release also allows integrating with other third-party storage
components using OpenTelemetry and the OpenTelemetry Collector.
.. figure:: static/opentelemetry_integration.png
:alt: OpenTelemetry Integration
:name: FigureOpenTelemetry
:align: center
:figwidth: 100%
OpenTelemetry Integration
Observability Dashboards
========================
A set of hierarchical Grafana dashboards sends alerts when a problem
occurs and provides visualizations to help perform root cause analysis.
The dashboards get the telemetry data from a Prometheus server and
the logs from a Grafana Loki server.
The first layer of the Grafana dashboards provides a health status
summary focused on five golden signals: Bandwidth, Saturation, Data
Loss, System Errors, and Delays.
The top-level dashboard also provides access to the system logs and
indicates the number of entities running in the system. To get
additional details on error conditions, select any of the golden
signals displaying an error.
.. figure:: static/dashboard_delays_error.png
:alt: Dashboard delay error
:name: delay_error
:align: center
:figwidth: 100%
How We Provide the Components
=============================
This section describes how *Observability Framework* components
are provided in the current release and how RTI will provide them
in future releases.
Monitoring Library 2.0
----------------------
*Monitoring Library 2.0* is provided as a shared and static library called
*rtimonitoring2*. For details on how to use the library, refer to
:ref:`section-monitoring-library-2`.
Collection, Storage, and Visualization Components
-------------------------------------------------
Current Release
^^^^^^^^^^^^^^^
.. _section-docker-compose-prepackaged:
Docker Compose (Prepackaged)
""""""""""""""""""""""""""""
The *Observability Framework* package enables you to deploy and run
*Observability Collector Service* and third-party components Prometheus,
Grafana Loki, Grafana, OpenTelemetry Collector (optional),
and NGINX (optional) using Docker Compose™ in a single Linux® host.
For details, see :ref:`section-docker-compose-env`.
RTI's prepackaged Docker Compose installation option facilitates
initial product evaluation because it does not require you to deploy all these
components individually.
*Observability Framework* can be deployed with or without using the
OpenTelemetry Collector. Both deployment options can be configured to be secure
or non-secure and to work on a LAN or WAN.
:numref:`FigureDeploymentPromLoki` shows the secure *Observability Framework*
deployment without OpenTelemetry Collector. The deployment uses
Prometheus and Grafana Loki to store metrics and logs, respectively.
.. figure:: static/rti_security_prom_loki.png
:alt: RTI Observability Framework when using Prometheus and Grafana Loki
:name: FigureDeploymentPromLoki
:align: center
:figwidth: 100%
RTI Observability Framework without OpenTelemetry Collector
:numref:`FigureDeploymentOTelPromLoki` shows the secure
*Observability Framework* deployment using OpenTelemetry Collector. The
deployment uses OpenTelemetry Collector to store metrics and logs in Prometheus
and Grafana Loki to store metrics and logs, respectively.
.. figure:: static/rti_security_otlp_prom_loki.png
:alt: RTI Observability Framework when using OpenTelemetry Collector
:name: FigureDeploymentOTelPromLoki
:align: center
:figwidth: 100%
RTI Observability Framework with OpenTelemetry Collector
For additional information on how to use Docker Compose™ to run *Observability
Framework*, see :ref:`section-install-components`.
.. _section-docker-compose-separate-deployment:
Docker (Separate Deployment)
""""""""""""""""""""""""""""
As an alternative to the prepackaged Docker Compose provided
by RTI, you can also run *Observability Framework* components standalone.
*Observability Collector Service* is distributed as a Docker
image hosted in `Dockerhub `_.
This is the same publicly available image used by the prepackaged Docker
Compose installation, and it requires a valid RTI license to run.
The Docker image included with *Collector Service* contains a built-in
configuration that enables it to run in :ref:`storage mode`
with the following operation modes:
.. list-table:: Docker Container Operation Modes
:name: TableDockerContainerOperationModes
:widths: 30 20 50 10
:header-rows: 1
* - Configuration Name
- Network
- Data Storage
- Security
* - NonSecureLAN
- LAN
- Prometheus and Grafana Loki
- No
* - NonSecureWAN
- WAN
- Prometheus and Grafana Loki
- No
* - SecureLAN
- LAN
- Prometheus and Grafana Loki
- Yes
* - SecureWAN
- WAN
- Prometheus and Grafana Loki
- Yes
* - NonSecureOTelLAN
- LAN
- Multiple through OpenTelemetry Collector
- No
* - NonSecureOTelWAN
- WAN
- Multiple through OpenTelemetry Collector
- No
* - SecureOTelLAN
- LAN
- Multiple through OpenTelemetry Collector
- Yes
* - SecureOTelWAN
- WAN
- Multiple through OpenTelemetry Collector
- Yes
For additional information on how to use the Docker image included with
*Collector Service*, refer to `Docker's Collector Service article `_.
The third-party components `Prometheus `_,
`Grafana Loki `_,
`Grafana `_,
`OpenTelemetry Collector `__ (Optional),
and `NGINX `_ (Optional)
are also distributed as Docker images by their respective vendors. You can
use these images standalone instead of
RTI's :ref:`prepackaged Docker Compose`.
Future releases
^^^^^^^^^^^^^^^
Executable
""""""""""
In future releases, *Collector Service* will be provided as
a standalone executable without using Docker to deploy.
Kubernetes
""""""""""
In future releases, the Docker images for *Collector Service*, Prometheus,
Grafana Loki, and Grafana can be deployed on an orchestrated platform
such as Kubernetes. RTI will provide example deployment configurations
for these deployments when they are made available.