.. _section-what-is-protobuf-ext:

About |PROTOBUF_EXT|
********************

|RTI_TM_CONNEXT_TM| |PROTOBUF_EXT| enables the use of |PROTOBUF| data types
in |CONNEXT| applications.

|PROTOBUF_EXT| is an experimental feature in |PRO| |CONNEXT_CURRENT_VERSION|.
See :link_connext_dds_pro_whats_new:`Experimental Features <experimental_features.html>` in
:link_connext_dds_pro_whats_new:`RTI Connext What's New <>`.

.. note::

   |PROTOBUF_EXT| is currently supported only on Linux 64-bit Intel architecture.

Use Cases
=========

**Protocol Buffers Definition Language**. |PROTOBUF_EXT| provides support for using
the |PROTOBUF| message definition language to define the data types used by |CONNEXT| applications. 
Developers can specify data types in ``.proto`` files, and the toolchain will automatically map them to 
corresponding |DDS-XTYPES| types, which serve as the foundation for communication over DDS.

.. figure:: static/Protobuf_to_IDL.png
    :alt: Protocol Buffers to |IDL| Conversion
    :name: Figure - Protocol Buffers to |IDL| Conversion
    :align: center
    :figWidth: 100%

**Generation of Equivalent DDS-XTYPES Types**. |PROTOBUF_EXT| generates ``.idl`` files 
containing |DDS-XTYPES| types equivalent to the |PROTOBUF| types defined in ``.proto`` files.
This enables easy integration between those |CONNEXT| applications using the |PROTOBUF|
data types, and those using the equivalent |DDS-XTYPES| types, regardless of programming
language.

**Protocol Buffers C++ Language Binding**. |PROTOBUF_EXT| allows developers
to use the C++ message types generated by the |PROTOBUF| compiler in combination
with |CONNEXT|'s "Modern" C++ API. Applications can create DataWriter and DataReader
endpoints to exchange objects of the |PROTOBUF| types directly over DDS topics.
Existing components that use these types can be easily and efficiently integrated onto 
the DDS databus.

.. figure:: static/Protobuf_Databus.png
    :alt: Protocol Buffers Integration with DDS
    :name: Figure - Protocol Buffers Integration with DDS
    :align: center
    :figWidth: 100%

**Integration with DDS**. The data-types defined using |PROTOBUF| are first-class 
citizens in the |CONNEXT| platform and can be used just like any other DDS types. 
Full support is provided for features such as type discovery, type matching, type evolution, 
dynamic types, dynamic data, content filtering, and more.

The ``.proto`` files can be enriched with custom |PROTOBUF| options defined
to control DDS-specific properties, such as indicating key members. Other options
allow developers to take advantage of |DDS-XTYPES| features not natively available in
standard |PROTOBUF|. This enhances the integration of |PROTOBUF| components with other 
|CONNEXT| applications.

Components
==========

|PROTOBUF_EXT| consists of two plugins for the |PROTOBUF| compiler
(``protoc``), and a set of DDS-specific options for |PROTOBUF| types:

|IDL| Converter Plugin
----------------------

The |IDL| *Converter Plugin* converts ``.proto`` files into equivalent ``.idl`` files:

.. code-block:: shell

    protoc --idl4_out=<outputdir> message.proto


* Example Input (``message.proto``):

  .. code-block:: protobuf

      syntax = "proto3";

      import "other.proto";

      package myapp;

      message MyMessage {
          int32 foo = 1;
          Other bar = 2;
      }

* Example Output (``message.idl``):

  .. code-block:: omg-idl

      #ifndef myapp_message_proto_IDL4_
      #define myapp_message_proto_IDL4_

      #include "other.idl"

      module myapp {
          @mutable
          struct MyMessage {
              @id(1) int32 foo;
              @id(2) @optional Other bar;
          };
      }; // module myapp

      #endif // myapp_message_proto_IDL4_

Section :ref:`section-proto2idl` provides a detailed description of the
mapping between |PROTOBUF| and |IDL| types.

C++ Code-Generator Plugin
-------------------------

The |CONNEXT| *C++ Code-Generator Plugin* is used together with |PROTOBUF|'s
built-in C++ Code Generator to generate C++ classes for each data type, e.g.:

.. code-block:: shell

    protoc --cpp_out=<outputdir> \
            --connext-cpp_out=<outputdir> \
            message.proto

* Example Output (``message.pb.h``, abridged):

  .. code-block:: c++

      #include "other.pb.h"

      // Injected by RTI code generator
      #include "rti/topic/cdr/ProtobufInterpreter.hpp"

      namespace myapp {

      class MyMessage final : public ::google::protobuf::Message {
      public:
          MyMessage();
          ~MyMessage() override;

          bool has_foo() const;
          void clear_foo();
          int32_t foo() const;
        
          bool has_bar() const;
          void clear_bar();
          const ::myapp::Other& bar() const;
          ::myapp::Other* mutable_bar();
          void set_allocated_bar(::myapp::Other* bar);

          // Injected by RTI code generator
          template <typename MessageType>
          friend struct ::rti::topic::interpreter::detail::protobuf_message_access;

      private:
          int32_t foo_;
          ::myapp::Other* bar_;
      }

      } // namespace myapp


.. _section-dds-options-intro:

DDS Options for Protocol Buffers
--------------------------------

Additional options are available to control the |IDL| types derived from 
the |PROTOBUF| definitions. See :ref:`section-dds-options`. These options are defined 
using |PROTOBUF| syntax and they can be applied after importing the associated `.proto` 
file (:ref:`section-omg-dds-descriptor`).

* Example Input (``message.proto``, annotated):

  .. code-block:: protobuf

      syntax = "proto3";
  
      import "other.proto";

      import "omg/dds/descriptor.proto";

      package myapp;

      message MyMessage {
          option (.omg.dds.type) = {
              default_id: DDS_DEFAULT_ID,
              auto_id: HASH
          };

          int32 foo = 1 [ (.omg.dds.member).key = true ];
          Other bar = 2 [ (.omg.dds.member) = {
            optional: true,
            hash_id: "baz"
          }];
      }
  
* Example Output (``message.idl``, customized):

  .. code-block:: omg-idl

      #ifndef myapp_message_proto_IDL4_
      #define myapp_message_proto_IDL4_

      #include "other.idl"
  
      module myapp {
          @mutable
          @autoid(hash)
          struct MyMessage {
              @key int32 foo;
              @hash_id("baz") Other bar;
          };
      }; // module myapp

      #endif // myapp_message_proto_IDL4_