How to make web integration service as DDS publisher

6 posts / 0 new
Last post
Offline
Joined: 03/13/2019
Posts: 5
How to make web integration service as DDS publisher

Hi, 

I am trying to make web intergation service as DDS publisher in our system.

What I already have at hand is <types> <struct> and <web_integration_service> <applications> definition in XML format in URSER_WEB_INTEGRATION_SERVICE.xml. which is used by the rtiddswebintegration process. And by curl POST/GET, the dynamic way of data send/get using web integration service is confirmed.

As I searched, it looks like I have to write additional .IDL file to make the <types> <struct> description, then use rtiddsgen to generate publisher/subscriber.c file and edit web integration service as publisher. I wonder if is there a more efficient way to realize making web intergation service as DDS publisher?

Thanks!

 

Fernando Garcia's picture
Offline
Joined: 05/18/2011
Posts: 192

Hi Yanyang,

From what I understand, you want Web Integration Service to behave as a DDS Publisher application. That is, you want to Web Integration Service to publish over DDS information that you pass to it via POST HTTP requests.

To do that, you have to add to your Web Integration Service application (under the appropriate <applications>tag) a DataWriter of the corresponding DDS Topic. For example:

<application name="ExampleApplication">
    <domain_participant name="MyParticipant" domain_id="0">
        <register_type name="HelloWorldType" type_ref="HelloWorld"/>
        <topic name="HelloWorldTopic" register_type_ref="HelloWorldType"/>
        <publisher name="MyPublisher">
            <data_writer name="HelloWorldWriter"
                    topic_ref="HelloWorldTopic"/>
        </publisher>
    </domain_participant>
</application>

This would create a DataWriter for HelloWorldTopic, which will be instantiated upon the execution of:

 $ rtiwebintegrationservice -cfgName ExampleApplication

Then you can write and send DDS data through Web Integration Service by doing a POST on the URL corresponding to the HelloWorldWriter DataWriter:

POST /dds/rest1/applications/ExampleApplication/domain_participants/MyParticipant/publishers/MyPublisher/data_writers/HelloWorldWriter HTTP/1.1
Host: <host>[:<port>]
Content-Type: application/dds-web+xml
Content-Length: 101
Cache-Control: no-cache

<data>
    <sender>Fernando</sender>
    <message>Hello World!</message>
    <count>0</count>
</data>

You will find more information on how to send and receive data, please refer to the Tutorials section of the RTI Web Integration Service User's Manual.

Is this the solution you were looking for?

Thanks,
Fernando.

Offline
Joined: 03/13/2019
Posts: 5

Hi Fernando,

Thank you for your reply. However this may not be what I'm looking for because I have already confirmed the steps you mentioned above and it works well by curl POST/GET the data sample. I am afraid that my question was not clear enough, here is more explanation:

The final goal of my work is like: ClientA (non-DDS data) --> Web integration service --> DDS platform ---> Database integration service --> ClientB (postgres database). But before the final product design, currently I am just studying its feasibility to go through the whole process in a dynamic way, using additional publisher/subscriber just for manual test. 

For step: ClientA (non-DDS data) --> Web integration service, it workd fine with no problem as I said, by adding the <struct > in <types> tag and <application> in <web_integration_service> tag in the Web integration service configuration XML file in our system.

For step: Web integration service --> DDS platform, I am not sure it works or not because I checked the DDS logical View in GUI tool rtiadminconsole, domain_participant name can be seen, but no publisher/subscriber was recognized at all even I have already written the publisher/subscriber in its configuration XML file. So this is the question I want to ask. 

For step: DDS platform ---> Database integration service --> ClientB (postgres database), I can follow the step in Chapter 10 of RTI_Database_Integration_Service_GettingStarted.pdf, and on rtiadminconsole, the topic, publisher and subscriber are all recognized in DDS logical View;

Could you please teach me how to realize the step of Web integration service --> DDS platformand even more, how to connect the web integration service and database integration service and make the whole process happen?

Thanks a lot!

Fernando Garcia's picture
Offline
Joined: 05/18/2011
Posts: 192

Hi Yanyang,

I understand. So you want to integrate the information you're publishing using Web Integration Service (as a result of a POST operation from a non-DDS application) into a larger DDS system where you have Database Integration Service running.

Database Integration Service can directly ingest the information that is sent by Web Integration Service. All you need is to configure Database Integration Service to subscribe to the DDS Topic or Topics you are publishing from Web Integration Service. To do that, you will need to configure a subscription in Database Integration Service as follows:

<types>
     <!-- Types you defined for Web Integration Service -->
</types>
<!-- ... -->

<postgresql_connection>
    <!-- ... -->
    <subscriptions>
        <subscription>
            <table_owner>MyUser</table_owner>
            <table_name>Mytable</table_name>
            <domain_id>0</domain_id>
            <topic_name>HelloWorldType</topic_name>
            <type_name>HelloWorldTopic</type_name>
        </subscription>
    </subscriptions>
</postgresql_connection>

Where:

  • table_owner and  table_name configure the table where the information will be stored.
  • domain_id points to the DDS Domain where Web Integration Service is publishing the information.
  • topic_name is the name of the Topic Web Integration Service is publishing (the name of the Topic the DataWriter writes).
  • type_name is the name under which the Type associated with the Topic Web Integration Service is publishing was registered.

However, if want other applications to subscribe to the data sent by Web integration Service; that is, other DDS applications that are not Database Integration Service. You can, as you mentioned create static DDS applications using the type definitions you used in Web Integration Service. You can simply pass the Web Integration Service types to rtiddsgen in XML format (creating a file with just the  <types> part or passing the whole configuration file) and it will generate code for those types -- you don't need to transform the information to IDL because rtiddsgen can already process XML.

Also, if you want to do manual tests as you mentioned, you can use rtiddsspy or RTI Admin Console for Data  Visualization. However, note that if your types (i.e., larger than the default maximum type serialized size setting, which in the case of rtiddsspy is 4096 bytes), you will need to increase the maximum type object serialized size setting as follows:

rtiddsspy -domainId <domainId> -tcMaxSize <SIZE> -printSample

Please, let me know if this helps.

Thanks!
Fernando.

Offline
Joined: 03/13/2019
Posts: 5

Hi Fernando,

Thanks for your great suggestion! After configuring the same type description in both web integration service and database integration service XML files, I confirmed that a simple json data can be sent by web integration service (Post) and subscribed/stored in the database.

When I use RTI admin console or rtiddsspy to check the status, the web integration service still was not recognized as a publisher to the topic; on the contrary, database integration is recognized as a subscriber. I could not understand why is this because I checked in my system, even there are different rti_connext_dds versions: the web integration service uses 5.3.0; database integration service uses 5.3.1; while  RTI admin console or rtiddsspy is using 5.2.3.

And for your reference, error message below was found by rtiddsspy when I send the data by web integration service:

DDS_ServiceQosPolicy_from_presentation_service_kind:ERROR: Failed to get service (unknown kind) DDS_SubscriptionBuiltinTopicDataTransform:ERROR: Failed to get service PRESCstReaderCollator_addSample:!transform DDS_ServiceQosPolicy_from_presentation_service_kind:ERROR: Failed to get service (unknown kind) DDS_PublicationBuiltinTopicDataTransform:ERROR: Failed to get service PRESCstReaderCollator_addSample:!transform

For now, I haven't define any QoS for the sample topic/subscriber yet (the QoS of database is used without any change, I guess it's been defined), which may explain the error above, what do you think?

Thanks and regards.

Yanyang

Offline
Joined: 03/13/2019
Posts: 5

Hi Fernando,

Some updates on this topic. For now it's possible to send json data in structs (top level or nested) or arrays by web integration service (curl POST) to the target database. However, this process is unstable / irreproducible, in other words, sometimes the sent data is stored by database but sometimes not...   

I have used the rtiddsspy (rti_connext_dds.5.3.1) to watch the newly defined DDStopic (the web integration service uses 5.3.0; database integration service uses 5.3.1), and got the status info like:

1553230593.179595  W +N  03F6F94C    MyTopic        MyType

1553230930.689774  R +N  03F6F94C    MyTopic        MyType

1553230593.191043  R +N  03F6F94C    MyTopic        MyType

1553231058.934042  D +N  03F6F94C    MyTopic        MyType

1553231087.559728  D +M  03F6F94C    MyTopic        MyType

Beside rtiddsspy, is there any other way to check whether the sent data is captured by DDS platform and stored by database? Right now I can only confirm the sent data through postgres database terminal.

Will appreciate it a lot if you have some advices!

Thanks and regards,

Yanyang