Dear All,
I'm really new to this RTI Connext DDS tools. I've installed the RTI Connext, I've succeed to run the example, but what I need is this case:
Publisher has Matrix data an array 1D of float, and Publisher want to divide and send this matrix to several Subscribers with some QoS, and each Subscriber will process his part of the matrix.
I was expecting this publishing and subscribing data will be simple. But when I tried to look at the example, for me it's complicated.
I hope anyone in this forum could help me.
Thank you,
Regards,
Lutfi A F
See this post regarding load sharing: https://community.rti.com/comment/397#comment-397
(read the entire thread for all the background).
This idl is published by the work engines to announce a) who they are, and b) what their current "load" is. For the demo, I used CPU load. For your purpose, I would simply set a load of 0 or !0 to indicate "I am working" or "I am free to take the next sample".
module demo { module rip { module loadBalance { struct Load { string<128> id; //@key long load; //0 <= Load <= 100 }; }; }; };This idl is your work package, as an example,
module demo { module rip { module loadBalance { struct WorkPacket { // id of worker engine that this sample is targeted at string<128> id; //@key long packetId; sequence<float, 10000> packet; // full on send to work, empty on return sequence<float, 10000> result; // full on return from work, empty on send // any other necessary fields, to set attributes like precision or algorithm to use }; }; }; };The system behavior is now,
Worker starts, sets up a topic filter on TopicWorkPackets, with "id = {myUniqueId}". It publishes a Load instance with the id set to {myUniqueId}, and load==1, announcing it is available.
Matrix application collects Load instances. It chunks the matrix data and sends a chunk to each available worker by publishing a WorkPacket on TopicWorkPackets, making sure to set the id field of the work packet to what is in the Load instance it is processing.
The DataWriter on TopicWorkPackets will only send the outgoing WorkPacket to the single Worker.
The single Worker can publish a Load instance of load==0 to indicate that it is busy.
The single Worker can publish the completed work packet either on TopicWorkPackets or maybe on another dedicated Topic. It then publishes a Load instance load==1 to indicate that it is available again, possibly after doing some housekeeping.
This scales easily -- just start multiple Worker applications on different machines and they will join the ring.
Now I understand the concept, but do you have any simple complete code that work on it? Because even I don't know how to use the IDL. I'm trying to read the hello_idl example, but it looks too much.
Thank you
I can, but because this is 'community' you'll have to be patient, hope someone else can put it together, or struggle through and do the work yourself.
If you want to use DDS, you'll either need to learn the IDL language (it's not that hard, just think "it's a C structure" (or "It's an object" if you are more OOP) and go from there. The rtiddsgen does all the work, creates all the necessary infrastructure from it, but you still have to understand how you got from IDL to structured data.
If you can read XML, there is also a way to generate types using an XML format. There's an XSD for it in the distribution, but I'm an XSD snob and it's kinda clunky. Look for information on how to use it in the Prototyper documentation. You can put the XML defined Types into your QOS file, and the application instantiation code will create the necessary infrastructure on the fly.
I'll be able to look at this no earlier than March, sorry.
Yes, I do do consulting engineering. Yes, I can do a quote. Unfortunately... I still won't have time until March.
Now I'm trying to understand the basic by doing this HowTo's
https://community.rti.com/howto/getting-started-connext-dds-defining-your-data-type-sending-and-receiving-data
After I investigate the code, I'm focusing on understanding how this dds send the data, and where they put the data. There are several questions:
1. From what I understand, I should put the float data of matrix as instance. In the example, it said (in helloworld_publisher):
instance = HelloWorldTypeSupport_create_data_ex(DDS_BOOLEAN_TRUE);
I'm trying to look for what does that function means, but i couldn't find it in generated folder. Where does that function defined? How they generate the data?
From the answer of that question, I hope I could know how to create an instance with WorkPacket and Load struct.
2. On helloworld_subscriber, I saw on main loop, the code only like this:
/* Main loop */
for (count=0; (sample_count == 0) || (count < sample_count); ++count) {
printf("HelloWorld subscriber sleeping for %d sec...\n",
poll_period.sec);
NDDS_Utility_sleep(&poll_period);
}
Where they read the data? Is it on different thread?
Just to make it clear, the code is attached.
You said rtiddsgen does all the work. In this case, as you said, I have two kind of IDL. First is struct Load, second is struct WorkPacket, should I put it in the same file or not? If not, how could I call the rtiddsgen? Because in HowTo's they're just using one HelloWorld.idl file, so the command is:
rtiddsgen -language C -example i86Linux2.6gcc4.4.5 HelloWorld.idl
And I think I cannot use -example parameters because my idl is not in the example. cmiiw
The TypeSupport_create_data and _create_data_ex methods ensure that there is memory space set aside for the instance. Consider it RTI Connext's "alloc()". The _ex(TRUE) variant works deeply, so that a Type that references a sub-Type, dedicated space for the sub-Type is also allocated. If you are managing your own memory, you can use the _ex(FALSE) variant and simply set the sub-Type field to point to your own memory.
In the documentation, a User defined type is handled metasyntactically, ie, using "Foo". cf. https://community.rti.com/static/documentation/connext-dds/5.2.0/doc/api/connext_dds/api_c/structFooTypeSupport.html#details
That page includes doc for both FooTypeSupport_create_data and _create_data_ex, as well as the other methods available to a generated Type.
In the default example application (generated by -example <arch> to rtiddsgen), for the subscriber, it attaches a listener to the DataReader. See the code around the DDS_Subscriber_create_datareader(*, ...) method. The listener runs under the Subscriber entity's internal threading SO DON'T BLOCK IT. For C, an example listener is defined at the top of FooType_subscriber.c file, followed by an implementation. Look for the FooTypeListener_on_data_available(...) method.
Putting Load and WorkPacket in the same IDL, or not, is an architectural design choice. I tend to having one file only, because I'm producing example code that third parties will extend. My example load-balancer (referenced elsewhere on this forum) had a single file. I make no suggestions as to how many you should use, for the most part I have no opinion.
-example simply uses the last struct MyType { ... } in the IDL file being processed, and generates the MyType_publisher.c and MyType_subscriber.c files in addition to all the other Type and Type support .c/.h files. One run of rtiddsgen will only produce one set of _publisher/_subscriber files, so if you have Load and then WorkPacket in the same IDL, you will only get the example application files for WorkPacket They can be a time-saver for boiler-plate code, but are not, strictly speaking, necessary. I used perl files that I'd written that generated subclasses of the generated types, so i'd use my own boilerplate (which followed my coding conventions, etc) and allowed me to regenerate the Type/TypeSupport files from the IDL without having to worry about blowing away my own files.
"Second is struct WorkPacket..." so I would expect to see WorkPacket_publisher.c and WorkPacket_subscriber.c. I find the _subscriber files useful because they generate a working Listener structure, which I can leverage.
If the Load and the WorkPacket do not need each other, then there is no problem with having them in separate files. You'd then
I'm deciding to put the idl (I have 3 of them "Load", "WorkPacket", and "Vector") in one file (LoadBalanceType.idl). I've generated all the files (Plugin, Support, publisher, subscriber, "main source") using rtiddsgen. It's nice to know that every struct in the idl file will have generated function in plugin, support, and "main source" file (i.e. LoadBalanceType.c & LoadBalanceType.h).
Now, in LoadBalanceType_publisher.c, I've generated matrix to be sent to subscriber with type float *bdia_h with size (1024x28). And I will send it row by row to each subscriber (i.e. 28 elements). How could I put this 28 elements into the instance so I could sent it to subscriber?
Just in case you need it, I include full project.
NB:
Btw, I still didn't understand what's this plugin and support file for. Does it called in _publisher and _subscriber file?
I will not look at your files. Couple of reasons, which I won't go in to. Please paste your IDL into your response, highlight it and select "Preformated" from the dropdown that (probably) shows "Paragraph".
The Type is the structure (or java class file, or C++ object file, or ...), called Type, generically.
TypeSupport files implement the DDS API calls which relate to the use of publishing/subscribing to IDL-defined types. The Standard requires things like the create_data(...) call, but which can not be supplied with the distribution, as the distribution can not, a priori, know what you will need as a Type. So they are generated from the IDL. The TypeSupport files are the glue logic between the IDL-generated Type and the generic code in the distribution. Read the standard, and the docs. it's all there.
Dear rip,
Is it possible to create publisher and subscriber in one file? How to do it? Should I use the same participant or different? Could I call publisher function in Listener_on_data_available?
I've tried it, but it give me error: INTERNAL ERROR: illegal operation due to exclusive area problem.
When I read it, the problem is related to wrong management on participant, subscriber, and publisher.
Thank you,
When you create a domain participant (in RTI Connext, other vendors may do this differently), there is a default Publisher and Subscriber created. Just by looking at the preceding information it should be clear that you can produce any number of readers and writers on a single participant.
You want to absolutely minimize the number of DomainParticipants that you create for an application. There are reasons why you would want more than one (one for the data plane, one for the C2 plane for example), but if your application is only on one domain, then your application should only have one participant. Treat this as a rule, and only violate it when you understand why you need to violate it, and document why you violated it.
Do not -- EVER -- attempt to publish something from inside a listener callback. Luckily, Connext won't let you. Treat the callback as if it were an interrupt service routine in an RTOS. Get in, get out and do nothing that will block the thread. If you need to do something like publish A on response to receipt of B, then do the <datareader>.read(...) on your own thread (waitset or polled).