Hi,
I've been doing some performance testing and just wanted some clarification on what the one way latency is. as well as how to interpret the results that I am attaining.
I have run some 10 second latency tests and within the subscriber the last result mention that 451 packets have been sent. Is this number supposed to correspond with the amount of publisher results? There are 459 results for the publisher, so these numbers are quite close but some clarification on what these numbers would be should help greatly.
Am I right to assume that the latency is measured from the publisher to the subscriber and then back and then halved? The results mention the one way latency and this makes me assume that it is 'one way'. It would be fantastic to get some clarification and thank you for your time.
Thanks,
Blitz3r
Hello,
I believe when you say that you have been doing some performance testing you mean with the RTI Perftest application, am I right?
In that case, you are right: We measure the RTT (Round Trip Time), which corresponds to the time it takes for a sample to be sent from the publisher to the subscriber, send it back and be received in the publisher side. We assume that time is symetric (time to go equals to time to receive) and then we divide by 2, getting the Latency (One way latency).
Now, the next question is, what kind of test are you performing? Let me explain, in RTI Perftest by default you do a Throughput test, and you might be interested in doing a latencyTest:
Doing a throughput test means that it will start sending samples as fast as possible, and once every `latencyCount` samples (which for a Throughput test is 10000 samples), it will mark the sample so it is answered by the subscriber. The answer is the exact same sample, so, by getting the time when you sent it and the time when you receive it back you can get the Round-Trip Time (RTT).
The important thing here is that: Doing a Throughput test will give you the maximum throughput (mbps) at what the publisher will be able to send samples to a subscriber. The latency you will get there is there impacted by the fact that you are in a crowed network, with your queues being full. If you want to test the minimum latency you will be able to achieve, you will need to do a Latency Test instead.
In order to do a latency test, you will only need to add `-LatencyTest` to your RTI Perftest publisher command line parameters. This will change the behavior of the test. Now, instead of trying to send as fast as possible, it will do a ping pong test: Send a sample and wait until it receives it back to calculate the RTT (and therefore the Latency), then send again.
Let me know if this helps!
Hi Javier,
Thank you very much for the very descriptive explanation. I really appreciate it.
I am doing latency tests with the following commands:
Publisher: perftest_java.bat -pub -dataLen 4096 -numSubscribers 1 -domain 1 -executionTime 10 -latencyTest
Subscriber: perftest_java.bat -sub -dataLen 4096 -domain 1
I managed to attain these results:
I was just wondering how are the first values for the average, standard deviation, and maximum calculated? Shouldn't the numbers all be the same for the first record? So the average and maximum should be the same number?
For the throughput results here is what I got:
This test was run for 10 seconds as seen from the executionTime parameter. However, there are only 9 rows. Do you know why this might be?
I really appreciate all the help and thank you again for your time!
Blitz3r
- With regards to your first question, you exposed something that can be improved in RTI Perftest, let me show you:
In the code, the first time we receive a sample of a given size, we don't print the latency, we just print the message you see with the datalen chage:
There in the code, internally, we do this:
Hi Javier,
Thank you very very very much for the amazingly descriptive explanation again!
Coming back to my original post, here is an example of some of the tests I have run:
As seen above, the subscriber states that there are 473 cumulative packets sent. However, on the publisher side we notice that there are 518 records (excluding the first row for the headings). Why are these numbers not corresponding? Should they be corresponding? Here are other examples of longer tests where this has occured:
Above we can see that 6232 is not equal to 6210.
Here we see that 24445 is not equal to 24370.
However, there are the rare cases where the numbers do correspond. Thank you again for taking your time to answer my questions.
Blitz3r
Hi,
Let me see if I understand correctly the question: Since you are running a latency test (-latencyTest), you would expect every single sample to be answered, and therefore you would expect to see a row per sample in the publisher side, however you are seeing more than the number of samples being sent.
This is weird, because, as you well thought, I would expect every sample to be received once, and only once.
I was unable to reproduce this behavior, I've been using the following command line parameters:
Publisher
Subscriber
But every single time I was getting 999 lines. I also tried by using the -executionTime, with similar results.
I instrumented the code adding these lines to the "LatencyListener.java" file, right after where we print the "One-Way Latency:"
But in my case, numbers match.
Could you also instrument the code as I did adding these lines and see where you receive more samples? Also, this is not probable, but could you check if in your system you have more RTI Perftest applications running.
What command lines are you using and what version of RTI Perftest?
Regards,
Javi