Round-Trip Latency

[chuckmo]

A benchmark shows that ICE was not as impressive as ZeroC claimed:

Benchmarking the Round-Trip Latency of Various Java-Based Middleware Platforms

ICE 1.5 was about 3 times slower than ORBacus and Java RMI on JRMP. How do you comment on the result?

[matthew]

- The link does not work. I guess you meant to link http://www.lifl.fr/~merle/benchmarking.pdf.
- The version of Ice that Phillipe tested is very old and the current version is much faster.

We document exactly how we tested the performance of Ice here http://www.zeroc.com/performance/. Anyone who wants to can feel free to replicate our testing methodology.

In addition from what I recall this test was testing apples & oranges in some cases -- namely the concurrency models involved in the tests were not the same. The next update to both Ice & Ice E will feature some more fairly significant performance improvements.

[chuckmo]

Matthew,

Thanks for your prompt reply.

Actually I wanna know, from your observation in the benchmark, how the concurrent model was different from each other?

I have projects about .NET and Java integration on hand that will kick off in months and I'm considering ICE for it's performance. So I want to know more about the performance issue.

Thanks again.

[matthew]

Ice 1.5 only supported thread pool model. The benchmark I think was only testing the default concurrency model... which for ORBacus would have been blocking on the client side, and thread per client on the server side. This concurrency model is, in general, faster -- but much less flexible.

I would also point out that latency is only one of many issues you should consider when you talk about performance & evaluation of middleware in general. For example, is latency important for your application? How about throughput? How fast is the middleware in sending data that YOU are interested in? Is bandwidth important? How much CPU is involved in the processing of the data? What is the footprint of the platform?

Then there are a whole host of issues that are not tied to the performance of the developed application, but rather to the overall effectiveness of the platform. Is development time important? Cleanliness of the API? What language support do you need? Platform support. What services are available?

[marc]

This paper has been discussed here before:

http://www.zeroc.com/vbulletin/showthread.php?t=780

As an aside, can you please update your signature as described here?

[chuckmo]

Matthew and Marc,

Thanks very much. The info is very useful.