ICE creates localhost to localhost connections

[nefilim]

Hi

I am trying to diagnose a problem where we notice sporadic high utilization and load spikes on an ICE server application. Interestingly corresponding exactly to these spikes are small bursts in traffic over the lo interface (usually there is virtually no traffic).

Now, I have noticed that the ICE application opens & maintains connections to itself:

java      10213   root   41u     IPv6           18046494                 TCP localhost.localdomain:42569->localhost.localdomain:42568 (ESTABLISHED)
java      10213   root   42u     IPv6           18046495                 TCP localhost.localdomain:42568->localhost.localdomain:42569 (ESTABLISHED)
java      10213   root   50u     IPv6           18046506                 TCP localhost.localdomain:42571->localhost.localdomain:42570 (ESTABLISHED)
java      10213   root   51u     IPv6           18046507                 TCP localhost.localdomain:42570->localhost.localdomain:42571 (ESTABLISHED)
java      10213   root   58u     IPv6           18046515                 TCP localhost.localdomain:42573->localhost.localdomain:42572 (ESTABLISHED)
java      10213   root   59u     IPv6           18046516                 TCP localhost.localdomain:42572->localhost.localdomain:42573 (ESTABLISHED)

Three pairs it looks like. From searching through the forum the only tidbit I learned was that it apparently these are opened for each thread pool?

Here is the relevant config:

EventSystemAdapter.Endpoints=tcp -p 21500 -t 10000
EventSystemAdapter.ThreadPool.Size=15
EventSystemAdapter.ThreadPool.SizeMax=35

EventSystemAdminAdapter.Endpoints=tcp -p 21550 -t 2000
EventSystemAdminAdapter.ThreadPool.Size=5  
EventSystemAdminAdapter.ThreadPool.SizeMax=25

Ice.ThreadPool.Client.Size=5
Ice.ThreadPool.Client.SizeMax=35

Ice.ThreadPool.Server.Size=5
Ice.ThreadPool.Server.SizeMax=35

Could you tell me what these localhost to localhost connections are used for and why there would be this correlation between localhost traffic and high load?

Thanks
Peter

[benoit]

Hi,

Which Ice version do you use? It sounds like you're using an old Ice for Java version (Ice for Java 3.3.0 do not open anymore these socket pairs). These sockets are indeed used by the thread pool implementation. There's traffic on these sockets when new connections are opened/closed but I would expect the traffic to be unnoticeable.

Note that we do not provide free support for old Ice versions on the forums. If you need further help with this matter, I recommend to upgrade to Ice 3.3.0 if not already done.

Cheers,
Benoit.

[nefilim]

Hi Benoit

Indeed, we are using 3.0.1. I'm currently evaluating 3.3.0 for upgrading in the near future (I'll start another topic for this, interesting performance comparisons between the two).

The traffic is very small indeed but the correlation is undeniable:

lo traffic

CPU load

I was just trying to get a better understanding of the ICE internals to help me narrow down the cause of these spikes. From what you are saying it seems that one possibility is an extreme amount of connections being opened/closed.

Cheers
Peter