Exception while async call timeout

[Andrew S]

Hello.

We are trying to make some sort of a sessions (distributed GC), so we made session keeper, that sends "heart beat" to the remote side by calling proxy "refresh" AMI method .
When we kill remote process, or break down the network connection, sometimes (not always!) we see this messages:

IceUtil::Timer::run(): uncaught exception:
TcpTransceiver.cpp:158: Ice::ConnectionLostException:
connection lost: recv() returned zero
Assertion failed: !__os, file OutgoingAsync.cpp, line 76

or

IceUtil::Timer::run(): uncaught exception:
TcpTransceiver.cpp:191: Ice::ConnectionLostException:
connection lost: WSAECONNRESET
Assertion failed: !__os, file OutgoingAsync.cpp, line 76

BTW, we redirect ice logger to ours logging framework, so all other messages goes directly to our log, except this one - this goes to the console.

While investigating this, we have seen, that this happens in such pattern: make async call and before response (we should fill server thread pool, o'coz, to do that) break down the network connection. So async call never completes and never causes exception. It just do nothing!
This behaviour is really disappoints us, may be you can bring some light on this situation? Thanks!

Ice version: 3.3.0
System: Windows XP, 2003, 2008
Compiler: VS 2008


--
Andrew Solodovnikov

[benoit]

Hi Andrew,

Unfortunately, it looks like you found a bug

After a quick investigation, it appears that this can only occur if you configure retry intervals with Ice.RetryIntervals and if the connection to the server is lost during the connection establishment of the retry. Can you confirm that you're setting this property with non zero retry intervals?

We'll look into fixing this ASAP.

Cheers,
Benoit.

[Andrew S]

Hi Andrew,

Unfortunately, it looks like you found a bug

After a quick investigation, it appears that this can only occur if you configure retry intervals with Ice.RetryIntervals and if the connection to the server is lost during the connection establishment of the retry. Can you confirm that you're setting this property with non zero retry intervals?

We'll look into fixing this ASAP.

Cheers,
Benoit.

Benoit, thanks for answer.
Yes, you are right - i set up retry intervals. But (possible - i didn't do any investigation) same things happens when retry intervals are set to -1. We see no any response (or exception) in some situations, f.e., when network goes down... If i will be able to reproduce it - i will let you know about it...

Thanks!

[benoit]

Hi,

I've posted a patch for this bug here. Thanks for the report!

Cheers,
Benoit.

[Andrew S]

Benoit, thanks for your reply and fix! We will try it and report results.

BTW, we found another strange behaviour. We had written special service locator, that tries to establish connection with the remote host (proxy::ice_getConnection). But when the remote server is under heavy load, call to the ice_getConnection just hangs - it waits for infinite time in the Monitor::wait. When we tried to connect to the remote host by telnet - it connects and sends/receives binary data. Very strange.. May be, ice_getConntection not just establish connection, but also make some handshake actions? I tried to examine ICE sources near that point, but without any success...

Thanks again!

[benoit]

Hi Andrew,

Yes, ice_getConnection() establishes the connection and waits for the server to send the connection validation message. It should eventually return once the server accepts the connection and sends this connection validation message. If you suspect a problem with Ice, the best way to investigate it would be to provide us with a small self-compilable test case.

Cheers,
Benoit.