IceGrid Admin session throws Ice::ObjectNotExistException after server act. timeout

[n2503v]

Hello, I have a question about the behavior of the IceGrid Admin session.

After server activation times out, any call to the IceGrid Admin object seems to result in Ice::ObjectNotExistException.
Does this makes sense? Is it possible to avoid this?

Thanks,
Alex

Ice-3.4.1, Linux.

Here's the log from the executable which talks to the Registry:

[ 02/28/11 22:50:17.786 pulque/sysadmin: debug: NodeObserverI::updateServer:
node: s
updatedInfo:
id: s.low
state: ActivationTimedOut
pid: 15418
enabled: 1 ]

[sorry, no time stamp here, but the traces are chronological] Outgoing.cpp:424: Ice::ObjectNotExistException:
object does not exist:
identity: `IceGrid/863E5A5B-C0C9-445F-A38A-93457F91E8A7'
facet:
operation: getServerState

[ 02/28/11 22:50:18.132 Outgoing.cpp:424: Ice::ObjectNotExistException:
object does not exist:
identity: `IceGrid/2122DE2A-50B3-4C07-A6E5-4453901F68A7'
facet:
operation: keepAlive ]


As can be seen from the icegridnode trace below, the Admin session does not time out until later, so the problem is not with failure to keep alive:'
-- 02/28/11 22:49:50.120 icegridnode: Session: admin session `pulque/sysadmin-0' keep alive
-- 02/28/11 22:49:57.122 icegridnode: Session: admin session `pulque/sysadmin-0' keep alive
-- 02/28/11 22:50:04.123 icegridnode: Session: admin session `pulque/sysadmin-0' keep alive
-- 02/28/11 22:50:11.125 icegridnode: Session: admin session `pulque/sysadmin-0' keep alive
-- 02/28/11 22:51:10.930 icegridnode: Session: admin session `pulque/sysadmin-0' destroyed

[benoit]

Hi Alex,

Is something perhaps causing the connection to be dropped between your client and the IceGrid registry right after receiving the node observer update indicating the server activation timeout? I would enable network connection tracing to see if the connection is being dropped. If it's dropped, this would explain the Ice::ObjectNotExistException (the session is tight to the connection).

Cheers,
Benoit.

[n2503v]

Thanks Benoit,
your guess was correct: there was a disconnection due to a misconfiguration in timeouts:
the server activation timeout (IceGrid.Node.WaitTime) was shorter than invocation timeout (Ice.Override.Timeout).

This resulted in Ice::TimeoutException which I guess also closes the connection. Is this right?

I've fixed that but I still sometimes get disconnections. I'll try to get to the bottom of this.

Meanwhile, I try to re-establish a session and reset the IceGrid observers. I don't get any errors but the observers don't receive any updates after the re-connections. I'll get more details and probably ask more questions.

Thanks for the support,
Alex

[benoit]

Hi Alex,

How do you reset the IceGrid observers? If you're using bi-directional
connections to receive observer updates, you will need to re-create the observer proxies with an object adapter which is associated to the new connection (the one used to create the session).

Cheers,
Benoit.

[n2503v]

Benoit,

to clarify...
I have one adapter which is activated at the start of the execution.
I create observer objects (once) and add them to the adapter with a UUID.
Then I set the IceGrid observers with the proxy to the observer objects.

After the session is reset, I try to set the observers again using the same proxies
and catch IceGrid::ObserverAlreadyRegisteredException.
This is what I'd expect so I assumed that it's ok.

So do I need to re-create the proxies even though it's the same adapter?

Cheers,
Alex

[benoit]

Is your object adapter listening on some endpoints (you configure the object adapter Endpoints property for example) or is it a bi-directional object adapter (you use connection->setAdapter() to associate the object adapter with a connection)?

Do you register the observers with the setObservers or setObserversByIdentity method?

In any case, while you should be able to re-use the same observers, I don't really recommend it. It's in general best to simply re-create new observer Ice objects (instantiate new servants and register them with a new identity). This ensures your new observers won't potentially receive updates from an old session.

Cheers,
Benoit.

[n2503v]

Is your object adapter listening on some endpoints (you configure the object adapter Endpoints property for example) or is it a bi-directional object adapter (you use connection->setAdapter() to associate the object adapter with a connection)?

object adapter with Endpoints property.

Do you register the observers with the setObservers or setObserversByIdentity method?

setObservers()

In any case, while you should be able to re-use the same observers, I don't really recommend it. It's in general best to simply re-create new observer Ice objects (instantiate new servants and register them with a new identity). This ensures your new observers won't potentially receive updates from an old session.

I see your point. I can change my code around to re-create the observers every time the session is created. I guess in this case they will receive initialization messages every time the observers are set?

Thanks,
Alex

[benoit]

Hi Alex,

Yes, you should get the initialization message when the observers are registered. This allows to catch up with the updates you might not have received while the connection was down.

Cheers,
Benoit.