Syscall exception when trying to start IcePackNode server

[Nis Baggesen]

I'm currently trying to move our current Ice server into the IcePackNode framework, and in that process I've encountered some problems.

Everything currently runs on a Redhat Linux system. When not using IcePack in any way the server can be launched in a default mode by executing the file gameserver with no arguments. Besed on the IcePack example Simple, I've tried to convert that to the following deployment file:

<icepack>
<variable name="gameserver.dir" value="xxx"/>
<application name="LocationServer">
<node name="node">
<server name="${name}" kind="cpp" exe="${gameserver.dir}/unix/gameserver" pwd="${gameserver.dir}" activation="manual">
<adapters>
<adapter name="Location1" endpoints="tcp:udp:ssl" register="true">
<!-- object identity="${name}"/ -->
</adapter>
</adapters>
</server>
</node>
</application>
</icepack>

Adding it using the administartion tool works fine, but when I try to start the server (again using the administration tool) I get the following trace

[ icepacknode: Server: changed server `LocationServer1' state to `Activating' ]
[ icepacknode: Activator: activating server `LocationServer1'
path = xxx/unix/gameserver
pwd = xxx
args = --Ice.Config=nodedb/node/servers/LocationServer1/config/config --Ice.Default.Locator=IcePack/Locator:default -h 192.168.0.201 -p 12000 --Ice.ServerId=LocationServer1 ]
[ icepacknode: Activator: activated server `LocationServer1' (pid = 10706) ]
[ icepacknode: Server: changed server `LocationServer1' state to `Active' ]
./unix/gameserver: PropertiesI.cpp:559: Ice::SyscallException:
syscall exception: No such file or directory
[ icepacknode: Activator: detected termination of server `LocationServer1' ]
[ icepacknode: Server: changed server `LocationServer1' state to `Deactivating' ]
[ icepacknode: Adapter: server adapter `LocationServer1.Location1' deactivated ]
[ icepacknode: Server: changed server `LocationServer1' state to `Inactive' ]

In particular I'm curious about
./unix/gameserver: PropertiesI.cpp:559: Ice::SyscallException:
syscall exception: No such file or directory

I sure the paths are correct, and I also know that if I make a deliberate error in the executable name I get an error message saying

icepacknode:No such file or directory

So I was wondering where this error comes from. There are no core dump, nor any of the output I expect from the server program, so it does not seem that it has ever gotten as far as actually calling the executable.

The icepacknode configutation file is the same as for the Simple example.

mvh

Nis

[Nis Baggesen]

Just a short addendum.

I have tried starting the executable as normal, but adding a Locator property to the default Ice configuration file, and there by adding the adapters to an IcePackRegistry running as a locator service, and that went witout a hitch.

[benoit]

I suspect the exception is caused by your server not being able to open the configuration file specified by the --Ice.Config command line option.

As you can see from the trace, the path of this configuration file is a relative path and this path is relative to the data directory configured for your IcePack node... However, this doesn't work in your case because your server is not using the same working directory as the IcePack node

You should be able to workaround this problem by using an absolute path for your IcePack node data directory (configured with the IcePack.Node.Data property).

I will look into fixing this for upcoming releases, but in the meantime, let me know if you still have problems after trying this workaround!

Benoit.

[Nis Baggesen]

I tried changing the following properties from :

IcePack.Node.Data=nodedb/node
IcePack.Registry.Data=nodedb/registry

to

IcePack.Node.Data=xxx/nodedb/node
IcePack.Registry.Data=xxx/nodedb/registry

(That is the same basic directories as the server I'm trying to start)

in the configuration file for the IcePackNode.

I further tried to start up the IcePackNode in the same directory as the executable 'gameserver'.

All processes are started by the same user, and that user has permissions to use all the indicated directories.

I still get the same error.

mvh

Nis

[Nis Baggesen]

Again a small addition

I've tried adding a bit more debugging, and I now know that I do get as far as starting my executable, which is an extension of Ice::Application. It gets as far as calling main (obviously), then calls on to app.main(argc,argv) and then fails somewhere before calling the run method.

[benoit]

Are the paths specified by your IcePack.Node.Data and IcePack.Registry.Data properties absolute paths?

In any case, I'll try to reproduce the problem and see if I can provide you a patch. Thanks for reporting this problem!

Benoit.

[Nis Baggesen]

Yes, they are absolute paths. I know the 'xxx' I wrote may have been a little confusing - I just didn't want to write out the entire directory structure. '/xxx' would probably have been clearer. Sorry about that.

[benoit]

Did you re-deploy your application after changing the configuration of your IcePack node? If not, can you try to re-deploy it? I'm unable to reproduce this problem if I use absolute paths for the node and registry as well as for the server working directory.

When you server is activated, the path of the configuration file specified by the --Ice.Config command line option should be an absolute path (you can check this with the IcePack node activator traces).

Benoit.

[Nis Baggesen]

Yes I did redeployed the server. I shut down the IcePackNode, deleted the data directories, restarted the node and redeployed the server.

The full stace was as follows (This time without any editing for privacy :

[qube@localhost Gameserver]$ cd nodedb/
[qube@localhost nodedb]$ rm -R *
[qube@localhost nodedb]$ mkdir node
[qube@localhost nodedb]$ mkdir registry
[qube@localhost nodedb]$ cd ..
[qube@localhost Gameserver]$ icepacknode --Ice.Config=node.conf
icepacknode: warning: administrative endpoints `IcePack.Registry.Admin.Endpoints' enabled
[ icepacknode: ObjectRegistry: added object `IcePack/Admin' ]
[ icepacknode: ObjectRegistry: added object `IcePack/Query' ]
[ icepacknode: AdapterRegistry: added adapter `IcePack.Registry.Internal' ]
[ icepacknode: AdapterRegistry: added adapter `IcePack.Node.node' ]
[ icepacknode: Server: created server `LocationServer1' ]
[ icepacknode: Adapter: created server adapter `LocationServer1.Location1' ]
[ icepacknode: AdapterRegistry: added adapter `LocationServer1.Location1' ]
[ icepacknode: ObjectRegistry: added object `LocationServer1' ]
[ icepacknode: Server: changed server `LocationServer1' state to `Activating' ]
[ icepacknode: Activator: activating server `LocationServer1'
path = /home/qube/runestone/Code/Gameserver/unix/gameserver
pwd = /home/qube/runestone/Code/Gameserver
args = --Ice.Config=/home/qube/runestone/Code/Gameserver/nodedb/node/servers/LocationServer1/config/config --Ice.Default.Locator=IcePack/Locator:default -h 192.168.0.201 -p 12000 --Ice.ServerId=LocationServer1 ]
[ icepacknode: Activator: activated server `LocationServer1' (pid = 17890) ]
[ icepacknode: Server: changed server `LocationServer1' state to `Active' ]
Gameserver - Main - Start
/home/qube/runestone/Code/Gameserver/unix/gameserver: PropertiesI.cpp:559: Ice::SyscallException:
syscall exception: No such file or directory
[ icepacknode: Activator: detected termination of server `LocationServer1' ]
[ icepacknode: Server: changed server `LocationServer1' state to `Deactivating' ]
[ icepacknode: Adapter: server adapter `LocationServer1.Location1' deactivated ]
[ icepacknode: Server: changed server `LocationServer1' state to `Inactive' ]

The configuration file node.conf looks as follows :
#
# The IcePack locator proxy.
#
Ice.Default.Locator=IcePack/Locator:default -p 12000

#
# IcePack registry configuration.
#
IcePack.Registry.Client.Endpoints=default -h 192.168.0.201 -p 12000
# IcePack.Registry.Client.Endpoints=default -p 12000
IcePack.Registry.Server.Endpoints=default
IcePack.Registry.Internal.Endpoints=default
IcePack.Registry.Admin.Endpoints=default
IcePack.Registry.Data=/home/qube/runestone/Code/Gameserver/nodedb/registry
#IcePack.Registry.DynamicRegistration=1

#
# IcePack node configuration.
#
IcePack.Node.Name=node
IcePack.Node.Endpoints=default
IcePack.Node.Data=/home/qube/runestone/Code/Gameserver/nodedb/node
IcePack.Node.CollocateRegistry=1
#IcePack.Node.Output=db

#
# Trace properties.
#
IcePack.Node.Trace.Activator=3
IcePack.Node.Trace.Adapter=2
IcePack.Node.Trace.Server=3
IcePack.Registry.Trace.AdapterRegistry=1
IcePack.Registry.Trace.ObjectRegistry=1

#LocationServer.Endpoints=default -h 192.168.0.201 -p 9041

#Ice.ThreadPool.Server.Size=5
#Ice.ThreadPool.Server.SizeWarn=5
#Ice.ThreadPool.Server.SizeMax=10
#Ice.Trace.Network=1
#Ice.Trace.Protocol=1
#Ice.Warn.Connections=1

All the directories

/home/qube/runestone/Code/Gameserver
/home/qube/runestone/Code/Gameserver/unix
/home/qube/runestone/Code/Gameserver/nodedb/node
/home/qube/runestone/Code/Gameserver/nodedb/registry

exist and the executable is

/home/qube/runestone/Code/Gameserver/unix/gameserver

I run the IcePackNode in the

/home/qube/runestone/Code/Gameserver

directory.

Is there any more tracing, debugmessages or anything that I can enable to find out exactly which directory it is unable to find ?

If I make a deliberate error in either of the properties

IcePack.Node.Data=/home/qube/runestone/Code/Gameserver/nodedb/node
IcePack.Registry.Data=/home/qube/runestone/Code/Gameserver/nodedb/registry

then the IcePackNode wont even start saying :

[qube@localhost Gameserver]$ icepacknode --Ice.Config=node.conf
icepacknode: warning: administrative endpoints `IcePack.Registry.Admin.Endpoints' enabled
[ icepacknode: ObjectRegistry: removed object `IcePack/Admin' ]
[ icepacknode: ObjectRegistry: added object `IcePack/Admin' ]
[ icepacknode: ObjectRegistry: removed object `IcePack/Query' ]
[ icepacknode: ObjectRegistry: added object `IcePack/Query' ]
[ icepacknode: AdapterRegistry: removed adapter `IcePack.Registry.Internal' ]
[ icepacknode: AdapterRegistry: added adapter `IcePack.Registry.Internal' ]
icepacknode: error: property `IcePack.Node.Data' is not set to a valid directory path

so at least form the perspective of icepacknode the path is good enough.

mvh

Nis

[benoit]

Mmm, everything looks right. How do you call Ice::Application::main() in your server? Are you specifying a configuration file (the third argument of the main method)?

You could try to manually start your server from the command line and see if it succeeds:

$ cd /home/qube/runestone/Code/Gameserver
$ /home/qube/runestone/Code/Gameserver/unix/gameserver \
--Ice.Config=/home/qube/runestone/Code/Gameserver/nodedb/node/servers/LocationServer1/config/config \
--Ice.Default.Locator=IcePack/Locator:default -h 192.168.0.201 -p 12000 \
--Ice.ServerId=LocationServer1

This is basically what the IcePack node activator uses to start your server. I'm afraid there's no other tracing properties that could help. If you can easily rebuild Ice, the easiest way to figure out which file can't be found would be to print the name of the file in the PropertiesI::load() method before the SyscallException line 559 in src/Ice/PropertiesI.cpp is raised.

Benoit.

[Nis Baggesen]

Thanks, I finally cleared it up. I hadn't thought about the extra Ice arguments that the node uses to launch the executable and caused my executable to be launched in non-default mode where it did expect a seperate configuration file.

My bad. Sorry for all the inconvinience.

mvh

Nis