Does icegrid app remove, stop the servers of the app?

[Wilko]

Hi,

for reregistration of our icegrid applications we start icegridadmin command line with -e option from the rpm postinstall and preremove-scripts.

postinstall

icegridadmin -u yy -p xx --Ice.Config=conf/iceadmin-config.properties -e "application add apps/sysnodeagent/grid-config.xml"

preremove

icegridadmin -u yy -p xx --Ice.Config=conf/iceadmin-config.properties -e "application remove app_sysnodeagent"

During preremove step the applications are usually running and need to be stopped. Does icegrid stop the servers of an application during remove or do we have to stop the server before the remove step?

Unregister currently takes a long time for running processes but the application is successfuly unregistered after ~4min (the application processes remain running - this might be an error in the applications that requires a kill -9).

Best Regards,
Wilko

-------------------------------
Wilko Kempa
sd&m AG
Common Service Plattform

[benoit]

Hi Wilko,

The "application remove" command always stops the server before removing them. It sounds like your servers don't shutdown in a timely manner and this is causing application remove to block for some time.

The IceGrid node does the following to shutdown a server:

• if a process object is registered for your server, the IceGrid node invokes the shutdown method in the server process object (see "32.17.6 Process Registration" here for more information on process objects).
• if no process object is registered for the sever, the IceGrid node sends the SIGTERM signal to the process (on Windows, it sends a Ctrl-Break event).

Then, the node waits for the process to exit or for the server deactivation timeout to expire. If the server deactivation timeout expires, the node kills the server. You should be able to see this by activating IceGrid node activation tracing with the IceGrid.Node.Trace.Activator=2 property set in the node configuration file.

To avoid the delays when removing the application you have the following options:

• Ensure your servers always stop in a timely manner.
• Reduce the deactivation timeout for servers which are never expected to shutdown in a timely manner (you can configure this with the deactivation-timeout attribute in the server descriptor, see the IceGrid XML descriptor reference for more information)
• Explicitly kill the servers before issuing the "application remove" command. You can kill a server with the "server signal <serverid> SIGKILL" command.

Let us know if this doesn't help solving your problem!

Cheers,
Benoit.

[Wilko]

Hi Benoit,

after your advice to use Process to stop our applications i checked our configs. It seems we were missing the application site config of the process configuration (Ice.ServerId, ManagedNodeAdapter.RegisterProcess=1).

Now if i start the application manuall from my development environment the server state remains inactive(enabled). I guess this is because icegrid requires the pid/handle to the application so it requires to start the application itself via exe? Is this correct or is there something wrong with my configuration?

On the registration side we specify

<icegrid>
    <application name="app_xxx">
        <node name="mediaserver">
            <server id="srv_xxx"
                    exe="apps/xxx/run.sh"
                    user="xxx"
                    activation="manual">
                <adapter name="ManagedNodeAdapter"
                         id="ManagedNodeAdapterAtxxx"
                         register-process="true"
                         endpoints="tcp"/>
            </server>
        </node>
    </application>
</icegrid>

This already tells the registry to expect a process for the server via the given adapter.

At the application configuration we need to configure Ice.ServerId and also ManagedNodeAdapter.RegisterProcess=1

Ice.Default.Locator=IceGrid/Locator:tcp -h localhost -p 12000

# Object Adapter Config
ManagedNodeAdapter.AdapterId=ManagedNodeAdapterAtxxx
ManagedNodeAdapter.ReplicaGroupId=
ManagedNodeAdapter.Endpoints=default
ManagedNodeAdapter.ThreadPool.Size=10
# this should match server id attribute from deployment descriptor
Ice.ServerId=srv_xxx

# this indicates the communicator should register shutdown process.
ManagedNodeAdapter.RegisterProcess=1

The strange thing even with IceGrid.Node.Trace.Activator=2
IceGrid.Node.Trace.Server=2 enabled. There seems no contact between the application and the grid. Grid comes up and set all servers to inactive (manual start for debugging).

Thanks in advance
Wilko

[benoit]

Hi Wilko,

You're correct -- the node only cares about servers it starts so the behavior you're seeing is expected. If you start manually a server managed the IceGrid node, the node won't monitor the process.

There will be some interactions between the node and the server however, you should be able to see it if you enable adapter tracing on the node with IceGrid.Node.Trace.Adapter=2. You should see that the server still registers its object adapter endpoints with the node to make it possible for clients to resolve indirect proxies from the server object adapters even if it's started manually.

Let me know if you need more information on this!

Cheers,
Benoit.

[bernard]

Hi Wilko,

When you deploy a server with IceGrid (typically a server hosting indirect object adapters), IceGrid generates the Ice configuration for this server from your server descriptor. And then when IceGrid starts such a server, it passes --Ice.Config=<path to generated config file> as a command-line argument to the server (exe) your registered. You should not write your own regular Ice config file for such server.

You can enter this "server descriptor" either in an XML file processed by icegridadmin or with the IceGrid Admin GUI. You can even create such server descriptor programmatically, by writing a program/script that calls the IceGrid::Admin interface.

We do not recommend to edit the Ice configuration files generated by IceGrid for your servers; if you want to set an additional property in a server, you should edit the source XML file or use the IceGrid Admin GUI.

IceGrid, through its IceGrid nodes, monitors only the servers it starts. If you start a server on the command line (./myServer --Ice.Config=.../myServer.conf), IceGrid won't detect that the server is running even if you set everything right in myServer.conf (ServerId etc.).

Note that the activation mode "manual" means "the server is started through an explicit IceGrid admin command" -- it does mean "the server will be started on the command-line".
That's in contrast to the "on-demand" activation mode, where IceGrid starts automatically a server when a client looks for an object adapter in that server (while resolving an indirect proxy).

I hope this is clearer now!

Best regards,
Bernard

[Wilko]

Hi Bernoit and Bernard,

thanks for the information. So I think we will switch to xml-config only. Is there a DTD or Schema that we can use? I've seen the description at chapter 38.15. but tool based validation and editor support - always helps.

Best Regards
Wilko

[bernard]

Hi Wilko,

We don't have a schema or DTD for the IceGrid XML; it's parsed and validated by icegridadmin.

There is however a convenient tool to create syntactically correct IceGrid XML files: the IceGrid Admin GUI.

If you did not try it yet, you can launch it on Windows by double-clicking on IceGridGUI.jar (in the bin directory). On Linux with a RPM installation, you can launch it with:
$ java -jar /usr/lib/Ice-3.2.0/IceGridGUI.jar

Make sure to use Sun JDK/JRE >= 1.4.2 and not the the GCJ JVM.

Best regards,
Bernard