Observations on 3.3 upgrade

[ctennis]

This morning I shut down our facility to do the mass upgrade of everything remaining to Ice 3.3. Here are some things I noticed that may be helpful for you or other people going through the same issues:

1) I migrated the icegrid database to 3.3. This went okay (I had used the upgradeicegrid.py --server-version 3.2.1 trick). However, I had problems with the IceGridGUI client after that (the one from 3.2). I had marshalling exceptions, and it had some random spurious bad data in various fields. This was mitigated by starting to use the 3.3.0 IceGridGUI - so it's not really a bug, just something I suppose we have to be on the lookout for.

As an example, using the IceGridGUI from 3.2.1, t put "3.2.1" into the "Working Directory" area on my Server Properties screen, but when I use IceGridGUI 3.3.0, it puts "3.2.1" into the Ice Version on my server properties screen.

2) Now I'm running IceGridGUI 3.3.0 and I've started up just my IceStorm services at the moment. I have the cursor highlighted on one of them, and the "Build Id/Properties" area on the Server Properties window is going crazy. It's constantly "Retrieving..." and updating a hundred times a second.

I pinpointed this to one of my other servers on another node that was flapping (it was constantly trying to start, but had an error because of the upgrade, so it wasn't able to start). As soon as I disabled this server, the IceGridGUI settled down, and this massive refresh of the Build ID went away. But I thought you might want to be aware of this in case it was something that needed to be fixed.

3) In some of my servers when I was doing the original upgrade, I had to add this property (I think I noted this in another post):

IceBox.Service.IceStorm
IceStormService,33:createIceStorm --Ice.Config="/g4/nodedata/servers/${instance-name}/config/config_IceStorm"

This worked fine. However, when I saved my registry to XML it saved like this:

<server-instance template="IceStorm" instance-name="IceStorm-${node}">
   <properties>
      <property name="IceBox.Service.IceStorm" value="IceStormService,33:createIceStorm --Ice.Config="/g4/nodedata/servers/${instance-name}/config/config_IceStorm""/>
      </properties>
</server-instance>

And it wouldn't let me re-import it via XML due to the extra quotes on the value of the property above. I was able to fix this, but I think it may not be quoting values properly when you save to XML.

4) I got this backtrace in my shell window from IceGridGUI. I just noticed it when I was finishing up, so I don't remember exactly what I did that might have caused it:

Exception in thread "AWT-EventQueue-0" Ice.AlreadyRegisteredException
kindOfObject = "servant"
id = "GinaGrid-DD4DC526-672A-4E43-82EB-C285B31502F4/IceStorm-tc10 -f IceBox.ServiceManager"
at IceInternal.ServantManager.addServant(ServantManag er.java:34)
at Ice.ObjectAdapterI.addFacet(ObjectAdapterI.java:40 5)
at IceGridGUI.SessionKeeper$Session.addCallback(Sessi onKeeper.java:188)
at IceGridGUI.SessionKeeper.addCallback(SessionKeeper .java:1389)
at IceGridGUI.Coordinator.addCallback(Coordinator.jav a:1627)
at IceGridGUI.LiveDeployment.Server.update(Server.jav a:860)
at IceGridGUI.LiveDeployment.Server.<init>(Server.jav a:658)
at IceGridGUI.LiveDeployment.Node.createServer(Node.j ava:840)
at IceGridGUI.LiveDeployment.Node.update(Node.java:40 6)
at IceGridGUI.LiveDeployment.Root.applicationUpdated( Root.java:482)
at IceGridGUI.Coordinator.applicationUpdated(Coordina tor.java:787)
at IceGridGUI.ApplicationObserverI$3.run(ApplicationO bserverI.java:140)
at java.awt.event.InvocationEvent.dispatch(Invocation Event.java:209)
at java.awt.EventQueue.dispatchEvent(EventQueue.java: 461)
at java.awt.EventDispatchThread.pumpOneEventForHierar chy(EventDispatchThread.java:242)
at java.awt.EventDispatchThread.pumpEventsForHierarch y(EventDispatchThread.java:163)
at java.awt.EventDispatchThread.pumpEvents(EventDispa tchThread.java:157)
at java.awt.EventDispatchThread.pumpEvents(EventDispa tchThread.java:149)
at java.awt.EventDispatchThread.run(EventDispatchThre ad.java:110)


5) I had some servers disabled via IceGridGUI. I changed some properties for them and saved it to the registry. When I went back to "Live Deployment" they were not greyed out anymore, they were shown as though they were "enabled". However, when I tried to start them it gave an error "Server did not start: The server is disabled". However, when clicking on the server to enable it, that's not an option. My only option is "disable". And when I do so, it doesn't change status, it stays "enabled".

Restarting the GUI fixed this issue.

That's all I found for now. I'll advise if anything pops up. Again, these were all easily worked around, but I just wanted to make them known in case they were things that needed to be/could be fixed.

Caleb

[bernard]

Hi Caleb,

Thanks for the detailed upgrade report, we'll investigate these issues.

1) is actually expected: the IceGrid administrative interfaces changed between Ice 3.2 and Ice 3.3, so the Ice version of your IceGrid registry and IceGrid Admin tool must be the same.

Maybe we need to update the documentation to make this clearer.

Best regards,
Bernard

[ctennis]

Thanks Bernard.

Everything is up and running well. I had one server making use of ThreadPerConnection that I had to make some changes to, but otherwise I had no real problems.

I thought that if I noted these as I went through the process they may help either expose bugs or be of value to someone else when they do their upgrades if they see the same things.

With regards to using IceGridGUI 3.2 on a 3.3 IceGrid, it makes perfect sense to me that they could be incompatible. It still let me run though, and I'm curious if I tried to submit a change to the XML if it would have caused any issues or not. Potentially adding some logic that keeps the gui from starting with a warning about incompatible versions might be helpful here.

Caleb