specific design question

[sylvain]

Hi,

I have a question regarding the design of an application with Ice...
I need to first introduce a bit the context in which I am asking the question.

Sometime before I knew Ice, I wrote a kind of "devlopement environnement" for my own use. It consists of a program that create c++ codes from definition files (a little bit like Ice but much more simpler). The code created is mainly class definitions that contains almost only data (from type and name found in the definition file). Using these class definitions together with a library it becomes easy to handle object contaning data in a multithreaded environnement and to create persistent images of the objects in a database (mysql). Objects can be created and destroyed, and each object can be loaded from the database using a unique id. Loading and saving is automated, and access to the data is synchronized object by object.

For example, and to be quick, if I write a definition file containing the definition of a class named "CThing" containing a variable of type string named "name", I can write the following snippet of code:"
{
CThingP athing(12);
athing->name="Blue thing";
}

At the creating of the object "athing", the object number 12 will be loaded from database (if not already in a memory cache) and a mutex will be acquired for that object. Then the member variable "name" will be assigned. At the exit of the scope, athing is destroyed, saving the value of "name" in the database and releasing the mutex.

Now I would like to add a touch if Ice in all that. I would like a client program to access the objects CThings living on a server. Basically, I would like the client to be able to get and set the value of the variable "name" of the object CThing number 12 from a server program running on another machine.

It seems to me that I have mainly to approches:

1) I can drop my whole system and making the CThing objects Ice objects using the correct slice definition, instanciating one Ice object for each entry in the database and making these objects avalaible to client, for example through stringified proxies containing the id number. I will also need to write code for setting and getting the "name" variable through Ice operation that can handle the database stuff (or use Freeze, I don't know yet...).

2) I can keep my system, and create an Ice object that is some kind of "accessors": Let say a client can get a proxy "Things" to a unique Ice object of type "CThingAccessor" that is instanciated on the server, it can then do

Things->get_name(12,the_name);

The operation "get_name" of the Ice Object CThingAccessor will basically execute the code above and return the value of "name" to "the_name". Of course the benefit of this solution is that I can keep a lot of code already using succesfuly my system. But are there big drawbacks?

What do you think is the best approach in the sens of using Ice at its best, and to create the most "professional" application?

If a get really a lot of entries in my database, is there a problem to instanciate a lot of small objects for the whole time of the server activity?

In the first approach if I want to get the name of all the CThings objects, I need to do a lot of server requests, but in the second approach I can add a "get_all_name" operation that return a sequence of string containing all the value at once...

In the second approach, It seems that there is too much synchronizations: one for the object ThingAccessor and one for the CThings, is it right?

In the fist approach, can a safely and quickly get the correct proxy for the correct id (I need to keep the unique relation object-id).

Thank you very much in advance for any hint.

[ganzuoni]

Hi,
I preferer a mix of the two approaches, that preserves the existing
system, adding an Ice layer in front of it.
You can have a Things object that gives you operations such as:

Iterator getAllNames()

together with a "real" Thing Ice object with setter and getter.
Things object would be equipped with a

Thing *getById(..)

that gives you a proxy to a Thing object with a particular id.

On the server side the whole system could be represented by a single Ice Thing object, say IceThing,
impersonating different implementation Thing object.
If you decidee to have an entire ObjectAdapter devoted to Thing objects, you could have a
single instance of IceThing (Servant) with a default ServantLocator returning always that instance.
In turn, IceThing can select the right implementation for a particular request using the Ice object id
of the Current argument of IceThing methods.
It is worth to note that you can create, on the server side, the proxy without accessing the real
object with:
adapter.createProxy(Ice.Util.stringToIdentity(id)) ;
where adapter is the ObjectAdapter where IceThing resides and id is the implementation Thing id.
You could also consider the possibility to use category part of Ice object identity to
represent Thing class.
In this way the Ice identity of a Thing implementation object, say 35, would be:
"35/thing"

Then, the ServantLocator would be registered, in the ObjectAdapter, for category "thing".

A final word on performances.
It depends on the use you make of Thing object, but I think that IceThing object should have
an accessor the gives you all the attributes in one call. Don't forget that, normally, each
call on the proxy goes remote. If Thing has a lot of attributes.....
Hope it helps,
Guido.

[marc]

Originally posted by sylvain
Hi,

I have a question regarding the design of an application with Ice...
I need to first introduce a bit the context in which I am asking the question.

Sometime before I knew Ice, I wrote a kind of "devlopement environnement" for my own use. It consists of a program that create c++ codes from definition files (a little bit like Ice but much more simpler). The code created is mainly class definitions that contains almost only data (from type and name found in the definition file). Using these class definitions together with a library it becomes easy to handle object contaning data in a multithreaded environnement and to create persistent images of the objects in a database (mysql). Objects can be created and destroyed, and each object can be loaded from the database using a unique id. Loading and saving is automated, and access to the data is synchronized object by object.

For example, and to be quick, if I write a definition file containing the definition of a class named "CThing" containing a variable of type string named "name", I can write the following snippet of code:"
{
CThingP athing(12);
athing->name="Blue thing";
}

At the creating of the object "athing", the object number 12 will be loaded from database (if not already in a memory cache) and a mutex will be acquired for that object. Then the member variable "name" will be assigned. At the exit of the scope, athing is destroyed, saving the value of "name" in the database and releasing the mutex.

Sounds a lot like what Freeze is doing.

Originally posted by sylvain


Now I would like to add a touch if Ice in all that. I would like a client program to access the objects CThings living on a server. Basically, I would like the client to be able to get and set the value of the variable "name" of the object CThing number 12 from a server program running on another machine.

It seems to me that I have mainly to approches:

1) I can drop my whole system and making the CThing objects Ice objects using the correct slice definition, instanciating one Ice object for each entry in the database and making these objects avalaible to client, for example through stringified proxies containing the id number. I will also need to write code for setting and getting the "name" variable through Ice operation that can handle the database stuff (or use Freeze, I don't know yet...).

If you go this route, Freeze will do all this for you.

Originally posted by sylvain


2) I can keep my system, and create an Ice object that is some kind of "accessors": Let say a client can get a proxy "Things" to a unique Ice object of type "CThingAccessor" that is instanciated on the server, it can then do

Things->get_name(12,the_name);

The operation "get_name" of the Ice Object CThingAccessor will basically execute the code above and return the value of "name" to "the_name". Of course the benefit of this solution is that I can keep a lot of code already using succesfuly my system. But are there big drawbacks?

What do you think is the best approach in the sens of using Ice at its best, and to create the most "professional" application?

If a get really a lot of entries in my database, is there a problem to instanciate a lot of small objects for the whole time of the server activity?

I would define the objects as Ice objects. You can then either use Freeze (least amount of work), or write your own persistence service that uses MySQL.

This sounds complicated, but it is not so difficult. For example, you can use a default servant, then you only have one single servant for all Ice objects. The default servant then delegates the requests to the MySQL database.

Alternatively, you could keep your old system, and just write an Ice wrapper. The Ice objects, implemented with a default servant, would then just forward requests to your existing C++ objects, that do all the work.

In any case, there is no need to create more servants than a single default servant. Creating a servant per Ice object doesn't scale. If your database holds milliions of Ice objects, you cannot have a servant for each of them.

A more elaborate way to do this is an evictor pattern, similar to what Freeze uses. Then you use a pool of servants, that get evicted (and data saved) if they become idle for some time. New servants are automatically instantiated with the persistent data.

We will soon publish a new chapter "Advanced Server Programming", which will explain all these strategies in detail. In the meantime, there are two resources you can look at:

• For default servants, have a look at IcePatch
• For an evictor, have a look at Freeze

Originally posted by sylvain


In the first approach if I want to get the name of all the CThings objects, I need to do a lot of server requests, but in the second approach I can add a "get_all_name" operation that return a sequence of string containing all the value at once...

In the second approach, It seems that there is too much synchronizations: one for the object ThingAccessor and one for the CThings, is it right?

In the fist approach, can a safely and quickly get the correct proxy for the correct id (I need to keep the unique relation object-id).

Thank you very much in advance for any hint.

Again, most of your question will probably be answered by our upcoming chapter for the Ice documentation. We will publish a preview of this chapter soon.

[sylvain]

Thanks a lot Guido and Marc for your replies. It helps me a lot and gave me some (hopefuly good idea)

I get one more question regarding what you said Marc, in your post:

Originally posted by marc

A more elaborate way to do this is an evictor pattern, similar to what Freeze uses. Then you use a pool of servants, that get evicted (and data saved) if they become idle for some time. New servants are automatically instantiated with the persistent data.

This sound really cool and I have read again the Freeze chapter. But, I am a bit scared of what could happen if two different clients call the same mutating operation for the same logical object. Is there a chance that I will get 2 threads executing the same code for the same object, one modifying the data while the other is loading or saving it?

I am sure there is a mechanism to prevent this. Could you explain it to me quickly?

Originally posted by marc

Again, most of your question will probably be answered by our upcoming chapter for the Ice documentation. We will publish a preview of this chapter soon.

Great! I can't wait for it to be released.

Thank you.

[mes]

Originally posted by sylvain
This sound really cool and I have read again the Freeze chapter. But, I am a bit scared of what could happen if two different clients call the same mutating operation for the same logical object. Is there a chance that I will get 2 threads executing the same code for the same object, one modifying the data while the other is loading or saving it?

I am sure there is a mechanism to prevent this. Could you explain it to me quickly?

It is possible for multiple threads to be invoking operations on a servant. In this situation, it is the servant's responsibility to use synchronization to prevent the possibility of race conditions. Freeze does ensure that operations are not dispatched to a servant while the servant's persistent state is being saved or loaded. Have a look at the Freeze demos for some sample code.

Take care,
- Mark