application architecture question

[mangrish]

Hi,

I am going over an approach for an architecture of a system and i really need some help!

I have a condition where we have lots of client GUI's around the world that will talk to a small handful of servers over some sort of ice middleware (ice, icebox,icestorm.. we are flexible here).

The conditions of the architecture are that the gui's have to determine who is the closest server to them, and then execute a call to that server that will return data.

udp multicasting is not an option unfortunately so i am having to think of different possibilities.

My current thoughts involve using icestorm to broadcast, then a server will try to establish an ice callback. To me this would mean an icestorm instance and something like 2 ice applications running as threads; one to receive icestorm topic messages and one to invoke a callback dynamically (via an ip and port supplied in the call by the client).

Would this be sound or is there some other approach that might be more suitable.

cheers,

::mark

[mangrish]

i should also mention that clients should be able to find a server anywhere (i.e. resiliance)!

[bernard]

Hi Mark,

The most basic solution is to create a replicated manager/factory object implemented by your servers (distributed across the world), and tell each client which servers to try, and in which order to try them.
(There is no IceStorm, IceGrid or any other service involved in this: just plain Ice)

The client configuration would be something like:

MyFactory=factory:ssl -h server1 -p 10000:ssl -h server2 -p 10000:ssl -h server3 -p1000

(here "factory" is the object-id of this replicated object, and each client would list the servers in closest-to-furthest-away order)

Your client code would use communicator.propertyToProxy("MyFactory") to create the corresponding proxy. Without additional configuration, Ice would pick one of the server endpoints at random, which defeats the ordering. You need to add:

MyFactory.EndpointSelection=Ordered

to tell Ice to try the endpoints in the provided order.

(I recommend reading "Connection Management in Ice" in http://www.zeroc.com/newsletter/issue24.pdf for more details on how Ice established connections).

Now, this simple solution works, but is not very flexible. If you add a new server, or move a server from one machine to another one, you'll have to update the configuration of all your clients (otherwise, they'll still work but won't find the new/moved server).

One improvement would be to use DNS to list your servers, i.e. you would create a DNS name "replicatedserver" that returns the server IPs in the desired order (presumably each client uses its own, local, DNS server). So the client configuration would become:

MyFactory=factory:ssl -h replicatedserver -p 10000
MyFactory.EndpointSelection=Ordered

and each time you add a new server or move a server, you would update the DNS entry for "replicatedserver".

If this DNS approach doesn't work (perhaps it's too hard to control the order of IPs in your various DNS servers), then you could instead introduce an intermediary server, whose only job is to provide the clients with the "best" factory proxy. Its operation would be something like:

interface FactoryLocator
{
   Factory* getFactory(string clientLocation);
};

or

interface FactoryLocator
{
  Factory* getFactory();
};

if you can use the client's IP to determine its location.

Of course, for fault-tolerance, you'd want to replicate this FactoryLocator server. The client would connect to one of the factory locators (not necessarily the closest one), get a good Factory proxy and proceed from there. Each time you add a new server/move an existing server, you would just need to update your factory locator instances.

The next level in sophistication is to use IceGrid to manage your servers, and replace the "Factory locator" by your own Ice::Locator implementation. This implementation would delegate to the IceGrid registry, and provide the ordering logic (closest to furthest), since IceGrid doesn't include this policy.

Best regards,
Bernard

[mangrish]

Hi Bernard,

Thanks for getting back to me.

Unfortunately outside of using IceGrid the other options rely on hardcoding the locations of the servers in some respect.

Could you maybe verify my proposal of using a topic via icestorm for servers to recieve calls from any client (a client may execute something like a ping() slice operation on the server) and then using straight ice and the ip/port supplied by the client to ping execute an ice call back to the client so a client can reinitiate an ice call to the server who responded the most timely.

We are effectively making a p2p looking structure but unfortunately we require it!

::mark

[bernard]

Hi Mark,

I don't see how IceStorm can solve this bootstrap issue. How will your clients find the IceStorm Topic or TopicManager?

If you use IceGrid, you still need to tell your clients where to find the IceGrid registry, with something like:

Ice.Default.Locator=MyGrid/Locator:tcp -h registryhost -p 4061

At startup, you always need the direct addressing information of at least one object (in your own server, IceGrid, IceStorm). There are of course many ways to retrieve this first proxy (hard-coded in the program, in a config file, entered by the user, through a UDP broadcast/multicast mechanism ...).

Cheers,
Bernard

[mangrish]

Hi Bernard,

Having gone through your suggestions I have decided to take a more careful look at IceGrid.

I need to effectively have a peer to peer type set up where my clients need to set up reliable two way communication with the closest server. Clients can make requests to the server, but the server also publishes data that the clients need to know about.

With IceGrid, am i making a 'client' and a 'server' on each side to achieve this result? I was thinking about enuring reliable communication with the peers maybe making use of some sort of id's.

Do you think this approach is sound?

::mark

[bernard]

Hi Mark,

It is very common for a "client" to act as a server, for example to receive callbacks from a "server". And for such callbacks, the "server" acts as a client since it's invoking an operation on an object hosted by the "client".

And then, you can decide to use a bi-dir connection between your client and server (i.e. have the "server" reuse the connection established by the client to send callback-requests back to the "client"), or simply use the default behavior, and have the "server" open its own connections the "clients". This default (opening connections 'back to' the client) is usually not an issue on a LAN.

All the above is just plain Ice, and IceGrid would not be involved. I recommend reading "Teach Yourself IceGrid in 10 minutes" for an overview of IceGrid.

Best regards,
Bernard