[feline]

First, let me say that ICE is very impressive. Although I have not used it yet, I have read all the documentation and examined various bits of the source. All very impressive.

I have some scalability questions. I have a need to develop a server which will need to have several thousand simultaneous connections. About 2000-5000 on average with up to about 8000 connected users during peak loads. Based on what I have seen in the source it is not possible for an ICE server to handle more than 1024 connections due to the usage of select(). Now I realize there are some work around which can increase the FD_SETSIZE. I consider this a brute force solution....albeit an easy one on some platforms.

Have you considered using poll() instead of select() on the platforms which support poll()? (most unix platforms support poll(). The poll() usage would solve two issues....(a) number of connections, (b) if I open 500 files...then ICE would only accept ~500 connections....as the maximum fd supported by most select() calls is FD_SETSIZE (which is 1024 on most platforms by default).

Now I have read....select() and poll() do not scale well for >1000 connections in many cases.....mainly due to user/kernel context switching I think?? So achieving scalability via poll() or increasing FD_SETSIZE might not be the best way to support many thousands of simultaneous connections. What other solutions do you recommend? I looked at glacier...and it comes close to supporting what I had in mind...this was my thought....and I was curious if glacier already supported it...or if ice came close to supporting it. The general idea is that I have one server which runs on a well-defined port. Its sole purpose in life is to accept connections and re-direct them to a router process (like glacier starter appears to do). This server would keep track of how many connections each server contains and appropriately start more routers or kill routers depending on current load. Each router would connect to my main backend server and simply forward client messages to the backend server. Effectively I could get say 1000 connections to each router, and connect 1000 routers to my main backend server.....giving me more than enough connections.

Is there already support in glacier and/or ice for this? Is this feasible in ICE?....ideally I do not want to spend much time on the router part of implementation. It looks like Glacier comes very close to this....and it is likely I will need to use glacier anyway....what is not clear is if the glacier starter can handle the load-balancing/connection arbitration part of it all.

Is this clear? Has anyone tackled similar problems in the past? Anyone got better ideas?

Thanks in advance for your help