memory cost of bidir

[kan]

Hi, We are evaluating bidir.
I notice that the simple demo client will cost nearly 4M ram. and after implementing bidir in our app. the memory cost also increases nearly 4-5M ram.

Why bidir will increase so many RAM? can we reduce it? since the memory cost metrics is critical to our app.


Thanks
Kan

[matthew]

In order to provide you with support you must fill out your signature as detailed in this post http://www.zeroc.com/vbulletin/showthread.php?t=1697

[kan]

I notice that the initial memory cost of bidir client is nearly 4M ram, but after running 1 hour, the memory decreased to 800K. very interesting!
can anyone explain the reason?

and i tries to run 2000 clients concurrently, and set Ice.ThreadPerConnection=1 property.

the server runs very well, untill the thread.cpp reports out of memory.
But I notice there is still memory available.

Any how, I really appreciate the performance of ICE bidir. It's a great work!

Thanks

Kan

[kan]

as many guys told me the concurrent sockets opened by a process is limited by the OS.

I tested bidir demo in XP(SP1), and seems that 1024 is not the limit.
can anyone tell me the concrete maximum number for windows OS?

Kan

[matthew]

There is a FAQ in the latest connections regarding memory management under XP that will help answer your memory question. I suspect the problem you are running into with the bidir demo and 2000 connected client is related to running out of handles, not running out of sockets.

[matthew]

My colleague pointed that its most likely actually the size of all of the thread stacks that is causing the memory to run out. 2000 connected clients with thread per connection means 2000 threads. 2000 threads each with the default amount of stack space probably exceeds the 32 bit address space. You should lower the default stack space per thread (Ice.ThreadPerConnection.StackSize) and it may resolve this issue -- temporarily at least.

Why are you using thread per connection anyway?

[kan]

Thanks Matthew for your prompt reply.
I do not mean to use thread per connection anyway.
My primary purpose is to verify the concurrent connection limit of bidir.
Orignially I use Ice.ThreadPool.Server.Size=10. But I noticed that when the number of clients increased, the server will run low of thread pool.
that's why I use thread per connection instead.

Kan

[matthew]

Note that if you go over FD_SETSIZE (which we set to 1024) sockets then you will run into problems with the thread pool as well.

If you need more than this many connected clients concurrently then you should rebuild Ice yourself and change the value of FD_SETSIZE in the compiler preprocessor flags.

Keep in mind that if your clients are busy your server will probably not be able to process that many clients at a time anyway...

[marc]

Thanks Matthew for your prompt reply.
I do not mean to use thread per connection anyway.
My primary purpose is to verify the concurrent connection limit of bidir.
Orignially I use Ice.ThreadPool.Server.Size=10. But I noticed that when the number of clients increased, the server will run low of thread pool.
that's why I use thread per connection instead.

Kan

Does it really matter for your application if the thread pool runs low on threads? The only situation where it does matter is if you use nested callbacks, because then it is possible that a call blocks indefinitely if not enough threads are available to service the nested callback.

If you don't have such nested callbacks, I recommend to simply disable this warning, or to set Ice.ThreadPool.Server.SizeMax to the same value as Ice.ThreadPool.Server.Size. See the user manual for further information.