File transfer and ThreadPerConnection

[José Ángel]

Hello,

I have implemented the method for transferring files proposed in the article "Optimizing Performance of File Transfer" with a modification to write from an specified offset different pieces of a file.

I have upgraded to Ice 3.3 and with the ausence of ThreadPerConnection concurrency model files are not transferred correctly. I think this is because asyncronous calls are not received in the same order that they are requested.

Besides, sometimes remote objects are removed prematurely from the object adapter and next calls results in an Ice::ObjectNotExistException.

I have tried the Serialize property but it does not work properly, i do not know if i am doing something wrong.

How can i solve this issue?

Thanks in advance.

[dwayne]

The easiest way for us to help you with your problem is for you to post a small, complete example including any configuration you are using that demonstrates the problem you are seeing. Also, what os/compiler are you using?

[matthew]

I'm afraid really that you haven't given nearly enough information about what you are really doing to help. Source code, and a full description of the error you are encountering is always the best help.

[José Ángel]

Hello,

First of all, thanks for your reply. I have isolated the piece of code for transferring files and i have prepared an example with just the relevant cofiguration. As an example, i am trying to transfer the logo3d.blend file.

The fact is that if i comment all ThreadPool properties in both client and server cofiguration, the file is transferred correctly.

If i use the configuration of the cfg files for the ThreadPool, there is not error but when the transfer finishes the file is corrupted in the client and i can not open it.

Besides, sometimes the server crashes and i get:

*** glibc detected *** ./FileServer: double free or corruption (!prev): 0x089e2df0 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6[0xb7b646b4]
/lib/i686/cmov/libc.so.6(cfree+0x96)[0xb7b668b6]
/lib/i686/cmov/libc.so.6(fclose+0x144)[0xb7b542f4]
./FileServer[0x8054ac6]
./FileServer[0x8058e16]
./FileServer[0x80590ea]
/usr/lib/libIce.so.33(_ZN11IceInternal8Incoming6invokeERKNS_6HandleINS_14ServantManagerEEE+0x79f)[0xb7e6577f]
/usr/lib/libIce.so.33(_ZN3Ice11ConnectionI9invokeAllERN11IceInternal11BasicStreamEiihRKNS1_6HandleINS1_14ServantManagerEEERKNS4_INS_13ObjectAdapterEEE+0x10e)[0xb7e366de]
/usr/lib/libIce.so.33(_ZN3Ice11ConnectionI7messageERN11IceInternal11BasicStreamERKNS1_6HandleINS1_10ThreadPoolEEE+0x135)[0xb7e3c3d5]
/usr/lib/libIce.so.33(_ZN11IceInternal10ThreadPool3runEv+0xdbc)[0xb7f3f8cc]
/usr/lib/libIce.so.33(_ZN11IceInternal10ThreadPool18EventHandlerThread3runEv+0x7c)[0xb7f4041c]
/usr/lib/libIceUtil.so.33[0xb7ae6aab]
/lib/i686/cmov/libpthread.so.0[0xb7aaa4c0]
/lib/i686/cmov/libc.so.6(clone+0x5e)[0xb7bd661e]

and a long memory map.

With Ice 3.2 i used the Ice.ThreadPerConnection property and everything was working fine.

I am using Debian and the compiler is g++ (Debian 4.3.2-1) 4.3.2. Hope this helps. If there is any relevant info i did not post, please make me know.

Regards.

[dwayne]

You are not setting the serialize property when you are configuring the thread pools. If you do not set the thread pool size then it defaults to 1. In this case your test works because with a single thread all requests are automatically serialized. However, once you increase the size of your thread pool if you still want all requests from a connection serialized then you have to set the serialize property. In this case add

Ice.ThreadPool.Server.Serialize=1

to server.cfg. I tried this and the file transferred properly.

Having said that, the fact that your server sometimes crashes indicates that your code most likely has other problems as well.

[dwayne]

I had a look at your code and the most likely cause of your crashes was the fact that you have no synchronization in your next() method and since you were not serializing requests you were getting concurrent access to code not meant to be accessed concurrently.

A couple of other things, in your example there is no need to use amd for the next() method as it provides no benefit here. Also there is no need to add more threads to the Server/Client thread pools in your client or to the Client thread pool in your server.