Mutex access from two different servants

[Ewgenijkkg]

Hello. I implemented a server with one object adaptor and several servants assigned two it. I use the Read-Write-Recursive-Mutex to synchronize these two servants. But now I see a rather strange behaviour. Two of the servants are A and B. A is supposed to set a write lock, B is also supposed to set a write lock.
Now if I call A, prevent it from unlocking after it executes, and then again A - it blocks as expected, because the mutex is still locked. But if I call A and then B, B doesn't block. I can not understand it. B uses a pointer to the same mutex, I checked the pointer adress. But it locks the mutex again. In the manual is written that this is possible if both locks are called from the same thread. But these are two different servants, although assigned to the same adaptor. Could it be that they are started in the same thread???

Ewgenij

[Ewgenijkkg]

I checked it again.
That:

"Now if I call A, prevent it from unlocking after it executes, and then again A - it blocks as expected, because the mutex is still locked."

is not true. The program also does not block as expected in this case. Could the reason be that the servants belong to the same adaptor?

[benoit]

Hi,

Locking a recursive mutex will succeed if the mutex has already been locked by the thread. It doesn't have much to do with the servant or object adapter used to lock the mutex. So most likely, you're always locking the mutex from the same thread.

If you lock the mutex from servant dispatched methods, this is likely because the Ice server thread pool by default has only one thread.

Cheers,
Benoit.

[Ewgenijkkg]

Hello! But as I understood the Ice-Manual the server takes for each servant invocation a new thread out of the thread pool, so if I lock the mutex from to different proxies, it should be locked by two different threads. I set the Ice.ThreadPool.Server.SizeMax to 15, so the default value was overwritten.

Besides, what does it mean that the server takes threads from the thread pool? Does it mean, that the server creates a new thread, or is the thread already somewhere idle and the server uses it.

And how is a mutex supposed to be used if it does not block the servants of the server? I mean the use of a mutex in a server is that one servant A locks it and the mutex has to assure that no other servant B can access the critical region while A is working there. But apparently B does not care and accesses the critical region...

Greetings
Ewgenij

[benoit]

Originally Posted by Ewgenijkkg

Hello! But as I understood the Ice-Manual the server takes for each servant invocation a new thread out of the thread pool, so if I lock the mutex from to different proxies, it should be locked by two different threads. I set the Ice.ThreadPool.Server.SizeMax to 15, so the default value was overwritten.

I assume you meant "lock the mutex from two different servants" (not "proxies")? (see this FAQ for the correct terminology).

By setting the thread pool maximum size to 15, you tell the Ice runtime that the Ice server thread pool can grow to up 15 threads. Initially it will only have one thread if you didn't set other properties. Depending on the load of your server (i.e.: if there's many concurrent incoming calls) additional threads might be created.

Besides, what does it mean that the server takes threads from the thread pool? Does it mean, that the server creates a new thread, or is the thread already somewhere idle and the server uses it.

The Ice runtime will take a thread from the thread pool if one is available (i.e.: it's not currently dispatching an incoming call) or it might create a new one if necessary and if the thread pool is a dynamic thread pool (and the maximum size of the thread pool hasn't been reached).

See 32.9 The Ice Threading Models and the thread pool property reference for more information.

And how is a mutex supposed to be used if it does not block the servants of the server? I mean the use of a mutex in a server is that one servant A locks it and the mutex has to assure that no other servant B can access the critical region while A is working there. But apparently B does not care and accesses the critical region...

I'm afraid it's not clear to me what you're trying to do. When you lock the mutex in the servant A method, do you also unlock it once the method is done?

If you do unlock it, B will will be able to lock it since it's unlocked. But if you don't unlock it and if the mutex is recursive, then only the thread that locked it should be able to lock it again and access the critical region, other threads shouldn't be able to lock it.

When using the thread pool concurrency model, there's no way to know which thread will dispatch an incoming call from a client so you shouldn't rely on this. Are you perhaps expecting the invocations from a given Ice client to be always dispatched with the same thread in the server? If that's the case, you should use the thread per connection concurrency model.

In any case, you should be careful with locking a mutex from a method dispatch without unlocking it. If the Ice client connection is closed or the client crashes, your mutex will never be unlocked. You should use a session mechanism to ensure that everything is cleaned up in the server if the client goes down.

I recommend reading Michi's article "The Grim Reaper: Making Objects Meet Their Maker" from the Ice newsletter connection issue #3 for more information.

Cheers,
Benoit.

[Ewgenijkkg]

Hi.

I assume you meant "lock the mutex from two different servants" (not "proxies")

Yes

So, if I understood you right we have the following behaviour of an Ice server:

If a client calls a servant, the server takes a thread from its thread pool and assigns this thread to the servant. After the servant has finished, the thread becomes free and can be assigned to another servant if it is called. So if the first servant locked a recursive mutex and didn't unlock it, nevertheless the second servant can lock it again because it does this in the same thread which was just reassigned.

Is my description correct?

Now the reason for my question. I have a server with one client C1. In addition I have an application P which does not use any of the server's functionality. The goal is to prevent C1 from working on the server while P is performing its task. I thought a good idea would be to implement a mutex on the server side which P would lock at the beginning of its work and unlock at the end. So I implemented a servant which simply locks the mutex with its function lock() and unlocks it with its function unlock(). This servant is then called by P at the beginning and at the end of P's work. But as I have written before, it doesn't work. Apparently because after the lock() function was executed, the same thread is assigned to the servant called by C1 and the mutex is locked again.

So is there a possibility to synchronize my applications? It should work with the thread per connection concurrency model, I think. Or is it also possible with the thread pool model?

Best regards
Ewgenij

[benoit]

Your description is correct.

I don't think using the thread per connection concurrency model would be a good idea. If the connection goes down and the mutex is locked, you wouldn't be able to unlock the mutex.

Instead, you should implement your own specific locking. Something like the following for example:

   void MyServantI::lock() // Called by P to prevent C1 from doing some work.
   {
       Lock sync(_monitor);
       while(_busy)
       {
           _monitor.wait();
       }
       _busy = true;       
   }

   void MyServantI::unlock() // Called by P to allow C1 calls to be processed.
   {
       Lock sync(_monitor);
       assert(_busy);
       _busy = false;
       notify();
   }

   void MyServantI::sayHello(const Ice::Current&) // Called by C1
   {
       Lock sync(_monitor);
       while(_busy) // Wait until P is done with his work.
       {
            _monitor.wait();
       }

       // Dome some work for C1
   }

Cheers,
Benoit.

[Ewgenijkkg]

OK, thanks a lot. Now I know what I got to do!