Ice is inherently a multi-threaded platform. There is no such thing as a single-threaded server in Ice. As a result, you must concern yourself with concurrency issues: if a thread reads a data structure while another thread updates the same data structure, havoc will ensue unless you protect the data structure with appropriate locks. In order to build Ice applications that behave correctly, it is important that you understand the threading semantics of the Ice run time. This section discusses the two concurrency models that Ice supports, called
thread pool and
thread-per-connection, and provides guidelines for writing thread-safe Ice applications.
The thread-pool model is the default concurrency model in Ice. A thread pool is a collection of threads that the Ice run time draws upon to perform specific tasks. Each communicator creates at least two thread pools:
•
The client thread pool services outgoing connections, which primarily involves handling the replies to outgoing requests and includes notifying AMI callback objects (see
Section 33.3). If a connection is used in bidirectional mode (see
Section 37.7), the client thread pool also dispatches incoming callback requests.
•
The server thread pool services incoming connections. It dispatches incoming requests and, for bidirectional connections, processes replies to outgoing requests.
By default, these two thread pools are shared by all of the communicator’s object adapters. If necessary, you can configure individual object adapters to use a private thread pool instead.
If a thread pool is exhausted because all threads are currently dispatching a request, additional incoming requests are transparently delayed until a request completes and relinquishes its thread; that thread is then used to dispatch the next pending request. Ice minimizes thread context switches in a thread pool by using a leader-follower implementation (see
[17]).
If name.SizeMax is less than
name.Size,
name.SizeMax is adjusted to be equal to
name.Size.
For configuration purposes, the names of the default client and server thread pools are
Ice.ThreadPool.Client and
Ice.ThreadPool.Server, respectively. As an example, the following properties establish minimum and maximum sizes for the thread pools:
This property sets a high water mark; when the number of threads in a pool reaches this value, the Ice run time logs a warning message. If you see this warning message frequently, it could indicate that you need to increase the value of
name.SizeMax. If not defined, the default value is 80% of the value of
name.SizeMax. Set this property to zero to disable the warning.
Configuring a private thread pool for an adapter is useful in avoiding deadlocks due to thread starvation by ensuring that a minimum number of threads is available for dispatching requests to certain servants.
These properties have the same purpose as those described earlier, but they both have a default value of zero. If neither property is defined, the adapter uses the communicator’s thread pools by default. If
SizeMax is not defined, it defaults to the value of
Size, meaning the thread pool does not grow dynamically.
Object adapter thread pools also support the SizeWarn and
StackSize properties described in the previous subsection.
Choosing an inappropriate size for a thread pool can have a serious impact on the performance of your application. Since the client and server thread pools each contain only one thread by default, it is important that you understand the implications of using them in their default configurations:
The thread-per-connection concurrency model creates a separate thread for each incoming and outgoing connection. The thread for an incoming connection dispatches requests and, if the connection is bidirectional, handles replies to outgoing bidirectional requests. The thread for an outgoing connection processes replies and, if the connection is bidirectional, dispatches incoming requests.
Ice allows you to use both threading models in the same communicator. We discuss the semantics of incoming and outgoing connections in separate sections.
The threading model associated with an incoming connection is ultimately determined by the object adapter servicing that connection. If an object adapter is not specifically configured to use a particular threading model, it uses the communicator’s default model. For example, the object adapter uses thread-per-connection if
Ice.ThreadPerConnection is defined, otherwise it uses the communicator’s thread pool.
Section 32.9.1 describes how to configure an object adapter to use a private thread pool, as shown in the example below:
Setting either ThreadPool.Size or
ThreadPool.SizeMax is sufficient for configuring a private thread pool. You can configure an object adapter to use thread-per-connection in a similar way:
An outgoing connection is created as a side-effect of using a proxy, therefore it is the proxy that determines the connection’s threading model. Normally the proxy uses the communicator’s default threading model, which is thread pool unless
Ice.ThreadPerConnection is defined.
You can also govern the threading model of a particular proxy using the ice_threadPerConnection method. If you pass the value
true, the proxy’s connection uses the thread-per-connection model, otherwise the proxy uses the thread pool. In C++, you can obtain a proxy that uses thread-per-connection as shown below:
See Section 32.10 for more information on configuring proxies.
Thread pool is the default concurrency model because it is an appropriate choice for most applications. Thread-per-connection is useful in certain situations and mandatory in others, such as when using IceSSL for Java (see
Chapter 42). This section discusses some issues to consider when deciding which concurrency model is appropriate for your application.
Since thread-per-connection creates a new thread for each connection, a program that establishes hundreds of connections also creates hundreds of threads. The use of active connection management (see
Section 37.4) to reap idle connections (and therefore the threads associated with them) can mitigate this somewhat, but thread-per-connection does not scale as well as a thread pool. In fact, the ability to set the maximum size of a thread pool allows you to tune an Ice application to match the hardware capabilities of its host. On a multi-processor machine, for example, you may decide to limit the thread pool to the number of physical processors in order to minimize the overhead of thread context switches.
The thread-per-connection model is useful when you need to serialize requests from a client. For instance, suppose that a transaction processing server must ensure that requests are dispatched in the order they are received. If the thread-per-connection model is enabled, only one thread can dispatch the requests received on a connection, and therefore serialization is guaranteed (assuming the client is not sending requests on multiple connections).
This requirement could also be satisfied with a thread pool, but only if you limit the maximum size of the thread pool to one or, alternatively, if you can guarantee that the client does not send requests from multiple threads and does not send oneway requests. The disadvantage of using a thread pool with only one thread is that it serializes requests from all clients, rather than just the requests from a single connection. If you use a larger thread pool, and the client sends requests from multiple threads or uses oneway requests, then the operating system’s thread scheduling behavior in the server could cause requests to be dispatched out of order (see
Section 32.13).
The fact that thread-per-connection serializes requests could just as easily be considered a disadvantage in a use case with different requirements. For example, thread-per-connection might be an inappropriate choice for a server with long-running operations when the client needs the ability to have several operations in progress simultaneously. There are ways to achieve the client’s concurrency requirements while using the thread-per-connection model. One option is to design the client so that it forces new connections to be established (see
Section 37.3), however this tightly couples the client with the server implementation. Another alternative is for the server to use asynchronous dispatch in order to avoid blocking the connection’s thread, and use a work queue to execute the requests in separate threads. However, unless the server needs to track the order of requests, a thread pool provides similar functionality with less effort.
A nested invocation is one that is made within the context of another Ice operation. For instance, the implementation of an operation in a servant might need to make a nested invocation on some other object, or an AMI callback object might invoke an operation in the course of processing a reply to an asynchronous request. It is also possible for one of these invocations to result in a nested callback to the originating process. The maximum depth of such invocations is determined by the concurrency models in use by the communicating parties.
Applications that use nested invocations must be carefully designed to avoid the potential for deadlock, which can easily occur when invocations take a circular path. For example,
Figure 32.5 presents a deadlock scenario when using the default thread pool configuration.
In this diagram, the implementation of opA makes a nested twoway invocation of
opB, but the implementation of
opB causes a deadlock when it tries to make a nested callback. As mentioned in
Section 32.9.1, the default thread pools have a maximum size of one thread unless explicitly configured otherwise. In Server A, the only thread in the server thread pool is busy waiting for its invocation of
opB to complete, and therefore no threads remain to handle the callback from Server B. The client is now blocked because Server A is blocked, and they remain blocked indefinitely unless timeouts are used.
Configuring the server thread pool in Server A to support more than one thread allows the nested callback to proceed. This is the simplest solution, but it requires that you know in advance how deeply nested the invocations may occur, or that you set the maximum size to a sufficiently large value that exhausting the pool becomes unlikely. For example, setting the maximum size to two avoids a deadlock when a single client is involved, but a deadlock could easily occur again if multiple clients invoke
opA simultaneously.
If Server A called
opB using a oneway invocation, it would no longer need to wait for a response and therefore
opA could complete, making a thread available to handle the callback from Server B. However, we have made a significant change in the semantics of
opA because now there is no guarantee that
opB has completed before
opA returns, and it is still possible for the oneway invocation of
opB to block (see
Section 32.13).
•
Implement opA using asynchronous dispatch and invocation.
By declaring opA as an AMD operation (see
Section 33.4) and invoking
opB using AMI, Server A can usually avoid blocking the thread pool’s thread while it waits for
opB to complete. As with oneway invocations, however, the asynchronous invocation of
opB can block the calling thread if network buffers fill up.
As another example, consider a client that makes a nested invocation from an AMI callback object using the default thread pool configuration. The (one and only) thread in the client thread pool receives the reply to the asynchronous request and invokes its callback object. If the callback object in turn makes a nested twoway invocation, a deadlock occurs because no more threads are available in the client thread pool to process its reply. The solutions are similar to some of those presented for
Figure 32.5: increase the maximum size of the client thread pool, use a oneway invocation, or call the nested invocation using AMI. The use of thread-per-connection might also solve this deadlock, but only if the nested invocation is sent over a different connection.
•
The concurrency models in use by all communicating parties have a significant impact on an application’s ability to use nested invocations. While analyzing the path of circular invocations, you must pay careful attention to the threads involved to determine whether sufficient threads are available to avoid deadlock. This includes not just the threads that dispatch requests, but also the threads that make the requests and process the replies.
As you can imagine, tracing the call flow of a distributed application to ensure there is no possibility of deadlock can quickly become a complex and tedious process. In general, it is best to avoid circular invocations if at all possible.
The Ice run time itself is fully thread safe, meaning multiple application threads can safely call methods on objects such as communicators, object adapters, and proxies without synchronization problems. As a developer, you must also be concerned with thread safety because the Ice run time can dispatch multiple invocations concurrently in a server. In fact, it is possible for multiple requests to proceed in parallel within the same servant and within the same operation on that servant. It follows that, if the operation implementation manipulates non-stack storage (such as member variables of the servant or global or static data), you must interlock access to this data to avoid data corruption.
The need for thread safety in an application depends on its chosen concurrency model. Using the default thread pool configuration typically makes synchronization unnecessary because at most one operation can be dispatched at a time. Thread safety becomes an issue once you increase the maximum size of a thread pool, or use thread-per-connection.
Ice uses the native synchronization and threading primitives of each platform. For C++ users, Ice provides a collection of convenient and portable wrapper classes for use by Ice applications (see
Chapter 31).
The marshaling semantics of the Ice run time present a subtle thread safety issue that arises when an operation returns data by reference. In C++, the only relevant case is returning an instance of a Slice class, either directly or nested as a member of another type. In Java, C#, Visual Basic and Python, Slice structures, sequences, and dictionaries are also affected.
The potential for corruption occurs whenever a servant returns data by reference, yet continues to hold a reference to that data. For example, consider the following Java implementation:
public class GridI extends _GridDisp
{
GridI()
{
_grid = // ...
}
public int[][]
getGrid(Ice.Current curr)
{
return _grid;
}
public void
setValue(int x, int y, int val, Ice.Current curr)
{
_grid[x][y] = val;
}
private int[][] _grid;
}
Suppose that a client invoked the getGrid operation. While the Ice run time marshals the returned array in preparation to send a reply message, it is possible for another thread to dispatch the
setValue operation on the same servant. This race condition can result in several unexpected outcomes, including a failure during marshaling or inconsistent data in the reply to
getGrid. Synchronizing the
getGrid and
setValue operations would not fix the race condition because the Ice run time performs its marshaling outside of this synchronization.
One solution is to implement accessor operations, such as getGrid, so that they return copies of any data that might change. There are several drawbacks to this approach:
Another solution is to make copies of the affected data only when it is modified. In the revised code shown below,
setValue replaces
_grid with a copy that contains the new element, leaving the previous contents of
_grid unchanged:
public class GridI extends _GridDisp
{
...
public synchronized int[][]
getGrid(Ice.Current curr)
{
return _grid;
}
public synchronized void
setValue(int x, int y, int val, Ice.Current curr)
{
int[][] newGrid = // shallow copy...
newGrid[x][y] = val;
_grid = newGrid;
}
...
}
This allows the Ice run time top safely marshal the return value of getGrid because the array is never modified again. For applications where data is read more often than it is written, this solution is more efficient than the previous one because accessor operations do not need to make copies. Furthermore, intelligent use of shallow copying can minimize the overhead in mutating operations.
Finally, a third approach changes accessor operations to use AMD (see Section 33.4) in order to regain control over marshaling. After annotating the
getGrid operation with
amd metadata, we can revise the servant as follows:
public class GridI extends _GridDisp
{
...
public synchronized void
getGrid_async(AMD_Grid_getGrid cb, Ice.Current curr)
{
cb.ice_response(_grid);
}
public synchronized void
setValue(int x, int y, int val, Ice.Current curr)
{
_grid[x][y] = val;
}
...
}
Normally, AMD is used in situations where the servant needs to delay its response to the client without blocking the calling thread. For getGrid, that is not the goal; instead, as a side-effect, AMD provides the desired marshaling behavior. Specifically, the Ice run time marshals the reply to an asynchronous request at the time the servant invokes
ice_response on the AMD callback object. Because
getGrid and
setValue are synchronized, this guarantees that the data remains in a consistent state during marshaling.
On occasion, it is necessary to intercept the creation and destruction of threads created by the Ice run time, for example, to interoperate with libraries that require applications to make thread-specific initialization and finalization calls (such as COM’s
CoInitializeEx and
CoUninitialize). Ice provides callbacks to inform an application when each run-time thread is created and destroyed. For C++, the callback class looks as follows:
1
class ThreadNotification : public IceUtil::Shared {
public:
virtual void start() = 0;
virtual void stop() = 0;
};
typedef IceUtil::Handle<ThreadNotification> ThreadNotificationPtr;
To receive notification of thread creation and destruction, you must derive a class from
ThreadNotification and implement the
start and
stop member functions. These functions will be called by the Ice run by each thread as soon as it is created, and just before it exits. You must install your callback class in the Ice run time when you create a communicator by setting the
threadHook member of the
InitializationData structure (see
Section 32.3).
class MyHook : public virtual Ice::ThreadNotification {
public:
void start()
{
cout << "start: id = " << ThreadControl().id() << endl;
}
void stop()
{
cout << "stop: id = " << ThreadControl().id() << endl;
}
};
int
main(int argc, char* argv[])
{
// ...
Ice::InitializationData id;
id.threadHook = new MyHook;
communicator = Ice::initialize(argc, argv, id);
// ...
}
The implementation of your start and
stop methods can make whatever thread-specific calls are required by your application.
For Java and C#, Ice.ThreadNotification is an interface:
public interface ThreadNotification {
void start();
void stop();
}
To receive the thread creation and destruction callbacks, you must derive a class from this interface that implements the
start and
stop methods, and register an instance of that class when you create the communicator. (The code to do this is analogous to the C++ version.)
