Why do I get Ice::MemoryLimitException, even with Ice.MessageSizeMax=2000000

[spookstaz]

Hi I have written a C# client/server application using the Zeroc Ice communication libary (v3.4.2).

I am transferring a sequence of objects from the server which are then displaying them in the client in a tabular format. Simple enough.

I defined following slice types

enum DrawType { All, Instant, Raffle };

struct TicketSoldSummary {
  int scheduleId;
  DrawType dType;
  string drawName;
  long startDate;
  long endDate;
  string winningNumbers;
  int numTicket;
  string status;
};
sequence<TicketSoldSummary> TicketSoldSummaryList;

interface IReportManager {
  [..]
  TicketSoldSummaryList getTicketSoldSummary(long startTime, long endTime);
};

When I call this method it usually works fine, but occasionally (approx 25% of the time) the caller gets a Ice::MemoryLimitException. We are usually running 2-3 clients at a time.

I searched on the Internet for answers and I was told to increase Ice.MessageSizeMax, which I did in both the client and server. I have increased MessageSizeMax right up to 2,000,000 Kb, but it made no difference, I just did a test with 31,000 records (approximately 1.8 Megs of data) and still get Ice.MemoryLimitException. 1.8 Megs is not very big!

Am I doing something wrong or is there a bug in Zeroc Ice?

Thanks so much to anyone that can offer some help.

[mes]

Hi Paul,

Welcome to the forum.

Keep in mind that the value of Ice.MessageSizeMax is expressed in KB, so the value Ice.MessageSizeMax=3076 sets the limit to 3MB.

If you are still receiving the MemoryLimitException, one possibility is that you have not correctly set Ice.MessageSizeMax. The quickest way to resolve this issue would be for you to post a small code example that demonstrates the problem, and also include your configuration settings.

Regards,
Mark

[spookstaz]

Thanks, it will take me a little while to put together a sample project, so for starters, here is how I am setting MessageSizeMax:

static string[] _args = new[] {
    "--Ice.MessageSizeMax=2000000"
};

public static void CreateSession(string host)
{
       Communicator _ic = Util.initialize(ref _args);
       [...]
}

Does this look correct?

[spookstaz]

If it helps I've found that in my case getting this error seems to be related to the network speed. For example over VPN I get the error 90% of the time, but over LAN only 25% of the time

[mes]

Hi,

I don't see anything wrong with the way you're setting the property.

If it helps I've found that in my case getting this error seems to be related to the network speed. For example over VPN I get the error 90% of the time, but over LAN only 25% of the time.

A MemoryLimitException is raised when the size of an outgoing or incoming protocol message exceeds the value of Ice.MessageSizeMax. The speed of the network has no effect on the size of a protocol message.

Please post the complete stack trace of the MemoryLimitException, as that might provide some additional clues.

Regards,
Mark

[spookstaz]

I got this traceback recently. Would it make any difference if I did a synchronous call instead?

Exception of type 'Ice.MemoryLimitException' was thrown
at IceInternal.OutgoingAsyncBase.wait__()
at UserService.IReportManagerPrxHelper.end_getTicketSoldSummary(AsyncResult r__)
at Client.Reports.TicketSoldView.<>c__DisplayClass23.<BtnDetailsReport_Click>b__1e(AsyncResult r)

[mes]

No, it shouldn't make any difference whether you invoke the operation synchronously or asynchronously. However, I would be interested in seeing the stack trace of MemoryLimitException for a synchronous invocation.

I recommend setting Ice.Warn.Connections=1 in your client. When MemoryLimitException is raised, you should see a corresponding message in the log output like this:

-! 12/8/2011 10:24:01:679 client.exe: warning: connection exception:
   Ice.MemoryLimitException
       reason = "requested 2097182 bytes, maximum allowed is 1048576 bytes (see Ice.MessageSizeMax)"
   ...

The cause of the exception is stated pretty clearly in the reason member.

Regards,
Mark

[spookstaz]

Managed to reproduce the same error again. I have a log file now and here are my current ice args:

"--Ice.ThreadPool.Client.Size=4",
"--Ice.ThreadPool.Server.Size=4",
"--Ice.ACM.Client=0", 
"--Ice.RetryIntervals=10 50 250 3000",
"--Ice.Warn.Connections=1",
"--Ice.MessageSizeMax=2000000",
"--Ice.Trace.Network=3",
"--Ice.Trace.Protocol=1",
"--Ice.Trace.Retry=2",
"--Ice.LogFile=ice.log"

Odd thing is that even though my code caught an Ice.MemoryLimitException, I don't see that exception in the log file. However there are a quite lot of connection refused errors in the log file which I did not see in my application.

[mes]

Hi Paul,

One other thing to keep in mind: Ice does not place any limits on how rapidly a program can send asynchronous requests. If the program is making async invocations faster than the network can send the data, the invocations will queue up in the Ice run time until they have a chance to drain out, or until you exhaust available memory.

I don't know if your program is making async invocations in rapid succession, but if it is, you should consider making use of Ice's flow-control API. The C# docs can be found on this page.

Regards,
Mark

[spookstaz]

Hi Mark,

I don't really make that many asynchronous calls, but some of the calls I make carry large byte sequences (around 20-100 megs).

Switching to the synchronous calls seems to have helped (based on my observations). I've ran that method 70 times in a row without error using a data size of about 35 Mb.

Generally speaking my application runs at most three async operations at the same time.

Thanks for your help. Why would switching to synchronous calls fix this issue?

[spookstaz]

Hi Mark,

Too bad it happened again, this time with the synchronous version - my client application received a MemoryLimitException

However the crazy thing is that my ice.log only contains the following:

-! 1/5/2012 15:52:53:413 SMS-Client.exe: warning: connection exception:
   Ice.ConnectionLostException
       error = 0
      at IceInternal.TcpTransceiver.finishRead(Buffer buf)
      at Ice.ConnectionI.finishAsync(Int32 operation)
   Caused by: System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
      at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
      at IceInternal.TcpTransceiver.finishRead(Buffer buf)
   local address = 10.0.8.72:52597
   remote address = 10.0.9.32:10000
-! 1/5/2012 15:52:53:477 SMS-Client.exe: warning: connection exception:
   Ice.ConnectionLostException
       error = 0
      at IceInternal.TcpTransceiver.finishRead(Buffer buf)
      at Ice.ConnectionI.finishAsync(Int32 operation)
   Caused by: System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
      at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
      at IceInternal.TcpTransceiver.finishRead(Buffer buf)
   local address = 10.0.8.72:57858
   remote address = 10.0.9.32:10000