[zeromq-dev] Resource puzzle with the Divide and Conquer (Ventilator->Workers->Sink) Example

[Mark Hotchkiss]

Pieter:

With 1024 jobs the Sink application freezes at 19790.

You have asked me to strip down the example before and I don't know how without deviating from your explicit approach. This is your code from page 17 of your book. The only changes I have made is to include variable arguments to adjust the hard coded values that you have for the following items:

1. Number of jobs sent from the Ventilator which is the same value of confirmations to the Sink.
2. Sleep time of the Worker
3. Message size sent from the Ventilator to the Workers. We construct a test Message using the following additional code:

for (task_nbr = 0; task_nbr < numberOfJobs; task_nbr++) {

	// Stuff a character array with a bunch of AAAAA
	// characters of quantity based on messageSize
	unsigned char *p = (unsigned char *) malloc(
	messageSize * sizeof(unsigned char));
	 memset((void *) p, 'A', messageSize);
	 p[messageSize - 1] = '\0';

	// Send the message and reclaim memory
	 s_send(sender, p);
	 free(p);
}

If you compare each of the ZMQ specific calls in the code, they are the same as your example. I am also using your zhelpers.h intact according to  your recommendations. If you can point out any specific deviation I am making from your instructive patterns I would be glad to correct them, but I have copied this code from your source repository referenced in the book.

According to your documentation, ZMQ is supposed to handle "blobs of any size, from zero upwards, that fit in memory." I also see the following under Messaging Patterns in Chapter 2: "It manages these queues carefully to ensure processes don't run out of memory, over-flowing to disk when appropriate".

That is not what we are experiencing with your provided example. We experience "Killed" or frozen processes when the message gets too large. We experience freezes when we run a large number of processes (somewhere just over 1000), which appear to be from memory starvation caused by the Sink (using Linux "top" command). The hardware we are running this on has 128 Gibabytes of installed memory.

Any help you can offer with this example would be appreciated. 

Thanks.
On Wednesday, April 30, 2014 3:54 AM, Pieter Hintjens <ph at imatix.com> wrote:
 
On Wed, Apr 30, 2014 at 1:09 AM, Mark Hotchkiss
<mark.hotchkiss at yahoo.com> wrote:

> If I push the number of Worker processes to just 1025, the Sink process
> freezes at 19770 jobs and eats up all of the memory available.

1025 is a suspect number. Does it work with 1024 workers?

A good approach is to strip down the test case until it works, and
thus figure out the critical issue.


-Pieter
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140430/5ad77a10/attachment.htm>