[zeromq-dev] PUSH/PULL HWM documentation

[James Cipar]

I think I'm confused about how PUSH/PULL sockets work.  Specifically about how messages are routed to different PULL sockets connected to a single PUSH.  My impression with 0MQ PUSH/PULL (based on the docs and my test program that I sent out) was that it basically does this internally.  Every consumer has a buffer.  When the sender gets a message to send, it picks the first consumer (in a round-robin fashion) whose buffer is not full.  If all consumer's buffers are full, the sender blocks until one of them is no longer full.  Is this incorrect?

 


On Oct 5, 2011, at 10:08 AM, Pieter Hintjens wrote:

> On Tue, Oct 4, 2011 at 8:32 PM, James Cipar <jcipar at cmu.edu> wrote:
> 
>> Hi.  I found something that may be an error in the ZMQ documentation regarding "load balancing" with PUSH/PULL sockets.  At best, it's pretty confusing.
> 
> This is true. "Load-balancing" is inaccurate. I've fixed the doc for
> 2-1 and 3-0, and am updating the api site.
> 
> Using the HWM to control load is the wrong approach; this does not
> accurately or reliably map to consumer activity.

What do you mean by this?  Doesn't it map to consumer activity in that a slow consumer will cause its buffer to fill up, so no more data will be sent to it?

The behavior I am trying to replicate is a blocking queue, where a thread can add items to the queue, and workers pull items off one-by-one.  In this setup the slow worker will get at most one item from the queue, because other items will be consumed by the fast worker.

At this stage it would be pretty trivial to swap out the inproc sockets for a normal C++ blocking queue implementation, but I was trying to see if I could do it "the 0MQ way" first, so that it would work even if I decide to split the program up into separate processes with separate address spaces.


> You need to use an
> application-level pattern to ensure work is sent as you want. The
> Guide explains this starting with the LRU pattern. A more flexible
> approach is to use credit-based flow control such as explained here:
> http://unprotocols.org/blog:15
> 

I don't think I understand the difference between credit-based flow control and what 0MQ currently does.  If I'm reading this correctly, in credit-based flow control each consumer has a fixed sized buffer for incoming messages.  When it connects it notifies the sender of the number of slots in its buffer.

The sender gives the consumer an initial number of credits equal to the size of its buffer.  Every time the sender sends a message to the consumer it decrements the number of credits.  Every time a consumer processes a message it notifies the sender to increment its number of credits.  Messages are sent to any consumer with outstanding credits.  Messages are never sent to a consumer with 0 credits; if all consumers have 0 credits the sender will block until a consumer increments its credit count.

How are 0MQ PUSH/PULL sockets different?


> For multithreading, I can recommend CZMQ's thread package, which is
> simple and designed for 0MQ apps: http://czmq.zeromq.org.
> 

The rest of the application is already using boost::thread.  Adding a second threading library isn't really an option.  My comment the boost threading primitives was mostly because there is no semaphore implementation (or queue implementation).  That's why my example code included that "Counter" class.


> -Pieter
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>