[zeromq-dev] Explicit patterns

Brian Granger ellisonbg at gmail.com
Tue May 25 20:02:32 CEST 2010


Pieter,

> I'm writing a guide for 0MQ beginners and I've hit two problems in
> understanding and explaining 0MQ, which I think are faced by many
> people before they really know 0MQ well. The first problem is that 0MQ
> patterns, which are really important building blocks, are not 1st
> class entities.  A pattern is a set of socket types and devices but
> those names don't mention patterns at all.  There is no hint of what
> combinations of socket and device are legal.  We have to learn it all
> by heart.

This is great, I think a guide would be very helpful.

But, my experience learning 0MQ has been a bit different than the "patterns"
as building blocks.  In my experience, the building blocks are the sockets
and the devices solve common design problems that many users run into.
Also, with the new device API, the possibilities for new device types
are endless.  Thus, for a beginner guide I would focus on the core socket types,
including the less common ones that are used in devices.

> Second problem is that the socket types are irregular.  For
> request-reply they indicate the type of message a node sends.  For
> pub-sub they are the role of the node.  For paralyzed processing they
> indicate the flow of data.  For pair they indicate the nature of the
> connection.

I am not sure I agree with this.  In my understanding, 0MQ sockets
can be described in a uniform manner by specifying things like:

* Bidirectional or unidirectional
* send/recv pattern (ssssss, rrrrrrr, srsrsrsr, etc.)
* Outbound/inbound message queuing pattern (load balanced, fair queued)
* Number of allowed clients.
* How multiple clients are handled.
* Number of allowed in flight messages (synch, async)
* Algorithm used when the queue fills.
* Allowed peer socket types.
* I may be missing certain things.
* How identities are used in message routing.

The main problem that I see right now is that some of these things are
not clearly documented.  Making a list of all these things and
documenting them for each socket type would be immensely helpful and
clarify the abstraction of a 0MQ socket.

Once the basic sockets are clearly described, devices can be
introduced as particular combinations of sockets that have additional
common application logic builtin.  Another way of saying this is that
a device *is* an application+0MQ sockets that expresses a common 0MQ
usage case.

> So it seems inevitable that people will misuse socket types and
> patterns until they learn by error.  We lack a consistent model of
> what a pattern is and how the names of things are derived.

Again, I think the problem is a lack of complete socket documentation.

> Here's a proposal that will fix this, we think.  We being Mato and
> myself.  It's going to hurt but it's arguably better now than later.
> We propose these consistent rules:
>
> * All socket types and devices names contain the pattern identifier.
> We suggest RR, PS, BF, and EP for request-response, pub-sub,
> butterfly, and exclusive pair respectively.
> * Socket and device types are named ZMQ_XX_YYYY where XX is the
> pattern identifier and YYYY is the socket or device type.
> * Socket types always reflect the role of the node, period.

This seems really complicated, especially in that it still doesn't
answer all the questions that users will have about how 0MQ sockets
function.  It also suggests that the number of devices is small and
fixed.  My own vision is that in the long run there will be many
devices that 0MQ users create for their own purposes.  This also
leaves out certain usage cases like XREP+REQ and REP+XREQ.

Summary: if the goal is to help beginning users, this would only confuse them.

> This gives us (as example, the details may change):
>
> * ZMQ_RR_CLIENT, ZMQ_RR_SERVER for sockets, ZMQ_RR_QUEUE for device.
> * ZMQ_PS_PUBLISHER, ZMQ_PS_SUBSCRIBER for sockets, ZMQ_PS_FORWARD for device.
> * ZMQ_BF_CLIENT, ZMQ_WORKER, ZMQ_COLLECTOR for sockets,
> ZMQ_BF_STREAMER for device.
> * ZMQ_EP_PEER for sockets.
>
> Incidental change proposals are to rename parallelized process as
> "butterfly" and pair as "exclusive pair" to give these patterns more
> friendly names.
>
> Comments, thoughts?

I agree that certain aspects of learning 0MQ are somewhat difficult.
But I think it is mostly because of a lack of documentation and the
fact that the API is so simple, but powerful.

Cheers,

Brian

> -Pieter
>
> (Also posted at http://www.zeromq.org/draft:explicit-patterns)
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com



More information about the zeromq-dev mailing list