[zeromq-dev] Explicit patterns

Mark V mvyver at gmail.com
Fri May 28 04:26:31 CEST 2010


On Wed, May 26, 2010 at 4:02 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Pieter,
>
>> I'm writing a guide for 0MQ beginners and I've hit two problems in
>> understanding and explaining 0MQ, which I think are faced by many
>> people before they really know 0MQ well. The first problem is that 0MQ
>> patterns, which are really important building blocks, are not 1st
>> class entities.  A pattern is a set of socket types and devices but
>> those names don't mention patterns at all.  There is no hint of what
>> combinations of socket and device are legal.  We have to learn it all
>> by heart.
>
> This is great, I think a guide would be very helpful.
>
> But, my experience learning 0MQ has been a bit different than the "patterns"
> as building blocks.  In my experience, the building blocks are the sockets
> and the devices solve common design problems that many users run into.
> Also, with the new device API, the possibilities for new device types
> are endless.  Thus, for a beginner guide I would focus on the core socket types,
> including the less common ones that are used in devices.
>
>> Second problem is that the socket types are irregular.  For
>> request-reply they indicate the type of message a node sends.  For
>> pub-sub they are the role of the node.  For paralyzed processing they
>> indicate the flow of data.  For pair they indicate the nature of the
>> connection.
>
> I am not sure I agree with this.  In my understanding, 0MQ sockets
> can be described in a uniform manner by specifying things like:
>
> * Bidirectional or unidirectional
> * send/recv pattern (ssssss, rrrrrrr, srsrsrsr, etc.)
> * Outbound/inbound message queuing pattern (load balanced, fair queued)
> * Number of allowed clients.
> * How multiple clients are handled.
> * Number of allowed in flight messages (synch, async)
> * Algorithm used when the queue fills.
> * Allowed peer socket types.
> * I may be missing certain things.
> * How identities are used in message routing.
>
> The main problem that I see right now is that some of these things are
> not clearly documented.  Making a list of all these things and
> documenting them for each socket type would be immensely helpful and
> clarify the abstraction of a 0MQ socket.
>
> Once the basic sockets are clearly described, devices can be
> introduced as particular combinations of sockets that have additional
> common application logic builtin.  Another way of saying this is that
> a device *is* an application+0MQ sockets that expresses a common 0MQ
> usage case.
>
>> So it seems inevitable that people will misuse socket types and
>> patterns until they learn by error.  We lack a consistent model of
>> what a pattern is and how the names of things are derived.
>
> Again, I think the problem is a lack of complete socket documentation.
>

Up to a point more documentation is good - 'things' need to be
structured in a way that minimizes search costs.
Some users don't want to have to digest documentation unrelated to
their problem.  More likely is they want to, but don't have the time
to do so.
I thought Pieter's naming assisted in this regard.
I also don't see how the suggested naming makes documenting all
behaviors more difficult/obscure.
It _could_ mean that some content is duplicated, but this seems more
of an issue about how documentation is structured (e.g. links to
common behavior descriptions, or replicated descriptions) and
generated.

>> Here's a proposal that will fix this, we think.  We being Mato and
>> myself.  It's going to hurt but it's arguably better now than later.
>> We propose these consistent rules:
>>
>> * All socket types and devices names contain the pattern identifier.
>> We suggest RR, PS, BF, and EP for request-response, pub-sub,
>> butterfly, and exclusive pair respectively.
>> * Socket and device types are named ZMQ_XX_YYYY where XX is the
>> pattern identifier and YYYY is the socket or device type.
>> * Socket types always reflect the role of the node, period.
>
> This seems really complicated, especially in that it still doesn't
> answer all the questions that users will have about how 0MQ sockets
> function.

I think this depends on how tight your time/effort constraints are.
Initially some users may only be interested in the behavior of sockets
relevant to their specific use case.
Over time they might be interested in a deeper more general
understanding.  Those descriptions can still exist, they just don't
have to wade through them when they start out.

> It also suggests that the number of devices is small and
> fixed.

That this is not the case can be emphasized.

> My own vision is that in the long run there will be many
> devices that 0MQ users create for their own purposes.  This also
> leaves out certain usage cases like XREP+REQ and REP+XREQ.

Good point.
Is the idea to have suggestive/informative names for all possible
patterns/use-cases or just some 'common/beginners' subset?

>
> Summary: if the goal is to help beginning users, this would only confuse them.
>

Perhaps the difference lies in who the beginner is and what this piece
of documentation is trying to help this beginner do:
a) understand how 0MQ can address their problem (the user is a
'some-domain' specialist, messaging (and 0MQ) novice)?
b) understand how 0MQ can address general messaging problems (the user
is messaging specialist, 0MQ novice)?

>> This gives us (as example, the details may change):
>>
>> * ZMQ_RR_CLIENT, ZMQ_RR_SERVER for sockets, ZMQ_RR_QUEUE for device.
>> * ZMQ_PS_PUBLISHER, ZMQ_PS_SUBSCRIBER for sockets, ZMQ_PS_FORWARD for device.
>> * ZMQ_BF_CLIENT, ZMQ_WORKER, ZMQ_COLLECTOR for sockets,
>> ZMQ_BF_STREAMER for device.
>> * ZMQ_EP_PEER for sockets.
>>
>> Incidental change proposals are to rename parallelized process as
>> "butterfly" and pair as "exclusive pair" to give these patterns more
>> friendly names.
>>
>> Comments, thoughts?
>
> I agree that certain aspects of learning 0MQ are somewhat difficult.
> But I think it is mostly because of a lack of documentation and the
> fact that the API is so simple, but powerful.

I think I assumed Pieter proposed descriptive/informative names to
allow new users to focus their attention/time/effort on particular use
cases/patterns that hopefully are a close match to a specific problem.
I assumed a more detailed documentation 'superset' would exist for
those users interested in broader problems, as they invested more time
into 0MQ.

Cheers
Mark

>
> Cheers,
>
> Brian
>
>> -Pieter
>>
>> (Also posted at http://www.zeromq.org/draft:explicit-patterns)
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



More information about the zeromq-dev mailing list