[zeromq-dev] multiplexer pattern

MinRK benjaminrk at gmail.com
Fri Sep 16 22:13:34 CEST 2011


Hello,

We have been using a pattern in IPython that is essentially a multiplexer -
connecting one-to-many DEALERs to one-to-many ROUTERs via a central
ROUTER-ROUTER device, that functions similarly to a direct ROUTER-ROUTER
connection.

players:
* 1 (or conceivably more) mux devices, which binds on two ports - input and
output
* 0-many clients connected to the input of the mux
* 1-many engines connected to the output of the mux
* separate Hub that keeps track of state, so that clients can know how to
send messages to particular engines.  Everybody can talk to the hub.

The pattern is that clients make requests of particular engines by identity,
and engines ultimately reply back to the originating client.

Essentially, we have many clients that want to submit requests to particular
engines that are behind the multiplexer.  This is currently implemented with
a ROUTER-ROUTER device
that simply swaps the ordering of the first two parts of any message
(otherwise identical to the 2.1.x QUEUE device).  The workers tell the Hub
their IDENTITY on startup, so that clients can know what to use as a prefix.

Simplified Python mux implementation and client/worker sample here:
https://gist.github.com/1222903, which works with 2.1.x/3.0 ROUTER/DEALER
sockets.  In reality, we use the MonitoredQueue object in pyzmq.

What I want to know is how we will be able to do this in 4.0, now that
IDENTITY has been removed (or even in 3.0 with XREQ/XREP sockets that ignore
the IDENTITY sockopt).  Specifically, who has the information necessary for
a client to construct a message that will be relayed to the desired
endpoint?

Any variation on socket types is fine, as long as the following criteria are
met:

* clients can make requests of particular engines
* engine replies propagate to requesting client
* engines and clients can come and go over time (message loss for vanished
endpoints is fine)
* clients and engines may not bind, and only connect to the mux and Hub (mux
also connects to hub)

IDENTITY made this pattern *extremely* easy, and I don't see the right way
to build a MUX in its absence.

Thanks for any guidance.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110916/09893f0c/attachment.htm>


More information about the zeromq-dev mailing list