[zeromq-dev] Vulnerability of devices to incoming messages

[MinRK]

On Fri, Aug 13, 2010 at 16:24, Martin Sustrik <sustrik at moloch.sk> wrote:

> Hi Brian,
>
> > In my mind, a device is a particular send/recv pattern.  Those
> > send/recv patterns are independent of the socket types.  As long as
> > you use socket types that support the send/recv pattern, it should
> > work.  Locking devices to socket types will mean we have to implement
> > lots of different devices that do the same exactly send/recv pattern.
> >  I don't think that makes sense.
> >
>
> I was always thinking of devices in the same terms as Jon (i.e. device
> is bound to particular messaging pattern). Your idea of device send/recv
> patterns being orthogonal to messaging patterns sounds interesting and
> never occured to me. Can you give some examples?
>

In IPython, we have been thinking of them more explicitly as send/recv
models.

For instance, in our heartbeat monitor system, one of the devices we use is
a Forwarder with a SUB in, and XREQ out, which I'm sure people will consider
pretty bizarre, but it gets us something quite useful.

The model:

(1) Heartbeat Monitor:
PUB sends out a single message (time, or uuid)

(many) Hearts:
Forwarder Device:
SUB receives message
XREQ sends message, prepending it's IDENTITY

Monitor:
XREP receives many messages of the form ['123', 'original_msg' ], and uses
the IDENTITY prefixes to know who is still alive, and the message content to
ensure that they responded with the right message.

The reason we do this is that we need something that will not grab the GIL
in a Python process, so it will still be responsive when Python code in the
same process is blocking, so it must be pure C (or C++).  We could certainly
write our own device, but a ZMQ Forwarder with SUB/XREQ does this just fine
- keeping ms response times while the main Python thread is performing 10s
blocking BLAS operations at 100% CPU.

Similarly, we use a single slightly modified Queue device with both
XREP/XREQ (load balanced) and XREP/XREP (explicitly multiplexed) socket
pairs. This device also has a monitor socket which gets all messages in or
out. Most obviously, this is a PUB socket for broadcasting to parties
interested in what is passing through the Queues, but a user could just as
easily pick XREQ if the handling of queue monitoring wanted to be load
balanced instead of published, or even PAIR, if just to prevent snooping.

There's also the fact that FORWARDER/STREAMER devices don't (at the moment)
preserve ZMQ_MORE for multipart messages, so a Queue could be the device of
choice even if you are just relaying from a PUB socket that broadcasts
multipart messages.

-MinRK


> Martin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20100813/3422824d/attachment.htm>