[zeromq-dev] Multi-part messages

Brian Granger ellisonbg at gmail.com
Sat Mar 27 17:52:16 CET 2010


> I've implemented "multi-part message" functionality as it was discussed
> on the mailing list lately.

Fantastic, this is great.

> The patch was committed to the trunk and can be used this way:
> zmq::message_t part1 (100);
> s.send (part1, ZMQ_TBC); //  to be continued...
> zmq::message_t part2 (100);
> s.send (part2);

How does the receiver detect message group boundaries?

> Multi-part message behaves as a simply message. Specifically, it's
> atomic, meaning that you'll either get all parts or nothing. Also
> individual parts of the message cannot be torn apart by load balancing,
> fair queueing etc.


> Another use case is adding structure to the message. Message part
> boundaries are preserved as message passes the network, thus they can be
> used to delimit semantically separate parts of the message. This use
> case is going to be used extensively by 0MQ itself in the future
> (separating routing info from user data etc.)

I guess this means the routing string of XREP will be handled in a
multipart.  What will the algorithm be to cleanly separate the 0MQ
usage of multipart from the user's usage of it?  Will 0MQ internally
add and remove message parts so that the user only sees the
application message parts at the endpoints?

Another question: how do multipart message interact with topic
filtering.  Is topic filtering done on each message in a group
separately?  On the entire group?  Is the topic the first message in
the group?

One thing we are running into with Python is that if we recv a large
message that has a topic and routing info (XREP) we get the result as
a single large message.  Because Python strings are immutable, we have
to make expensive copies to extract the message.  It would be great if
we could use multipart to get the tag, routing info and message in
different recv's.

> The patch is quite complex and thus it would benefit from testing.

I will write a test suite in pyzmq for this.

> Also note that XREQ and XREP sockets are rendered unfunctional by the
> patch and will be rewritten shortly.

We are using XREQ and XREP extensively in a real app now, so this will
slow us a bit.  Other than handling the routing info as using message
parts, will the functionality of XREP/XREQ change?



> Martin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com

More information about the zeromq-dev mailing list