[zeromq-dev] Multi-part messages
ellisonbg at gmail.com
Sat Mar 27 18:46:43 CET 2010
> No API for it now. I have no sane idea how the API should look like :(
> As an workaround you can check the flag by hand:
> zmq_msg_t msg;
> int tbc = msg.flags & ZMQ_MSG_TBC ? 1 : 0;
I guess this is the API then :) I will think about how to represent
this in Python.
>> I guess this means the routing string of XREP will be handled in a
>> multipart. What will the algorithm be to cleanly separate the 0MQ
>> usage of multipart from the user's usage of it? Will 0MQ internally
>> add and remove message parts so that the user only sees the
>> application message parts at the endpoints?
> Yes. 0MQ will do that for you. In multi-hop req/rep scenario each
> intermediate node will add it's ID to the request. Terminal REP node would
> copy the stack of IDs to the reply and send it back. Each intermediate node
> would trim one ID from the reply.
>> Another question: how do multipart message interact with topic
>> filtering. Is topic filtering done on each message in a group
>> separately? On the entire group? Is the topic the first message in
>> the group?
> Only the first part is used for topic matching. Not doing so would result in
> multi-part message not being atomic.
Great, this is exactly what we need!
>> One thing we are running into with Python is that if we recv a large
>> message that has a topic and routing info (XREP) we get the result as
>> a single large message. Because Python strings are immutable, we have
>> to make expensive copies to extract the message. It would be great if
>> we could use multipart to get the tag, routing info and message in
>> different recv's.
> Yes, exactly. Multi-part messages should solve the zero-copy problems for
> both 0MQ itself and user applications alike.
>>> The patch is quite complex and thus it would benefit from testing.
>> I will write a test suite in pyzmq for this.
>>> Also note that XREQ and XREP sockets are rendered unfunctional by the
>>> patch and will be rewritten shortly.
>> We are using XREQ and XREP extensively in a real app now, so this will
>> slow us a bit. Other than handling the routing info as using message
>> parts, will the functionality of XREP/XREQ change?
> No it won't except for REP socket where user currently has to manage the
> routing prefix by hand. It will be handled transparently for the user. And
> it's next on my todo list, so hopefully you can start using it shortly.
OK, this sounds good. I recently added a simple API to pyzmq for
handling the routing prefix by hand, but it would be nice to not have
to do that.
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com
More information about the zeromq-dev