[zeromq-dev] Multi-part messages
Brian Granger
ellisonbg at gmail.com
Sat Mar 27 18:46:43 CET 2010
Martin,
> No API for it now. I have no sane idea how the API should look like :(
>
> As an workaround you can check the flag by hand:
>
> zmq_msg_t msg;
> ...
> int tbc = msg.flags & ZMQ_MSG_TBC ? 1 : 0;
I guess this is the API then :) I will think about how to represent
this in Python.
>> I guess this means the routing string of XREP will be handled in a
>> multipart. What will the algorithm be to cleanly separate the 0MQ
>> usage of multipart from the user's usage of it? Will 0MQ internally
>> add and remove message parts so that the user only sees the
>> application message parts at the endpoints?
>
> Yes. 0MQ will do that for you. In multi-hop req/rep scenario each
> intermediate node will add it's ID to the request. Terminal REP node would
> copy the stack of IDs to the reply and send it back. Each intermediate node
> would trim one ID from the reply.
Fantastic.
>> Another question: how do multipart message interact with topic
>> filtering. Is topic filtering done on each message in a group
>> separately? On the entire group? Is the topic the first message in
>> the group?
>
> Only the first part is used for topic matching. Not doing so would result in
> multi-part message not being atomic.
Great, this is exactly what we need!
>> One thing we are running into with Python is that if we recv a large
>> message that has a topic and routing info (XREP) we get the result as
>> a single large message. Because Python strings are immutable, we have
>> to make expensive copies to extract the message. It would be great if
>> we could use multipart to get the tag, routing info and message in
>> different recv's.
>
> Yes, exactly. Multi-part messages should solve the zero-copy problems for
> both 0MQ itself and user applications alike.
Again, great.
>>> The patch is quite complex and thus it would benefit from testing.
>>
>> I will write a test suite in pyzmq for this.
>
> Great!
>
>>> Also note that XREQ and XREP sockets are rendered unfunctional by the
>>> patch and will be rewritten shortly.
>>
>> We are using XREQ and XREP extensively in a real app now, so this will
>> slow us a bit. Other than handling the routing info as using message
>> parts, will the functionality of XREP/XREQ change?
>
> No it won't except for REP socket where user currently has to manage the
> routing prefix by hand. It will be handled transparently for the user. And
> it's next on my todo list, so hopefully you can start using it shortly.
OK, this sounds good. I recently added a simple API to pyzmq for
handling the routing prefix by hand, but it would be nice to not have
to do that.
Cheers,
Brian
> Martin
>
--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com
More information about the zeromq-dev
mailing list