[zeromq-dev] Problem: CLIENT/SERVER types cannot send multi-part messages.

Pieter Hintjens ph at imatix.com
Sat Sep 12 13:16:40 CEST 2015

We've had a discussion on the list, some time ago when I made that
"multipart is dead!" blog posting... of course as many people pointed
out, it's an essential feature.

Deprecating the request-reply pattern is going to take time. No hurry.
I like the idea of an atomic API scatter/gather send. This also
removes that bogus implication that multipart is somehow a form of

On Sat, Sep 12, 2015 at 1:09 AM, Doron Somech <somdoron at gmail.com> wrote:
> I think it is a good solution to the problem. We can continue and support
> old API which will not be thread safe and have thread safe methods for
> multipart messages. We then can drop those tests which already problematic.
> Binding will have some work adapting to new safe API.
> On Sep 11, 2015 4:46 PM, "Joe McIlvain" <joe.eli.mac at gmail.com> wrote:
>> Solution: Add new libzmq API for sending and receiving multi-part messages
>> atomically/statelessly.
>> ---
>> With this commit, the REQ, REP, DEALER, and ROUTER types are marked as
>> soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq
>> master:
>> https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd
>> This email is not about fighting the deprecation, but I do think that one
>> specific deficit in the current implementation of the CLIENT and SERVER
>> types: the lack of multi-part messages.
>> I think the lack of multi-part message support will limit the usefulness
>> of the CLIENT and SERVER types and as a consequence make it harder to
>> encourage adoption.  Also, I think multi-part messages are part of what
>> makes ZeroMQ incredibly convenient for rapid development of useful
>> applications.
>> Furthermore, the lack of multi-part messages seems entirely inspired by
>> the difficulty of creating thread-safe sockets within the C/C++
>> implementation (libzmq).  This is because multi-part messages are sent using
>> the ZMQ_RCVMORE and ZMQ_SNDMORE flags, making the process a stateful one.
>> However, it strikes me that many high level bindings and ports are already
>> conceptually covering this statefulness with a stateless abstraction in
>> which multi-part messages are contained in a single object and sent with a
>> single API call (for example, zmsg in CZMQ contains multiple zframes).  In
>> such high level bindings and ports, removing multi-part messaging is quite
>> an artificial restriction, with no basis in conceptual technical necessity.
>> Because many such abstractions exist around the concept of multi-part
>> messages as a single object, both in bindings/ports and in applications,
>> removing support for multi-part messages in new socket types and deprecating
>> the old types that supports them creates a cascading effect of rewrites that
>> is likely to frustrate many maintainers across the spectrum.  So, to be
>> sure, modern technologies will always undergo major changes if they wish to
>> remain modern, so this is not necessarily a bad thing, but make no mistake
>> that this design decision is quite a significant change in  the ZeroMQ
>> ecosystem, even though it has been executed so far with minimal impact to
>> the API of libzmq.
>> I'd like to take a moment to consider the alternative - what if we could
>> make a more significant change to the API of libzmq in order to make a much
>> smaller impact on the broader ZeroMQ ecosystem, including the APIs of
>> bindings and ports as well as the multitude of applications?  What if we can
>> have both thread-safe sockets and keep multi-part messages - with the
>> requirement that bindings and applications desiring thread-safety use the
>> new atomic API for multi-part messages?  For existing applications using the
>> existing ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could
>> continue to use this (probably deprecated) API, but not if they wanted to
>> use the thread-safe socket types.  This is already the case - the new
>> ZMQ_CLIENT type will simply drop any messages with the ZMQ_SNDMORE flag
>> (https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55),
>> meaning that we're no worse off in this way.
>> I'm arguing that such an API, if done right would be:
>> - Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would
>> still exist, but probably be deprecated)
>> - Less of an impact on the broader ZeroMQ community than the current
>> design direction of deprecating multi-part messaging.
>> - Useful for making threadsafe implementations of the other patterns such
>> as the publish-subscribe pattern.
>> So far I fail to see any disadvantages that outweigh those advantages.
>> ---
>> I'd like this email thread to be about discussing the design of such an
>> API.
>> If we can come up with a coherent plan, I'd be happy to working on the
>> implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea
>> pigs they already are.
>> One way to implement such an API that comes to mind, is with an approach
>> that mimics the `readv` and `writev` functions used in scatter/gather I/O
>> (http://linux.die.net/man/2/writev).  Such an API call in ZeroMQ could
>> accept a pointer and count for an array of pointers to `zmq_msg_t` objects,
>> instead of a pointer and count for an array of pointers to byte buffers.
>> Alternatively, a different struct definition other than `zmq_msg_t` could be
>> created that holds the pointer and count for an array of pointers to byte
>> buffers.  I think the first approach has a smaller API impact, but the
>> second approach is more elegant.  Both approaches would be preferable to
>> dropping support for multi-part messages.
>> Thoughts?
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

More information about the zeromq-dev mailing list