[zeromq-dev] Problem: CLIENT/SERVER types cannot send multi-part messages.

Michael Haberler mail17 at mah.priv.at
Sat Sep 12 15:24:35 CEST 2015


> Am 12.09.2015 um 13:16 schrieb Pieter Hintjens <ph at imatix.com>:
> 
> We've had a discussion on the list, some time ago when I made that
> "multipart is dead!" blog posting... of course as many people pointed
> out, it's an essential feature.
> 
> Deprecating the request-reply pattern is going to take time. No hurry.
> I like the idea of an atomic API scatter/gather send. This also
> removes that bogus implication that multipart is somehow a form of
> streaming.

Amen. Bury ZMQ_MORE eventually, but not multipart. That'd be throwing out the baby with the bathwater.

- Michael

> 
> 
> 
> On Sat, Sep 12, 2015 at 1:09 AM, Doron Somech <somdoron at gmail.com> wrote:
>> I think it is a good solution to the problem. We can continue and support
>> old API which will not be thread safe and have thread safe methods for
>> multipart messages. We then can drop those tests which already problematic.
>> Binding will have some work adapting to new safe API.
>> 
>> On Sep 11, 2015 4:46 PM, "Joe McIlvain" <joe.eli.mac at gmail.com> wrote:
>>> 
>>> Solution: Add new libzmq API for sending and receiving multi-part messages
>>> atomically/statelessly.
>>> 
>>> ---
>>> 
>>> With this commit, the REQ, REP, DEALER, and ROUTER types are marked as
>>> soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq
>>> master:
>>> 
>>> https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd
>>> 
>>> This email is not about fighting the deprecation, but I do think that one
>>> specific deficit in the current implementation of the CLIENT and SERVER
>>> types: the lack of multi-part messages.
>>> 
>>> I think the lack of multi-part message support will limit the usefulness
>>> of the CLIENT and SERVER types and as a consequence make it harder to
>>> encourage adoption.  Also, I think multi-part messages are part of what
>>> makes ZeroMQ incredibly convenient for rapid development of useful
>>> applications.
>>> 
>>> Furthermore, the lack of multi-part messages seems entirely inspired by
>>> the difficulty of creating thread-safe sockets within the C/C++
>>> implementation (libzmq).  This is because multi-part messages are sent using
>>> the ZMQ_RCVMORE and ZMQ_SNDMORE flags, making the process a stateful one.
>>> However, it strikes me that many high level bindings and ports are already
>>> conceptually covering this statefulness with a stateless abstraction in
>>> which multi-part messages are contained in a single object and sent with a
>>> single API call (for example, zmsg in CZMQ contains multiple zframes).  In
>>> such high level bindings and ports, removing multi-part messaging is quite
>>> an artificial restriction, with no basis in conceptual technical necessity.
>>> 
>>> Because many such abstractions exist around the concept of multi-part
>>> messages as a single object, both in bindings/ports and in applications,
>>> removing support for multi-part messages in new socket types and deprecating
>>> the old types that supports them creates a cascading effect of rewrites that
>>> is likely to frustrate many maintainers across the spectrum.  So, to be
>>> sure, modern technologies will always undergo major changes if they wish to
>>> remain modern, so this is not necessarily a bad thing, but make no mistake
>>> that this design decision is quite a significant change in  the ZeroMQ
>>> ecosystem, even though it has been executed so far with minimal impact to
>>> the API of libzmq.
>>> 
>>> I'd like to take a moment to consider the alternative - what if we could
>>> make a more significant change to the API of libzmq in order to make a much
>>> smaller impact on the broader ZeroMQ ecosystem, including the APIs of
>>> bindings and ports as well as the multitude of applications?  What if we can
>>> have both thread-safe sockets and keep multi-part messages - with the
>>> requirement that bindings and applications desiring thread-safety use the
>>> new atomic API for multi-part messages?  For existing applications using the
>>> existing ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could
>>> continue to use this (probably deprecated) API, but not if they wanted to
>>> use the thread-safe socket types.  This is already the case - the new
>>> ZMQ_CLIENT type will simply drop any messages with the ZMQ_SNDMORE flag
>>> (https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55),
>>> meaning that we're no worse off in this way.
>>> 
>>> I'm arguing that such an API, if done right would be:
>>> - Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would
>>> still exist, but probably be deprecated)
>>> - Less of an impact on the broader ZeroMQ community than the current
>>> design direction of deprecating multi-part messaging.
>>> - Useful for making threadsafe implementations of the other patterns such
>>> as the publish-subscribe pattern.
>>> 
>>> So far I fail to see any disadvantages that outweigh those advantages.
>>> 
>>> ---
>>> 
>>> I'd like this email thread to be about discussing the design of such an
>>> API.
>>> 
>>> If we can come up with a coherent plan, I'd be happy to working on the
>>> implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea
>>> pigs they already are.
>>> 
>>> One way to implement such an API that comes to mind, is with an approach
>>> that mimics the `readv` and `writev` functions used in scatter/gather I/O
>>> (http://linux.die.net/man/2/writev).  Such an API call in ZeroMQ could
>>> accept a pointer and count for an array of pointers to `zmq_msg_t` objects,
>>> instead of a pointer and count for an array of pointers to byte buffers.
>>> Alternatively, a different struct definition other than `zmq_msg_t` could be
>>> created that holds the pointer and count for an array of pointers to byte
>>> buffers.  I think the first approach has a smaller API impact, but the
>>> second approach is more elegant.  Both approaches would be preferable to
>>> dropping support for multi-part messages.
>>> 
>>> Thoughts?
>>> 
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> 
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev




More information about the zeromq-dev mailing list