[zeromq-dev] Problem: CLIENT/SERVER types cannot send multi-part messages.

Maurice Barnum msb at yahoo-inc.com
Sat Sep 12 07:31:12 CEST 2015


 blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important;  padding-left:1ex !important; background-color:white !important; }  what about enhancing zmq_sendiov to do the right thing for these new sockets and then remove the experimental label when it works?

zmq_sendv(const zmq_msg_t*, int n)
that did the right thing would be nicer for some use cases but I think there could be some benefits in experimenting with an existing "experimental" interface first. 




On Friday, September 11, 2015, 16:09, Doron Somech <somdoron at gmail.com> wrote:


I think it is a good solution to the problem. We can continue and support old API which will not be thread safe and have thread safe methods for multipart messages. We then can drop those tests which already problematic. Binding will have some work adapting to new safe API.
On Sep 11, 2015 4:46 PM, "Joe McIlvain" <joe.eli.mac at gmail.com> wrote:

Solution: Add new libzmq API for sending and receiving multi-part messages atomically/statelessly.

---

With this commit, the REQ, REP, DEALER, and ROUTER types are marked as soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq master:
https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd

This email is not about fighting the deprecation, but I do think that one specific deficit in the current implementation of the CLIENT and SERVER types: the lack of multi-part messages.

I think the lack of multi-part message support will limit the usefulness of the CLIENT and SERVER types and as a consequence make it harder to encourage adoption.  Also, I think multi-part messages are part of what makes ZeroMQ incredibly convenient for rapid development of useful applications.

Furthermore, the lack of multi-part messages seems entirely inspired by the difficulty of creating thread-safe sockets within the C/C++ implementation (libzmq).  This is because multi-part messages are sent using the ZMQ_RCVMORE and ZMQ_SNDMORE flags, making the process a stateful one.  However, it strikes me that many high level bindings and ports are already conceptually covering this statefulness with a stateless abstraction in which multi-part messages are contained in a single object and sent with a single API call (for example, zmsg in CZMQ contains multiple zframes).  In such high level bindings and ports, removing multi-part messaging is quite an artificial restriction, with no basis in conceptual technical necessity.

Because many such abstractions exist around the concept of multi-part messages as a single object, both in bindings/ports and in applications, removing support for multi-part messages in new socket types and deprecating the old types that supports them creates a cascading effect of rewrites that is likely to frustrate many maintainers across the spectrum.  So, to be sure, modern technologies will always undergo major changes if they wish to remain modern, so this is not necessarily a bad thing, but make no mistake that this design decision is quite a significant change in  the ZeroMQ ecosystem, even though it has been executed so far with minimal impact to the API of libzmq.

I'd like to take a moment to consider the alternative - what if we could make a more significant change to the API of libzmq in order to make a much smaller impact on the broader ZeroMQ ecosystem, including the APIs of bindings and ports as well as the multitude of applications?  What if we can have both thread-safe sockets and keep multi-part messages - with the requirement that bindings and applications desiring thread-safety use the new atomic API for multi-part messages?  For existing applications using the existing ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could continue to use this (probably deprecated) API, but not if they wanted to use the thread-safe socket types.  This is already the case - the new ZMQ_CLIENT type will simply drop any messages with the ZMQ_SNDMORE flag (https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55), meaning that we're no worse off in this way.

I'm arguing that such an API, if done right would be:
- Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would still exist, but probably be deprecated)
- Less of an impact on the broader ZeroMQ community than the current design direction of deprecating multi-part messaging.
- Useful for making threadsafe implementations of the other patterns such as the publish-subscribe pattern.

So far I fail to see any disadvantages that outweigh those advantages.

---

I'd like this email thread to be about discussing the design of such an API.

If we can come up with a coherent plan, I'd be happy to working on the implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea pigs they already are.

One way to implement such an API that comes to mind, is with an approach that mimics the `readv` and `writev` functions used in scatter/gather I/O (http://linux.die.net/man/2/writev).  Such an API call in ZeroMQ could accept a pointer and count for an array of pointers to `zmq_msg_t` objects, instead of a pointer and count for an array of pointers to byte buffers.  Alternatively, a different struct definition other than `zmq_msg_t` could be created that holds the pointer and count for an array of pointers to byte buffers.  I think the first approach has a smaller API impact, but the second approach is more elegant.  Both approaches would be preferable to dropping support for multi-part messages.

Thoughts?

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

 
   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150912/6a8161be/attachment.htm>


More information about the zeromq-dev mailing list