[zeromq-dev] Notes from a hackathon

Doron Somech somdoron at gmail.com
Mon Feb 2 17:55:11 CET 2015


Joe the problem with the api of zmsg for single frame is that you cannot
calculate the frame size in advance causing a lot of re allocation and
copying data around (it's ok for receiving, the problem is sending).



On Mon, Feb 2, 2015 at 6:35 PM, Joe McIlvain <joe.eli.mac at gmail.com> wrote:

> In my opinion, maintaining some kind of multi-frame abstraction at the API
> level is in fact very useful to applications, even if they are sent
> atomically on the implementation level as you describe.
>
> For example, in CZMQ It's quite useful in program flow to be able to push
> or pop a single zframe onto or off of a zmsg, then send that zmsg as an
> entire message.  Rarely do I find myself using zframe_send or zframe_recv
> with the "send more" and "receive more" flags in a CZMQ application.  The
> CZMQ "picture" messages are nice, but they are not always the easiest API
> to integrate into an application.  Having zmsg and zframe remain (or
> something like them), while removing zframe_send and zframe_recv could ease
> applications relying on multi-frame semantics into the transition.  The
> implementation for zmsg would simply serialize the frames in some
> zmsg-determined way into a single-part zmq_msg to send and deserialize
> according to the same scheme to receive.  This multi-frame serialization
> essentially be at the application level rather than at the ZMTP level.
>
> In short, I think a multi-frame abstraction is important for applications,
> but if we remove the ability to send one frame at a time at the fundamental
> level, this abstraction can remain at a higher level (whether in libzmq, or
> in CZMQ and other wrappers) and coexist happily with the proposed changes.
>
>
>
> On Mon, Feb 2, 2015 at 7:37 AM, Doron Somech <somdoron at gmail.com> wrote:
>
>> So the new sockets are in the github master, you can take a loot at the
>> test to see how to use the new routing id field:
>>
>> https://github.com/zeromq/libzmq/blob/master/tests/test_client_server.cpp
>>
>> Few of the reasons I didn't like multi frames:
>> * I tried in the past to make both zeromq and NetMQ thread safe, think of
>> sending from multiple threads or receiving from multiple threads. we cannot
>> do that with Multipart.
>> * There are a lot of good transport that already implement message
>> framing (UDP, WebSockets, SCTP and even HTTP), but because zeromq required
>> it is own framing it was not easy to add them.
>> * Multipart, router usage (routing id frame) and not being thread safe
>> make the learning curve of zeromq hard to beginners. Without three of them
>> zeromq become much simpler.
>> * At least with NetMQ single part message is much faster than multipart.
>> * New stacks, multipart is complicated to implement, with the new API it
>> will much more easier to implement new stacks (like JSMQ) or any native
>> stack.
>>
>> Pieter I'm looking forward to see how you expose the routing id in the
>> czmq API.
>> I also like the czmq API for sending mutlipart messages (the picture
>> feature) so maybe we can use that to generate single frame which is also
>> compatible with zproto.
>>
>> About the implementation, none of new sockets support any option now.
>> server behave like mandatory router, so when router is not reachable or
>> highwater mark is reached an error will be returned.
>>
>> As ZMTP 3.1 is still in raw status, what do you think of removing the
>> multipart from it? maybe the 3.1 will only support the new socket types.
>>
>> Anyway I really excited about this change.
>>
>>
>>
>>
>>
>> On Mon, Feb 2, 2015 at 4:17 PM, Thomas Rodgers <rodgert at twrodgers.com>
>> wrote:
>>
>>> What we really want IMO is per-peer metadata, and an API to get/set
>>>> this. Using messages is a hack.
>>>
>>>
>>> Currently working on that :)
>>>
>>> Having two layers that both
>>>> carry per-message data is... wrong IMO.
>>>
>>>
>>> Protocols supporting 'out of band' data aren't exactly uncommon.
>>>
>>> However the key thing is, what's the problem. Then we can discuss
>>>> solutions to that.
>>>
>>>
>>> I don't have an immediate use-case justifying it. I noted it, mostly
>>> because it has come up a few times since I started paying attention.
>>>
>>> On Mon, Feb 2, 2015 at 7:55 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>>
>>>> > It seems to me that in order to remove multi-part messages and
>>>> introduce new
>>>> > socket types (e.g. SERVER/CLIENT) that would
>>>> > necessitate a revision of the wire protocol. If we are going to do
>>>> that, it
>>>> > might be worth considering per-message
>>>> > metadata -
>>>>
>>>> We'd have to be very clear about the problem that per-message metadata
>>>> is aiming for. There is an elegance to delivering blobs and nothing
>>>> more. Metadata can be added on top using zproto.
>>>>
>>>> What we really want IMO is per-peer metadata, and an API to get/set
>>>> this. Using messages is a hack. If we are sending/receiving data on a
>>>> per-message basis, that is the message. Having two layers that both
>>>> carry per-message data is... wrong IMO.
>>>>
>>>> However the key thing is, what's the problem. Then we can discuss
>>>> solutions to that.
>>>>
>>>> -Pieter
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150202/b8c1574b/attachment.htm>


More information about the zeromq-dev mailing list