[zeromq-dev] multipart messages in a realtime environment

[Michael Haberler]

Am 18.04.2014 um 00:33 schrieb Goswin von Brederlow <goswin-v-b at web.de>:

> On Thu, Apr 17, 2014 at 10:54:59PM +0200, Michael Haberler wrote:
>> Goswin,
>> 
>> Am 17.04.2014 um 11:24 schrieb Goswin von Brederlow <goswin-v-b at web.de>:
>> 
>>> On Wed, Apr 16, 2014 at 08:07:27PM +0200, Michael Haberler wrote:
>>>> we're using zeroMQ in machinekit (formerly LinuxCNC)
>>>> 
>>>> to retain the end-to-end principle, we need to be able to send multipart messages to and from RT threads
>>>> 
>>>> that isnt possible with libzmq code since it's C++ and uses dynamic memory allocation
>>>> 
>>>> the code which we'll be using is based on a lock-free ringbuffer in shared memory (ring_*), and a multipart framing layer on top of that (bring_*): http://psha.org.ru/cgit/psha/ring/?h=wip-buffer
>>>> 
>>>> a proxy thread at the RT boundary translates from zmq to ringbuffer framing and back
>>>> 
>>>> just in case somebody has a similar requirement
>>>> 
>>>> - Michael
>>> 
>>> doesn't zero-copy fix that?
>> 
>> I dont understand what are you are suggesting, could you elaborate?
>> 
>> - Michael
> 
> You have your data in some buffer somewhere in memory. Then when you
> can call zmq_msg_init_data() to make a message out of it without
> copying the data:

I think I should have mentioned the environment restrictions:

The application is a cooperative ensemble of normal userland threads, and RT threads.

All these threads share is a shm segment; nothing else. RT threads may live in the kernel, as a hypervisor thread like in Xenomai, or as a 'posix thread on steroids' with RT-PREEMPT. Or even on a core running an entirely different OS, or a bare metal code blob. The application works with either, which is why this framing was needed.

The fact that you have some data outside such a shared memory segment has no bearing; it's not accessible. 

That said, the methods in the repo I pointed to support zero-copy writing at the frame level _within_ the shm segment.

- Michael

> 
>  The zmq_msg_init_data() function shall initialise the message object
>  referenced by msg to represent the content referenced by the buffer
>  located at address data, size bytes long. No copy of data shall be
>  performed and ØMQ shall take ownership of the supplied buffer.
> 
>  If provided, the deallocation function ffn shall be called once the
>  data buffer is no longer required by ØMQ, with the data and hint
>  arguments supplied to zmq_msg_init_data().
> 
> The deallocation function allows you to reuse buffers that zmq has
> finished with so you don't need to dynamically allocate them. And the
> data isn't copied so the zmq_msg functions run in constant time no
> matter how large the buffer.
> 
> MfG
> 	Goswin
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev