[zeromq-dev] Zmq 4.2.0 aligned memory

[KIU Shueng Chuan]

A common use case for me is sending an array of floats.

First message part is some small metadata. Second message part is the float
array.

On reception, zmq_msg_data is cast to float* and accessed directly.

This non-alignment would be problematic.
Or perhaps there never was any alignment guarantee?

On 15 Nov 2016 3:34 a.m., "Jens Auer" <jens.auer at betaversion.net> wrote:

> Hi,
>
> I think I have an idea why you are seeing unaligned messages, but this
> only applies to messages where the payload is not stored in the msg_t
> object itself. I think the threshold for this is 64 bytes. In ZeroMQ 4.1,
> receiving messages was done by first receiving from the socket into a
> static 8kb buffer, and then a new message object was created that allocated
> memory externally by calling malloc.  The payload was then copied from the
> receive buffer to the message buffer. The malloced message buffer was
> aligned probably.
>
> In ZeroMQ 4.2, this is changed to reduce the number of malloc calls and
> copy operations. The receive buffer is now dynamically allocated as a 8kb
> block, and messages are constructed as zero-copy messages using the part of
> the receive buffer containing the payload. This saves malloc calls and copy
> operations and increases performance. However, the payload may now start at
> basically arbitrary addresses. As an example, let's assume that we receive
> a small message of 10 bytes and a large message of 1kb, both received in a
> single call to recv on the socket. The engine allocates a new buffer of
> 8kb, calls recv(socket, buffer) and the data is written to the buffer. A
> small message is then created which contains the data from byte 2-11 in the
> msg_t, byte 1 contains the header. At byte 12 starts the header of the next
> message, and at byte 22(?) starts the payload. The large message is created
> as a zero-copy message using the pointer to byte 22 as storage. This is not
> aligned to a 4-byte address.
>
> Could you provide some more information about the sizes of the messages
> that you receive? How do you decode the buffer content?
>
> Best wishes,
> Jens
>
> -----Ursprüngliche Nachricht-----
> Von: zeromq-dev [mailto:zeromq-dev-bounces at lists.zeromq.org] Im Auftrag
> von Emmanuel Taurel
> Gesendet: Montag, 14. November 2016 16:49
> An: zeromq-dev at lists.zeromq.org
> Betreff: [zeromq-dev] Zmq 4.2.0 aligned memory
>
> Hello all,
>
> We are using zeromq since years now without troubles. We have recently
> tried our software using Zmq 4.2.0 (on linux hosts).
> For our application, we are using multipart messages with 4 parts in
> publish/subscribe mode.
> With Zmq 4.0.5, on the subscriber side, when we get the last message part,
> the received buffer was memory aligned (at least on 0x4 border).
> Unfortunately, with Zmq 4.2.0, the buffer is not aligned any more.
>  For instance with Zmq 4.0.5, the buffer was at address xxx08 while with
> Zmq 4.2.0, it is at address xxx23.
>
> I don't know if it is relevant but our messages are relatively small
> messages (few tens of bytes) This is a problem for us in decoding the
> buffer content.
>
> Is there something to be done to have memory aligned buffers?
>
> Thank's in advance for your answers
>
>  Emmanuel
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> ---
> Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
> https://www.avast.com/antivirus
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20161115/775ea706/attachment.htm>