[zeromq-dev] Unaligned memory for specific message sizes

[Luca Boccassi]

There is no promise of data alignment in the API. However, at the
moment you can get the behaviour you want by setting the
ZMQ_ZERO_COPY_RCV context option to false.

On Fri, 2019-07-19 at 17:48 +0200, Felix Frank wrote:
> 
> Hello all,
> 
> I have an issue with zeromq delivering message content which is not
> aligned to standard 32/64 bit memory addresses. I noticed it when I
> ran into SIGBUS errors on a raspberry pi using zmq. 
> 
> I am working with the latest zeromq from github (via cloning). I
> compile it using Cmake with the following options:
> > cmake \
> > > -DCMAKE_BUILD_TYPE=Release \
> > > -DCMAKE_INSTALL_PREFIX=/usr \
> > > -DCMAKE_INSTALL_LIBDIR=lib \
> > > -DWITH_OPENPGM=ON \
> > > -DWITH_PERF_TOOL=OFF ..
> > make
> 
> I made a tiny example using TCP and the PUB/SUB protocol, in which
> the sender sends messages of increasing size. The receiver simply
> tracks the pointer to the received message data and prints the
> address in case it changes compared to last one.
> 
> I get the following alignment profile depending on message size (in
> bytes):
> size < 32: Address ends with 8
> 32 < size < 256: Address ends with a
> 256 < size < 8184: Address ends with 1
> 8184 < size: Address ends with 8
> What happens in with messages in the intermediate size range, so they
> end up being unaligned? Can I disable this behavior somehow?
> 
> 
> Initially I thought it's related to the VSM mechanics, but that only
> explains the size < 32 switch.
> 
> I appreciate any help.
> Best wishes from Germany,
> Felix
> 
> -------------------------------
> My machine:
> -------------------------------
> > uname -a:
> > > Linux xxxxxx 5.2.1-arch1-1-ARCH #1 SMP PREEMPT Sun Jul 14
> > > 14:52:52 UTC 2019 x86_64 GNU/Linux
> > cpu:
> > > Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz
> > gcc:
> > > gcc (GCC) 9.1.0
> > cmake:
> > > cmake version 3.14.6
> > > 
> -------------------------------
> Sender code:
> -------------------------------
> #include <zmq.h>
> #include <stdlib.h>
> #include <unistd.h>
> 
> int main(int argc, char const *argv[])
> {
>   void *c = zmq_ctx_new();
>   void *s = zmq_socket(c, ZMQ_PUB);
>   // Bind socket
>   zmq_bind(s, "tcp://*:3490");
>   usleep(1000000);
> 
>   uint64_t counter = 1;
>   // Send messages in a loop
>   while(true) {
>     counter++;
>     if (counter >= 200000) {
>       printf("%s\n", "Limit reached. Shutdown.");
>       break;
>     }
>     zmq_msg_t msg;
>     zmq_msg_init_size(&msg, counter);
>     zmq_msg_send(&msg, s, 0);
>     usleep(100);
>   }
> 
>   zmq_close(s);
>   return 0;
> }
> 
> -------------------------------
> Receiver code:
> -------------------------------
> #include <zmq.h>
> #include <stdlib.h>
> 
> int main(int argc, char const *argv[])
> {
>   void *c = zmq_ctx_new();
>   void *s = zmq_socket(c, ZMQ_SUB);
>   // Connect socket
>   zmq_connect(s, "tcp://localhost:3490");
> 
>   zmq_setsockopt(s, ZMQ_SUBSCRIBE, "", 0);
> 
>   // Recv messages in a loop
>   void *last_addr = NULL;
>   while(true) {
>     zmq_msg_t msg;
>     zmq_msg_init(&msg);
>     zmq_msg_recv(&msg, s, 0);
> 
>     if (zmq_msg_data(&msg) != last_addr) {
>       last_addr = zmq_msg_data(&msg);
>       printf("Data address: %p\n", last_addr);
>       printf("Msg Size: %lu\n", zmq_msg_size(&msg));
>     }
>     zmq_msg_close(&msg);
>   }
> 
>   zmq_close(s);
>   return 0;
> }
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> 
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
> 
> 
-- 
Kind regards,
Luca Boccassi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190719/1c87dfc1/attachment.sig>