[zeromq-dev] Unaligned memory for specific message sizes

Felix Frank felix.se.frank at gmail.com
Fri Jul 19 17:48:14 CEST 2019


Hello all,

I have an issue with zeromq delivering message content which is not aligned
to standard 32/64 bit memory addresses. I noticed it when I ran into SIGBUS
errors on a raspberry pi using zmq.

I am working with the latest zeromq from github (via cloning). I compile it
using Cmake with the following options:

cmake \

-DCMAKE_BUILD_TYPE=Release \

-DCMAKE_INSTALL_PREFIX=/usr \

-DCMAKE_INSTALL_LIBDIR=lib \

-DWITH_OPENPGM=ON \

-DWITH_PERF_TOOL=OFF ..

make


I made a tiny example using TCP and the PUB/SUB protocol, in which the
sender sends messages of increasing size. The receiver simply tracks the
pointer to the received message data and prints the address in case it
changes compared to last one.

I get the following alignment profile depending on message size (in bytes):

   - size < 32: Address ends with 8
   - 32 < size < 256: Address ends with a
   - 256 < size < 8184: Address ends with 1
   - 8184 < size: Address ends with 8

What happens in with messages in the intermediate size range, so they end
up being unaligned? Can I disable this behavior somehow?

Initially I thought it's related to the VSM mechanics, but that only
explains the size < 32 switch.

I appreciate any help.
Best wishes from Germany,
Felix

-------------------------------
My machine:
-------------------------------

uname -a:

Linux xxxxxx 5.2.1-arch1-1-ARCH #1 SMP PREEMPT Sun Jul 14 14:52:52 UTC 2019
x86_64 GNU/Linux

cpu:

Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz

gcc:

gcc (GCC) 9.1.0

cmake:

cmake version 3.14.6

-------------------------------
Sender code:
-------------------------------
#include <zmq.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char const *argv[])
{
  void *c = zmq_ctx_new();
  void *s = zmq_socket(c, ZMQ_PUB);
  // Bind socket
  zmq_bind(s, "tcp://*:3490");
  usleep(1000000);

  uint64_t counter = 1;
  // Send messages in a loop
  while(true) {
    counter++;
    if (counter >= 200000) {
      printf("%s\n", "Limit reached. Shutdown.");
      break;
    }
    zmq_msg_t msg;
    zmq_msg_init_size(&msg, counter);
    zmq_msg_send(&msg, s, 0);
    usleep(100);
  }

  zmq_close(s);
  return 0;
}

-------------------------------
Receiver code:
-------------------------------
#include <zmq.h>
#include <stdlib.h>

int main(int argc, char const *argv[])
{
  void *c = zmq_ctx_new();
  void *s = zmq_socket(c, ZMQ_SUB);
  // Connect socket
  zmq_connect(s, "tcp://localhost:3490");

  zmq_setsockopt(s, ZMQ_SUBSCRIBE, "", 0);

  // Recv messages in a loop
  void *last_addr = NULL;
  while(true) {
    zmq_msg_t msg;
    zmq_msg_init(&msg);
    zmq_msg_recv(&msg, s, 0);

    if (zmq_msg_data(&msg) != last_addr) {
      last_addr = zmq_msg_data(&msg);
      printf("Data address: %p\n", last_addr);
      printf("Msg Size: %lu\n", zmq_msg_size(&msg));
    }
    zmq_msg_close(&msg);
  }

  zmq_close(s);
  return 0;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190719/a00e0e57/attachment.htm>


More information about the zeromq-dev mailing list