[zeromq-dev] Unaligned memory for specific message sizes
Felix Frank
felix.se.frank at gmail.com
Fri Jul 19 17:48:14 CEST 2019
Hello all,
I have an issue with zeromq delivering message content which is not aligned
to standard 32/64 bit memory addresses. I noticed it when I ran into SIGBUS
errors on a raspberry pi using zmq.
I am working with the latest zeromq from github (via cloning). I compile it
using Cmake with the following options:
cmake \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INSTALL_LIBDIR=lib \
-DWITH_OPENPGM=ON \
-DWITH_PERF_TOOL=OFF ..
make
I made a tiny example using TCP and the PUB/SUB protocol, in which the
sender sends messages of increasing size. The receiver simply tracks the
pointer to the received message data and prints the address in case it
changes compared to last one.
I get the following alignment profile depending on message size (in bytes):
- size < 32: Address ends with 8
- 32 < size < 256: Address ends with a
- 256 < size < 8184: Address ends with 1
- 8184 < size: Address ends with 8
What happens in with messages in the intermediate size range, so they end
up being unaligned? Can I disable this behavior somehow?
Initially I thought it's related to the VSM mechanics, but that only
explains the size < 32 switch.
I appreciate any help.
Best wishes from Germany,
Felix
-------------------------------
My machine:
-------------------------------
uname -a:
Linux xxxxxx 5.2.1-arch1-1-ARCH #1 SMP PREEMPT Sun Jul 14 14:52:52 UTC 2019
x86_64 GNU/Linux
cpu:
Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz
gcc:
gcc (GCC) 9.1.0
cmake:
cmake version 3.14.6
-------------------------------
Sender code:
-------------------------------
#include <zmq.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char const *argv[])
{
void *c = zmq_ctx_new();
void *s = zmq_socket(c, ZMQ_PUB);
// Bind socket
zmq_bind(s, "tcp://*:3490");
usleep(1000000);
uint64_t counter = 1;
// Send messages in a loop
while(true) {
counter++;
if (counter >= 200000) {
printf("%s\n", "Limit reached. Shutdown.");
break;
}
zmq_msg_t msg;
zmq_msg_init_size(&msg, counter);
zmq_msg_send(&msg, s, 0);
usleep(100);
}
zmq_close(s);
return 0;
}
-------------------------------
Receiver code:
-------------------------------
#include <zmq.h>
#include <stdlib.h>
int main(int argc, char const *argv[])
{
void *c = zmq_ctx_new();
void *s = zmq_socket(c, ZMQ_SUB);
// Connect socket
zmq_connect(s, "tcp://localhost:3490");
zmq_setsockopt(s, ZMQ_SUBSCRIBE, "", 0);
// Recv messages in a loop
void *last_addr = NULL;
while(true) {
zmq_msg_t msg;
zmq_msg_init(&msg);
zmq_msg_recv(&msg, s, 0);
if (zmq_msg_data(&msg) != last_addr) {
last_addr = zmq_msg_data(&msg);
printf("Data address: %p\n", last_addr);
printf("Msg Size: %lu\n", zmq_msg_size(&msg));
}
zmq_msg_close(&msg);
}
zmq_close(s);
return 0;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190719/a00e0e57/attachment.htm>
More information about the zeromq-dev
mailing list