[zeromq-dev] Best practice
sustrik at fastmq.com
Thu Apr 9 10:19:55 CEST 2009
> I am just getting started with 0mq and I would like some advice.
> I have an application which basically combines N streaming feeds of
> information and outputs a single combined feed of information. My
> application has a thread for each feed and an output thread with a queue
> that I implemented that is shared among the threads. After reading your
> design docs, I feel your messaging implementation will probably perform
> better. My question lies in the local_scope verses process_scope.
> Should each feed have a local scope exchange connected to a process
> scope queue? If so how would I bind the queue to the exchange on the
> consuming side?
This is a nice little use case for in-process messaging.
1. Open a locator. There are no components running on the network so you
can do without a zmq_server. Pass NULL instead of zmq_server address:
locator_t locator (NULL);
2. Open a dispatcher. Say you have 3 sender threads and a single
receiver thread. Initialise dispatcher to have 4 threads:
dispatcher_t dispatcher (4);
3. Create an api_thread_t object in each thread:
api_thread_t *api = api_thread_t::create (&dispatcher, &locator);
API thread gives you the interface to 0MQ functionaity. Note that an
instance of api_thread_t MUST be used from just a single thread.
4. In receiver thread create a process-wide queue. Making it
process-scoped means that other threads will be able to see it and bind
api->create_queue ("Q", scope_process);
5. In each sender thread create a local exchange (nobody needs to bind
to it except the thread that created it) and bind it to the queue
created by receiver thread:
int eid = api->create_exchange ("E", scope_local);
6. Now you are able to send messages from sender threads:
message_t msg (10);
api->send (eid, msg);
7. In receiver thread, you can receive the messages:
Some additional comments:
If you are striving for extra-high performance - and if your messages
have non-negligible length - you should take overhead intoduced by
copying the message into account.
0MQ provides you with two ways to create a message to send:
message_t msg (100);
Creates a message with 100 byte buffer for you. It is optimised for
performance. If the message small it allocates the buffer on the stack,
if it's larger, it allocates it on the heap, however, it allocates a
single chunk for both message data and metadata.
The other way to create a message should be used when the message is
handed to you that's already allocated and you have no way to control
it's creation (mostly with legacy applications):
message_t msg (data, size, free);
Third parameter is pointer to the function to use to deallocate the data.
If what you need is a extremely high performance on a multicore box, you
should also consider using optimised memory allocation mechanism (like
tcmalloc) instead of the raw malloc.
Let us know what kind of performance improvement you've achieved using 0MQ!
More information about the zeromq-dev