[zeromq-dev] Best practice

[Martin Sustrik]

Hi Todd,

> I am just getting started with 0mq and I would like some advice.
> 
> I have an application which basically combines N streaming feeds of 
> information and outputs a single combined feed of information.  My 
> application has a thread for each feed and an output thread with a queue 
> that I implemented that is shared among the threads.  After reading your 
> design docs, I feel your messaging implementation will probably perform 
> better.  My question lies in the local_scope verses process_scope.  
> Should each feed have a local scope exchange connected to a process 
> scope queue?  If so how would I bind the queue to the exchange on the 
> consuming side? 

This is a nice little use case for in-process messaging.

1. Open a locator. There are no components running on the network so you 
can do without a zmq_server. Pass NULL instead of zmq_server address:

locator_t locator (NULL);

2. Open a dispatcher. Say you have 3 sender threads and a single 
receiver thread. Initialise dispatcher to have 4 threads:

dispatcher_t dispatcher (4);

3. Create an api_thread_t object in each thread:

api_thread_t *api = api_thread_t::create (&dispatcher, &locator);

API thread gives you the interface to 0MQ functionaity. Note that an 
instance of api_thread_t MUST be used from just a single thread.

4. In receiver thread create a process-wide queue. Making it 
process-scoped means that other threads will be able to see it and bind 
to it:

api->create_queue ("Q", scope_process);

5. In each sender thread create a local exchange (nobody needs to bind 
to it except the thread that created it) and bind it to the queue 
created by receiver thread:

int eid = api->create_exchange ("E", scope_local);
api->bind ("E","Q");

6. Now you are able to send messages from sender threads:

message_t msg (10);
...
api->send (eid, msg);

7. In receiver thread, you can receive the messages:

message_t msg;
api->receive (&msg);

Some additional comments:

If you are striving for extra-high performance - and if your messages 
have non-negligible length - you should take overhead intoduced by 
copying the message into account.

0MQ provides you with two ways to create a message to send:

message_t msg (100);

Creates a message with 100 byte buffer for you. It is optimised for 
performance. If the message small it allocates the buffer on the stack, 
if it's larger, it allocates it on the heap, however, it allocates a 
single chunk for both message data and metadata.

The other way to create a message should be used when the message is 
handed to you that's already allocated and you have no way to control 
it's creation (mostly with legacy applications):

message_t msg (data, size, free);

Third parameter is pointer to the function to use to deallocate the data.

If what you need is a extremely high performance on a multicore box, you 
should also consider using optimised memory allocation mechanism (like 
tcmalloc) instead of the raw malloc.

Let us know what kind of performance improvement you've achieved using 0MQ!

Martin