[zeromq-dev] Best practice

[Martin Sustrik]

Hi Todd,

I cannot compile your code as it's just bits and pieces, however, I am
attaching a functional code (tested with 0MQ/0.6 on Linux). Have a look
at it.

Few comments:

1. You don't need to create the I/O thread. You are doing no I/O, just
passing messages within a process, so there's no need for it.

2. When sending a message you don't supply deallocation function. This
means 0MQ won't deallocate the buffer once it is not needed => memory leak.

Hop this helps!
Martin

Todd Gruben wrote:
> excellent reply, thanks for that.  I am still having a bit of a time.  
> Messages are going into the exchange, but I never see them on the 
> receiver side.  Here is the gist of my app
> 
> mainthread:
>    locator_t l(NULL);// the process locator
>     dispatcher_t dispatch (feedcount + 2); //feeds + main+receiver
> 
> producer threads:
>                     i_thread *pt = io_thread_t::create (&_dispatch);
>                     api_thread_t *api = api_thread_t::create 
> (&_dispatch, &_locator);
>                     int eid = api->create_exchange ("E",scope_local);
>                     api->bind ("E", "Q",pt,pt);
> while(readdata){
>                         message_t message((void*)data,data._rec_size,NULL);
>                                                 api->send(eid,message);
> 
> }
> 
> receiver Thread:
> 
>         api_thread_t *api = api_thread_t::create (&_dispatch, &_locator);
>        api->create_queue ("Q",scope_process);
>     while(1){
>                         message_t message;
>                         api->receive (&message);
> 
> }
> 
> 
> 
> everything compiles and I receive my feed and send it on, but I never 
> receive any messages on the Receiver side of the queue.  Any thoughts?
> 
> -Todd
> 
> 
> On Thu, Apr 9, 2009 at 3:19 AM, Martin Sustrik <sustrik at fastmq.com 
> <mailto:sustrik at fastmq.com>> wrote:
> 
>     Hi Todd,
> 
> 
>         I am just getting started with 0mq and I would like some advice.
> 
>         I have an application which basically combines N streaming feeds
>         of information and outputs a single combined feed of
>         information.  My application has a thread for each feed and an
>         output thread with a queue that I implemented that is shared
>         among the threads.  After reading your design docs, I feel your
>         messaging implementation will probably perform better.  My
>         question lies in the local_scope verses process_scope.  Should
>         each feed have a local scope exchange connected to a process
>         scope queue?  If so how would I bind the queue to the exchange
>         on the consuming side?
> 
> 
>     This is a nice little use case for in-process messaging.
> 
>     1. Open a locator. There are no components running on the network so
>     you can do without a zmq_server. Pass NULL instead of zmq_server
>     address:
> 
>     locator_t locator (NULL);
> 
>     2. Open a dispatcher. Say you have 3 sender threads and a single
>     receiver thread. Initialise dispatcher to have 4 threads:
> 
>     dispatcher_t dispatcher (4);
> 
>     3. Create an api_thread_t object in each thread:
> 
>     api_thread_t *api = api_thread_t::create (&dispatcher, &locator);
> 
>     API thread gives you the interface to 0MQ functionaity. Note that an
>     instance of api_thread_t MUST be used from just a single thread.
> 
>     4. In receiver thread create a process-wide queue. Making it
>     process-scoped means that other threads will be able to see it and
>     bind to it:
> 
>     api->create_queue ("Q", scope_process);
> 
>     5. In each sender thread create a local exchange (nobody needs to
>     bind to it except the thread that created it) and bind it to the
>     queue created by receiver thread:
> 
>     int eid = api->create_exchange ("E", scope_local);
>     api->bind ("E","Q");
> 
>     6. Now you are able to send messages from sender threads:
> 
>     message_t msg (10);
>     ...
>     api->send (eid, msg);
> 
>     7. In receiver thread, you can receive the messages:
> 
>     message_t msg;
>     api->receive (&msg);
> 
>     Some additional comments:
> 
>     If you are striving for extra-high performance - and if your
>     messages have non-negligible length - you should take overhead
>     intoduced by copying the message into account.
> 
>     0MQ provides you with two ways to create a message to send:
> 
>     message_t msg (100);
> 
>     Creates a message with 100 byte buffer for you. It is optimised for
>     performance. If the message small it allocates the buffer on the
>     stack, if it's larger, it allocates it on the heap, however, it
>     allocates a single chunk for both message data and metadata.
> 
>     The other way to create a message should be used when the message is
>     handed to you that's already allocated and you have no way to
>     control it's creation (mostly with legacy applications):
> 
>     message_t msg (data, size, free);
> 
>     Third parameter is pointer to the function to use to deallocate the
>     data.
> 
>     If what you need is a extremely high performance on a multicore box,
>     you should also consider using optimised memory allocation mechanism
>     (like tcmalloc) instead of the raw malloc.
> 
>     Let us know what kind of performance improvement you've achieved
>     using 0MQ!
> 
>     Martin
> 
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.cpp
Type: text/x-c++src
Size: 1378 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20090409/025c54e5/attachment.cpp>