[zeromq-dev] Thread safe Pub/Sub and Multicast

Dave Kuhlman dkuhlman at davekuhlman.org
Sat Mar 12 21:00:39 CET 2016


For those of us who use Python and the pyzmq library for ZeroMQ, I'm
wondering whether the solution to the problem and need for thread
safety is to avoid threads and to use either the tornado ioloop
support or asyncio ioloop.  Am I right that doing so would give a
Python developer the capabilities of multiple threads without the
worries about thread safety.

One reason I'm asking is that I'm the one who added the ioloop
examples to zguide (in examples/Python/tornado_ioloop and
examples/Python/ascyncio_ioloop).  A number of those examples use
(what I understand as) multiple concurrent tasks within a single
thread.  So, I'm wondering whether that really is thread safe (or
whatever the equivalent is for tasks), as I believe it is.

Any enlightenment will be welcome.

And, by the way, I suspect that something equivalent could be done
in Node.js (JavaScript), which also uses a single threaded approach
to "concurrency" (psuedo-concurrency, quasi-concurrency, whatever).
In Node.js, you could use callbacks, but there are libraries that
can wrap your code to make is seem more simple and parallel.  For
example, see async.js.  Here are a few quick examples from the
async.js Web page (https://www.npmjs.com/package/async):

     async.map([`file1`,`file2`,`file3`], fs.stat, function(err, results){
         // results is now an array of stats for each file
     });

     async.filter([`file1`,`file2`,`file3`], fs.exists, function(results){
         // results now equals an array of the existing files
     });

     async.parallel([
         function(){ ... },
         function(){ ... }
     ], callback);

     async.series([
         function(){ ... },
         function(){ ... }
     ]);

As a bit of justification for this question, the main intension of
the Python ioloop support in tornado and asyncio is to address
problems of network and I/O delays and to enable our code to do
something useful during those delays.  That seems like a central
concern for users of ZeroMQ.

Dave


On Sat, Mar 12, 2016 at 03:00:58AM +0200, Doron Somech wrote:
>    Hi All,
>    During the ZeroMQ hackathon I have added the Radio/Dish pattern (thread
>    safe version of pubsub) and multicast support (only for Radio/Dish
>    sockets) to both libzmq and czmq.
>    Radio/Dish is very similar to pub/sub with some differences:
>      * Thread safe, you can send and receive messages from multiple
>        threads, so for publisher you now don't need internal device to
>        publish from multiple threads and with subscriber you can use it as
>        load balancer.
>      * Only single frame can be sent
>      * Radio/Dish matching is exact matching (vs prefix of pubsub)
>      * Group (Radio/Dish topic) is string and currently limited to 15
>        chars (might be increased in the future)
>      * Group is set as part of the zmq_msg (libzmq) or zframe (czmq).
>      * You call join or leave on the socket (with methods instead of
>        socket options)
> 
>    To learn how to use it with czmq take a loot at the following gist:
>    [1]https://gist.github.com/somdoron/9423196a228775c8f5af
>    For libzmq take a loot at the test:
>    [2]https://github.com/zeromq/libzmq/blob/master/tests/test_radio_dish.c
>    pp#L90
>    And the most exciting news, with radio-dish you can use multicast
>    transport, just replace the address from following gist with udp
>    transport and multicast address, following is a valid address for
>    example:
>    udp://[3]239.0.0.1:55555
>    Regards
>    Doron
> 
> References
> 
>    1. https://gist.github.com/somdoron/9423196a228775c8f5af
>    2. https://github.com/zeromq/libzmq/blob/master/tests/test_radio_dish.cpp#L90
>    3. http://239.0.0.1:55555/

> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


-- 

Dave Kuhlman
http://www.davekuhlman.org



More information about the zeromq-dev mailing list