[zeromq-dev] Failing to synchronize 0mq sockets

[Michel Pelletier]

Your example works fine for me if you change the tcp PUB endpoint
transport to inproc.

I'm not sure what's going on, but the guide mentions only sharing
contexts between threads with inproc transports. I think bind is
synchronous but connect isn't, so perhaps your threads are
synchronizing faster than connect can connect to the pub socket over
tcp, and the sub socket is therefore missing a bunch of messages.

Personally I would recommend to you using a multiprocess approach with
python instead of threads.  It's my personal opinion and I don't want
to start a flame war, but threads and Python don't mix and can barely
scale beyond one core. We have good event driven integration with
zeromq sockets and "green" libraries like eventlet and gevent, it
makes sense to me disregard threads as a scaleable worker model and
work solely with processes.  They can scale across many cores and
boxes.

This is a must read: http://www.dabeaz.com/GIL/

-Michel

On Mon, Apr 2, 2012 at 2:11 PM, Daniele Varrazzo
<daniele.varrazzo at gmail.com> wrote:
> Hello,
>
> I'm learning some 0mq following the guide and implementing the test
> programs in python. I'm using zmq/pyzmq 2.1.9 on Ubuntu.
>
> I'm trying to synchronize subscribers with a publisher using pair
> sockets in a multithread script. The script is at
> <https://gist.github.com/2286899>. The pattern implemented is
> something like:
>
> main thread:
>  - create the pub and bind
>  - for each worker:
>    - create a pair control socket and bind
>    - spawn the thread
>    - wait for a message on the control socket
>  - publish the messages
>
> worker threads:
>  - create the sub and connect
>  - create a pair control socket and connect
>  - send a message on the control socket
>
> I'm experiencing "slow joiner syndrome" here: in a native threads
> script, none of the message is received by the workers despite the sub
> are connected before an ack message is sent back to the pub thread.
> Adding a pause before publishing the message, it works fine. Running
> the program in eventlet instead of native threads the slow joining is
> even more evident: on my machine one of the threads misses the first
> few messages, see the comment at the end of the gist.
>
> Any help in understanding what is going on would be appreciated, thanks.
>
> -- Daniele
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev