[zeromq-dev] socket event monitoring problem when using an event loop

[Chris Laws]

I was recently working on getting the socket event monitor working in an
async focused Python library, aiozmq, and I had great difficulty getting it
working reliably. Hopefully this post might save someone some time if they
are trying something similar in the future.

tl;dr when using async programming model (e.g. event loop) connect before
bind when using using the socket monitor PAIR sockets.

The library I was working with uses pyzmq for lots of its internals. It is
worth noting that I had no problem using the socket monitor from pyzmq
along with the typical blocking style approach where I would run the socket
event monitor poller in a separate thread.

However when I tried to use the socket monitor along with the asynchronous
model encouraged by the Python (3.4) asyncio module I found that the
delivery of socket events was very unreliable - often the events would only
come through when I was shutting down the monitor transport (the socket) at
the end of a test.

I started out using pyzmq's get_monitor_socket function which would enable
the socket monitor using the following sequence:
- dynamically generate an inproc:// address to use;
- request libzmq to create and bind a PAIR socket to the address;
- create a new PAIR socket and connect it to the inproc:// address;
- return the connected PAIR socket.

This sequence of events turned out to be the cause of the problem I
observed. However, this common approach only seems to be a problem when
using the socket in an event loop (async) which is watching the file
descriptor for a ready_ready event.

I eventually found this problem mentioned here:
https://github.com/mkoppanen/php-zmq/issues/130 and then here
http://lists.zeromq.org/pipermail/zeromq-dev/2014-January/024545.html

I changed the socket monitor enabling code sequence to:
- create a dynamic inproc:// address;
- create a new PAIR socket and attempt to connect it to the address (even
through there is no PAIR socket bound to receive the connection)
- request libzmq to create and bind a PAIR socket to the address;
- return the connected PAIR socket.

The exact implementation is here:
https://github.com/aio-libs/aiozmq/blob/master/aiozmq/core.py#L537

Using this sequence I was able to reliably monitor the PAIR socket's file
descriptor to receive socket monitor events while running within the
asyncio event loop.

Regards,
Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150702/b0690276/attachment.htm>