[zeromq-dev] Epolling on FD doesn't work for 2.1.11

[Robert G. Jakabosky]

On Tuesday 20, Andrzej K. Haczewski wrote:
> First of all I'd like to say hello to everyone, since this is my first
> post here. So:
> 
> Hello,
> 
> I've been trying to integrate ZeroMQ with libev event loop and I've
> hit a strange behavior (or a bug?). In particular, as anyone
> integrating with event loop, I've been watching (libev calls it like
> that) my sockets for any input events, passing file descriptors taken
> using getsockopt(ZMQ_FD).
> 
> How surprised I was that for REQ and REP sockets only REP socket
> received notifications. Trying to isolate an error I simulated the
> behavior using Python and epoll:
> http://pastebin.com/kiBtu4jv
> 
> On my system (linux 3.1.6 amd64) and ZeroMQ 2.1.11 the output of that
> code looks like this:
> [(16, 1)]
> SUCCESSFULLY RECEIVED REQUEST!
> []
> []
> []
> []
> []
> []
> ...
> 
> Upgrading ZeroMQ to 3.1.0 the situation changes dramatically,
> outputting what I expected:
> [(9, 1), (10, 1)]
> SUCCESSFULLY RECEIVED REQUEST!
> [(9, 1), (10, 1)]
> RECEIVED GOOD REPLY!
> 
> Trying to get help on #zeromq I was told that it might be an issue
> with level-triggering epoll, so I tried switching to edge-triggered
> poll. Unfortunately it doesn't work for neither 2.1.11 nor 3.1.0.

zmq sockets are alway level-triggered.  Your code should call 
zmq_send/zmq_recv until it returns EAGAIN, then register the FD for read 
events with your event loop.  Your code also needs to alway pass the 
ZMQ_NOBLOCK flag to zmq_send/zmq_recv.

For an lib-ev event loop use an idle watcher to call zmq_recv when it is not 
block.  Once zmq_recv returns EAGAIN, stop the idle watcher and start an io 
watcher for read events.  The ev_io callback needs to get the value of 
ZMQ_EVENTS with zmq_getsockopt(), then check the events value for ZMQ_POLLIN.  
As soon as ZMQ_EVENTS has ZMQ_POLLIN stop the io watcher and start the idle 
watcher again.

Basically if the zmq socket is readable, your code must keep calling 
zmq_recv() until it returns EAGAIN without waiting for another IO read event 
from the event loop.

I have written an example [1] in Lua that reads from a zmq SUB socket using 
the lib-ev event loop.

Also there is client [2] and server [3] example (using REQ/REP sockets) that 
can use either lib-ev [4] or epoll [5] event loops.

I hope that helps.

1. https://github.com/Neopallium/lua-
zmq/blob/master/examples/subscriber_ev.lua
2. https://github.com/Neopallium/lua-zmq/blob/master/examples/client_poll.lua
3. https://github.com/Neopallium/lua-zmq/blob/master/examples/server_poll.lua
4. https://github.com/Neopallium/lua-zmq/blob/master/examples/poller/ev.lua
5. https://github.com/Neopallium/lua-zmq/blob/master/examples/poller/epoll.lua

-- 
Robert G. Jakabosky