[zeromq-dev] missing events ZMQ_FD / ZMQ_EVENTS

Paul Colomiets paul at colomiets.name
Wed Jun 27 21:44:45 CEST 2012

Hi Justin,

On Tue, Jun 26, 2012 at 2:16 AM, Justin Karneges <justin at affinix.com> wrote:
> On Wednesday, May 02, 2012 03:27:42 AM Paul Colomiets wrote:
>> Ok. Behind the scenes ZMQ_FD, is basically a counter, which wakes up
>> poll when is non-zero. The counter is reset on each getsockopt ZMQ_EVENTS,
>> zmq_send and zmq_recv.
>> The following diagram shows race condition with two sockets A and B,
>> in a scenario similar to yours:
>> https://docs.google.com/drawings/d/1F97jpdbYMjjb6-2VzRtiL2LpHy638-AEOyrUX84
>> HL78/edit
>> Note: the last poll is entered with both counters set to zero, so it
>> will not wake up, despite the fact that there is pending message.
> Was there ever a resolution on this?
> I am using ZMQ_FD now to integrate into an event loop, and I am seeing some
> odd behavior when testing a hello world REQ/REP on the REP side.
> The REP server binds and waits for data. The fd is indicated as readable
> twice. First, the events are 0 (maybe this happens when the client connects?),
> then the events are 1 (ZMQ_POLLIN). The server considers the REP socket
> readable and so it reads a message without blocking. Now it wants to reply,
> but it considers the socket not yet writable. I was expecting that after
> reading from the socket, the fd would be indicated as readable and the events
> would be 2 (ZMQ_POLLOUT). However, this event never comes and so the server
> just idles.
> Now here's where it gets weird: if I kill the client (which was also waiting
> around, as it never got a reply), then the server gets new events with
> ZMQ_POLLOUT set. This causes the server to finally write its reply to the REP
> socket, without blocking. Of course there is no client, so this write goes
> into a black hole.
> My guess is that the events change with ZMQ_POLLOUT is somehow being
> backlogged, and the client disconnect helps push the queue another step
> forward. I found that if, immediately after reading from the REP socket, I
> query ZMQ_EVENTS, then I can see the ZMQ_POLLOUT being flagged even though I
> never got a read indication on the fd.
> Does this mean that maybe I need to check ZMQ_EVENTS not only after read
> indications on the fd, but also after anytime I call zmq_recv() ?

I've not tried REP sockets with asynchronous event loop (XREP usually
needed). But I'm pretty sure, you're right. You need to recheck
ZMQ_EVENTS after doing zmq_recv(), as the state of the socket changes
at that time (it's not writable before not because of network issues
but because of state machine).

However, checking ZMQ_EVENTS after each zmq_recv and zmq_send is
needed anyway, as described in current documentation and in this ML
thread. And it doesn't sounds like possible to change in any future
version of zeromq.


More information about the zeromq-dev mailing list