[zeromq-dev] missing events ZMQ_FD / ZMQ_EVENTS

Paul Colomiets paul at colomiets.name
Wed Jun 27 21:44:45 CEST 2012


Hi Justin,

On Tue, Jun 26, 2012 at 2:16 AM, Justin Karneges <justin at affinix.com> wrote:
> On Wednesday, May 02, 2012 03:27:42 AM Paul Colomiets wrote:
>> Ok. Behind the scenes ZMQ_FD, is basically a counter, which wakes up
>> poll when is non-zero. The counter is reset on each getsockopt ZMQ_EVENTS,
>> zmq_send and zmq_recv.
>>
>> The following diagram shows race condition with two sockets A and B,
>> in a scenario similar to yours:
>>
>> https://docs.google.com/drawings/d/1F97jpdbYMjjb6-2VzRtiL2LpHy638-AEOyrUX84
>> HL78/edit
>>
>> Note: the last poll is entered with both counters set to zero, so it
>> will not wake up, despite the fact that there is pending message.
>
> Was there ever a resolution on this?
>
> I am using ZMQ_FD now to integrate into an event loop, and I am seeing some
> odd behavior when testing a hello world REQ/REP on the REP side.
>
> The REP server binds and waits for data. The fd is indicated as readable
> twice. First, the events are 0 (maybe this happens when the client connects?),
> then the events are 1 (ZMQ_POLLIN). The server considers the REP socket
> readable and so it reads a message without blocking. Now it wants to reply,
> but it considers the socket not yet writable. I was expecting that after
> reading from the socket, the fd would be indicated as readable and the events
> would be 2 (ZMQ_POLLOUT). However, this event never comes and so the server
> just idles.
>
> Now here's where it gets weird: if I kill the client (which was also waiting
> around, as it never got a reply), then the server gets new events with
> ZMQ_POLLOUT set. This causes the server to finally write its reply to the REP
> socket, without blocking. Of course there is no client, so this write goes
> into a black hole.
>
> My guess is that the events change with ZMQ_POLLOUT is somehow being
> backlogged, and the client disconnect helps push the queue another step
> forward. I found that if, immediately after reading from the REP socket, I
> query ZMQ_EVENTS, then I can see the ZMQ_POLLOUT being flagged even though I
> never got a read indication on the fd.
>
> Does this mean that maybe I need to check ZMQ_EVENTS not only after read
> indications on the fd, but also after anytime I call zmq_recv() ?
>

I've not tried REP sockets with asynchronous event loop (XREP usually
needed). But I'm pretty sure, you're right. You need to recheck
ZMQ_EVENTS after doing zmq_recv(), as the state of the socket changes
at that time (it's not writable before not because of network issues
but because of state machine).

However, checking ZMQ_EVENTS after each zmq_recv and zmq_send is
needed anyway, as described in current documentation and in this ML
thread. And it doesn't sounds like possible to change in any future
version of zeromq.

-- 
Paul



More information about the zeromq-dev mailing list