[zeromq-dev] ZMQ_RCVMORE Set, But recv() Blocks

[Gregory Szorc]

I have an issue with recv() blocking when ZMQ_RCVMORE is set that I think 
might be a bug. Before I file a bug or send a full repro case, I thought I'd 
run it by the community to see if I have my head screwed on right.

The gist of my code is as follows:

   zmq::message_t msg;
   int64 more;
   size_t moresz = sizeof(more);

   zmq::poll(&pollitems[0], N, 0);

    if (pollitems[0].revents & ZMQ_POLLIN) {
        while (true) {
            // false only return if EAGAIN is set, which is impossibe since 
we are in blocking mode
            this->sock->recv(&msg, 0);

            moresz = sizeof(more);
            // all errors on getsockopt() are converted to exceptions in 
C++, so don't need to check bool (at least in this example)
            this->sock->getsockopt(ZMQ_RCVMORE, &more, &moresz);
            if (!more) break;
        }
    }

What I'm seeing is a handful of messages come through on the socket (an 
inproc:// bound to this thread with data coming from other threads - yes 
bind() is being called before connect()). Eventually, getsockopt() says 
RCVMORE is set, but when I call recv(), it blocks. A stack trace shows it is 
stuck in libc's recv().

I can get around the problem by setting NOBLOCK in the recv() flags, but 
that doesn't change the fact that the socket says there is a message to be 
read, but it blocks forever trying to read it. It is my understanding that 
if RCVMORE is set (or a revents & POLLIN for that matter), a blocking call 
to recv() should always return immediately. Is this not true? If so, what 
conditions can cause this to happen? Are they documented?

The condition is present on Linux (2.6.35 x86_64), but not Windows (although 
the difference could be attributed to my program, not 0MQ). I can reproduce 
on 2.0.10 and master (325dd2f0914de502ae7687f94927fa98c20380c9).

Like I said, I can likely boil this down to a full repro, but it will take 
some time and I'd prefer to not spend time doing it unless someone thinks 
this might be a legit bug instead of a stupid user problem.

Greg
gregory.szorc at gmail.com