[zeromq-dev] zmq_poll

Serge Aleynikov serge at aleynikov.org
Wed May 19 20:24:44 CEST 2010


Martin,

I am currently implementing the three functions we discussed in this 
thread, and consequently figuring out the implementation of the 
dispatcher.  You mentioned that there's only one file descriptor 
involved in handling 0MQ sockets.  I must be looking in the wrong place 
- from what I see a context (a.k.a. dispatcher) has a pool of 
app_threads and io_threads, and each app_thread has a signaler from 
which you can get a pollable fd.  Since there can be several app_threads 
it looks to me that there's more than one potential 0MQ fd.  Or is it 
that poll() must be called in each app_thread independently and will 
only return status of sockets belonging to that app_thread?  I suppose 
if that's the case, the Dhammika's driver for Erlang VM that I am fixing 
now would always have to have one app_thread.

Secondly in order to get the fd through 
app_thread->get_signaler()->get_fd() you must know app_thread.  The 
app_thread you can easily get from a socket, but if the only thing you 
are given is a context, the dispatcher is missing 
get_current_app_thread() method, so I suppose it has to be added (it'll 
just iterate through app_threads looking for a match to thread_t::id()). 
Correct?

It would also be useful if you explained about the design how the 
transport connections are managed when you have multiple app_threads 
having different ZMQ_REQ sockets - would it involve different transport 
connections to the peer ZMQ_REP socket, or they can all be sending 
messages through a single transport connection (if we are talking, of 
course, about homogeneous peer connections, e.g. TCP only)?  Is there 
some architecture document that explains this?

Serge

On 5/11/2010 3:35 PM, Martin Sustrik wrote:
> Hi Serge,
>
>>   >  Several points:
>>   >
>>   >  1. In the long run, 0MQ sockets should be turned into standard POSIX
>>   >  sockets so that there would be no need for zmq_poll anymore.
>>
>> Yes, this would make a lot of sense!
>>
>>   >  3. The API you propose doesn't match how things work inside 0MQ.
>>   >  There's only one file descriptor involved etc. Let me propose
>>   >  alternative interface:
>>   >     //  Returns file descriptor you can poll on (POLLIN) to find
>>   >     //  out whether _something_ is to happen within specified
>>   >     //  0MQ context. When the fd is signaled, zmq_process should
>>   >     //  be called to do the outstanding work. You should never
>>   >     //  read from the file descriptor yourself.
>>   >     int zmq_wait_fd (void *context_);
>>
>> I might be missing something but wouldn't it have to be:
>>
>>       int zmq_wait_fd (void *socket_);
>>                              ^^^^^^^
>> i.e. isn't there an fd per socket (0MQ's or other)?
>
> No, there is not. FD subsystem is too slow to be used in 0MQ. Instead
> there are lock-free queues to pass messages between I/O threads and
> application threads.
>
>> Also I saw the app_thread->process_commands() call inside zmq_poll to
>> handle 0MQ messages, would we also need to register that signaling fd
>> with an external loop as well?
>
> Yes, that's the only fd you have to handle. It's used only to pass
> "commands" to your application thread. Commands are notifications like
> "here's a new lock-free queue to read messages from" etc.
>
> So, proposed zmq_wait_fd would return this fd.
>
>>   >     //  Process whatever outstanding work within 0MQ context.
>>   >     int zmq_process (void *context_);
>
> Once we know there's a command available for our thread (IN is signaled
> on the fd) we can call zmq_process that will process all the commands
> waiting for the application thread.
>
> This means that new connections are going to be taken into account etc.
>
>> I am a little unclear with this call as to how would the external loop
>> set the context->revents to indicate POLLIN/POLLOUT activity, so that
>> zmq_process would be able to know if this is a read/write activity.
>> Wouldn't it need to be:
>>
>>        int zmq_process (void *context_, int events);
>>
>>        where events is a bitmask of POLLIN/POLLOUT
>
> I think the above explains that zmq_process is intended to work in a
> different way.
>
>>   >     //  Returns IN event, OUT event or both for the particular
>>   >     //  0MQ socket.
>>   >     int zmq_events (void *socket_);
>>
>> I think I am mis-reading your suggested API, as I don't see the purpose
>> of this one?
>
> It allows you to check whether there are messages to read from
> particular socket and/or whether messages can be written to the socket.
>
> One important point: You cannot use the file descriptor (the one
> discussed above) to "wait for a message".
>
> Say there are two messages available. You read one of them. If you wait
> for the fd at this point it may never unblock, because there's still one
> message to read. Command saying "there are new messages available" is
> sent to your thread only if you've read all the messages before. You can
> think of it as edge-triggered polling.
>
>>   >  Would that work with Erlang VM?
>>
>> Erlang has a function used to provide the VM with events to check for on
>> a given fd:
>>
>>        driver_select(..., int event, int mode, int on)
>>                                ^          ^         ^
>>                                |          |         |
>>                               fd    POLLIN/OUT   Turn on/off
>
> Ok, so it can POLLIN on our internal file descriptor.
>
>> And the VM calls one of two callbacks when there's activity
>> detected:
>>
>>        void (*ready_input)(..., int event);
>>        void (*ready_output)(..., int event);
>
> When ready_input is invoked, you can call zmq_process. That updates the
> state of all the sockets owned by this thread.
>
> At this point it's easy to find out whether particular socket is ready
> for reading/writing using zmq_events.
>
> Sounds complex? :) Maybe turning 0MQ sockets into full-blown fds is
> easier? It's up to you!
>
> Martin
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list