[zeromq-dev] zmq_poll

Dhammika Pathirana dhammika at gmail.com
Thu May 13 09:18:41 CEST 2010


On 5/12/10, Serge Aleynikov <serge at aleynikov.org> wrote:
> Hi Dhammika,
>
>  This was indeed my starting point ;-)

Cool!

>
>  After taking a quick look at the driver code I saw a rather big stumbling
> block there that there was a direct call to zmq_recv which would potentially
> block the Erlang VM (that is especially dangerous if Erlang is running with
> no SMP support).  So there are two solutions to this - implement
> asynchronous invocation of zmq_poll in a separate thread controlled by the
> driver, and signaling event completion to the driver or doing it the
> "proper" way of registering 0MQ's file descriptors with Erlang VM, so that
> its event loop would be responsible for event activity detection.
>


I'm still learning Erlang, probably have made some newbie mistakes.
My understanding is that Erlang VM spawns a native thread on each CPU
core, and schedules multiple "Erlang processes" on each native thread.
Even on SMP VMs, a blocking system call will make the driver
non-responsive to other Erlang events.


>  I am not much familiar with 0MQ's internals yet, so I don't know if the
> first approach of running zmq_poll in a thread is feasible at all, because
> Erlang has a pool of managed threads, and is free to assign this
> asynchronous operation (that would call zmq_poll) to any one thread in the
> pool.  I saw some checking done inside zmq_poll to ensure that 0MQ sockets
> belong to a particular application thread, which may not work in case of a
> thread pool managed externally since the call would happen in context of
> different threads.
>


Yes, it has to be on the same thread.
I don't know if polling will work here, internal socket is mostly used
for "signaling" IO events. I think a callback would be better, makes
it much easier to add hooks to existing polling APIs.


>  The second approach is much cleaner, and I am hoping to be able to
> implement it when I get to work on it.  When I make some progress I'll
> commit my changes in a branch, so that you can merge them (though I won't
> likely be able to work on this for another couple of weeks).
>
>  Serge
>
>
>  On 5/12/2010 4:32 AM, Dhammika Pathirana wrote:
>
> > Hi Serge,
> >
> > I'm working on an erlang driver, if you're interested check it out.
> > http://github.com/dhammika/erlzmq
> > This is experimental/half-baked code, but most of the basic
> > functionality is there.
> >
> >
> > Dhammika
> >
> >
> >
> >
> > On 5/11/10, Serge Aleynikov<serge at aleynikov.org>  wrote:
> >
> > > Hi,
> > >
> > >  The current design of zmq_poll(3) call assumes the "ownership" of a the
> > >  event objects such that it'll make the underlying poll call on the
> > >  number of event objects (i.e. file descriptors or zmq sockets) and
> block
> > >  for a specified timeout.
> > >
> > >  When integrating this library with other libraries (such as Erlang VM),
> > >  they may have their own event loop and facility to add/remove file
> > >  descriptors along with providing a callback indicating that there's
> > >  activity on given file descriptors.
> > >
> > >  I'd like to see if the following requirement sounds reasonable.
> > >
> > >  Objective
> > >  ---------
> > >  Allow external event loops control ZMQ communications layer.
> > >
> > >
> > >  Implementation
> > >  --------------
> > >  Add two new functions:
> > >
> > >  int zmq_poll_prepare(pollfd* fds_, zmq_pollitem_t *items_, int nitems_,
> > >                       int* zmq_signaler_fd_);
> > >      Initialize the fds_ array of size nitems_ with file
> > >      descriptors from items_.  If there are ZMQ sockets in the items_
> > >      array, *zmq_signaler_fd_ will contain the index of the ZMQ
> signaling
> > >      fd.  Otherwise *zmq_signaler_fd_ will be set to -1.
> > >
> > >  int zmq_poll_check(pollfd* fd_, zmq_pollitem_t* item_, int event_,
> > >                     short type_)
> > >      This function needs to be called after externally detected activity
> > >      on event_.  fd_ and item_ is the pollfd and zmq_pollitem_t structs
> > >      corresponding to event_ item.
> > >      The function will process internal ZMQ commands and update
> > >      item_->revents corresponding to the type_ of event detected.
> > >
> > >  I'd like to hear your feedback about adding this feature and can
> provide
> > >  the implementation if there's interest of including it in the library.
> > >
> > >  Serge
> > >  _______________________________________________
> > >  zeromq-dev mailing list
> > >  zeromq-dev at lists.zeromq.org
> > >  http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > >
> > >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>



More information about the zeromq-dev mailing list