[zeromq-dev] Non-blocking calls and Twisted

Brian Granger ellisonbg at gmail.com
Mon Apr 12 20:37:02 CEST 2010


Laurens,

Perhaps I can rephrase your question and then answer it....

How would one use pyzmq with twisted?

Because 0MQ/pyzmq implement their own poll/select functions, you will need to
create a new reactor type that calls the zmq poll/select function.
Then you will need
to wrap the pyzmq Socket type into something that Twisted knows how to
handle.  I
forget the exact details in twisted, but I think this will involve the
Transport API.
I would love to ship a 0MQ reactor with pyzmq itself.  I have already worked on
a pyzmq event loop for tornado, and I use twisted a lot, so this
definitely makes sense.
I am pretty busy right now, but I am more than willing to help in any way I can.

Cheers,

Brian



On Mon, Apr 12, 2010 at 5:34 AM, Laurens Van Houtven <lvh at laurensvh.be> wrote:
> Dear Martin,
>
> Thanks for your answer :-) Specifically ZMQ_NOBLOCK is interesting. I don't
> really care how 0MQ does stuff under the hood as long as it performs, but
> I'd like a Python API that behaves event-drivenly (i.e. non-blocking IO and
> callbacks) if possible, if only because Twisted contains fully functional
> SSL and HTTP servers (and code I don't have to write is way better than code
> I do have to write).
> thanks
> Laurens
>
> On Mon, Apr 12, 2010 at 8:14 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
>>
>> Hi Laurens,
>>
>> > I've noticed that 0MQ implementations are (at least the ones I've seen)
>> > exclusively blocking connection per thread. Is there a particular reason
>> > for this? I'd like to use Twisted and talk to 0MQ (from the role as a
>> > publisher) asynchronously. Are there any important caveats there?
>>
>> I am not 100% sure what you are asking about so let me give couple of
>> answers and you pick the right one :)
>>
>> 1. zmq_send/zmq_recv calls are not necessarily blocking. Use ZMQ_NOBLOCK
>> flag to make them non-blocking.
>>
>> 2. If you want to handle many physical connections at the same time, 0MQ
>> is doing this under the hood. Single 0MQ socket can be backed by 10,000
>> TCP connections.
>>
>> 3. If you still need multiple 0MQ sockets you can multiplex using
>> zmq_poll() function.
>>
>> 4. If you want to multiplex between 0MQ sockets and POSIX sockets, you
>> can still use zmq_poll().
>>
>> 5. If you still need to use native multiplexing mechanism, it's not
>> possible. There's an explanation on FAQ page:
>>
>> "Why can't I use standard I/O multiplexing functions such as select() or
>> poll() on ØMQ sockets?"
>>
>> "ØMQ socket is not a standard POSIX socket. It would be great if it was,
>> however, POSIX doesn't provide a mechanism to simulate file descriptors
>> in user space. To convert ØMQ sockets into POSIX file descriptors we
>> would have either to move ØMQ to kernel-space or hack the kernel to
>> provide the functionality needed. In both cases we would have to
>> sacrifice portability and stick to a single operating system."
>>
>> Martin
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com



More information about the zeromq-dev mailing list