[zeromq-dev] Incompatibility between PyZMQ and uWSGI

Jaime Fernández jjjaime at gmail.com
Thu Nov 25 13:13:30 CET 2010


Brian,

I thought that it was a problem like you commented but now I'm not so sure.
I also tried in the uWSGI mail list, and made some tests to isolate the
cause of the problem. ZMQ actually works with uWSGI and the problem I found
is really bizarre.

Here it is the reason (although no idea how to fix it) which you could also
see at http://lists.unbit.it/pipermail/uwsgi/2010-November/000912.html
----

According to ZMQ specification, the ZMQ context should be shared to create
all the ZMQ sockets of the same process. What we did, is to create a Python
module which creates such ZMQ context:

# zmq_context.py
import zmq
context = zmq.Context()

If you create the following WSGI app:

import zmq
import zmq_context
def application(environ, start_response):
    context = zmq_context.context
    socket_push = context.socket(zmq.PUSH)
    socket_push.connect('tcp://localhost:9876')
    socket_push.send_multipart(["Hello", "World"])

Then uWSGi dies when zmq tries to connect to the ZQM socket.

However, if we don't use the zmq_context module and create the context
directly:

import zmq
def application(environ, start_response):
    context = zmq.Context()
    socket_push = context.socket(zmq.PUSH)
    socket_push.connect('tcp://localhost:9876')
    socket_push.send_multipart(["Hello", "World"])

Everything works.

----


Thanks,
Jaime

On Tue, Nov 23, 2010 at 9:06 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Jaime,
>
> I think the issue is that uWSGI likely uses an asynchronous event loop
> underneath with non-blocking sockets.  When zeromq/pyzmq is used in that
> context, zeromq socket have to be integrated into the event loop at a very
> low level using:
>
> 1) zmq_poll (for zeromq 2.0.x and 2.1.x)
> 2) ZMQ_FD/ZMQ_EVENTS (for zeromq 2.1.x)
>
> The exact point of conflict is that both zeromq and uWSGI are using
> something like poll to handle IO in the event loop.  The two event loops
> don't know how to coordinate their activities and end up stepping on each
> other's toes.  The only libraries that I know of that handle this (in python
> that is) are:
>
> 1) pyzmq.eventloop.ioloop (a tornado-compatible event loop).
> 2) eventlet
>
> Hope this helps.
>
> Brian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20101125/e34c51fa/attachment.htm>


More information about the zeromq-dev mailing list