[zeromq-dev] Incompatibility between PyZMQ and uWSGI
Jaime Fernández
jjjaime at gmail.com
Thu Nov 25 13:13:30 CET 2010
Brian,
I thought that it was a problem like you commented but now I'm not so sure.
I also tried in the uWSGI mail list, and made some tests to isolate the
cause of the problem. ZMQ actually works with uWSGI and the problem I found
is really bizarre.
Here it is the reason (although no idea how to fix it) which you could also
see at http://lists.unbit.it/pipermail/uwsgi/2010-November/000912.html
----
According to ZMQ specification, the ZMQ context should be shared to create
all the ZMQ sockets of the same process. What we did, is to create a Python
module which creates such ZMQ context:
# zmq_context.py
import zmq
context = zmq.Context()
If you create the following WSGI app:
import zmq
import zmq_context
def application(environ, start_response):
context = zmq_context.context
socket_push = context.socket(zmq.PUSH)
socket_push.connect('tcp://localhost:9876')
socket_push.send_multipart(["Hello", "World"])
Then uWSGi dies when zmq tries to connect to the ZQM socket.
However, if we don't use the zmq_context module and create the context
directly:
import zmq
def application(environ, start_response):
context = zmq.Context()
socket_push = context.socket(zmq.PUSH)
socket_push.connect('tcp://localhost:9876')
socket_push.send_multipart(["Hello", "World"])
Everything works.
----
Thanks,
Jaime
On Tue, Nov 23, 2010 at 9:06 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Jaime,
>
> I think the issue is that uWSGI likely uses an asynchronous event loop
> underneath with non-blocking sockets. When zeromq/pyzmq is used in that
> context, zeromq socket have to be integrated into the event loop at a very
> low level using:
>
> 1) zmq_poll (for zeromq 2.0.x and 2.1.x)
> 2) ZMQ_FD/ZMQ_EVENTS (for zeromq 2.1.x)
>
> The exact point of conflict is that both zeromq and uWSGI are using
> something like poll to handle IO in the event loop. The two event loops
> don't know how to coordinate their activities and end up stepping on each
> other's toes. The only libraries that I know of that handle this (in python
> that is) are:
>
> 1) pyzmq.eventloop.ioloop (a tornado-compatible event loop).
> 2) eventlet
>
> Hope this helps.
>
> Brian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20101125/e34c51fa/attachment.htm>
More information about the zeromq-dev
mailing list