[zeromq-dev] 0MQ 1.0.1 Python Global Interpreter Lock patch?

Aamir M intuitionist at gmail.com
Fri Oct 16 23:16:27 CEST 2009


Hi Ben,

In regards to the code review that you mention, did you have a look at
txAMQP by any chance? txAMQP is a new Python client for AMQP based  on
the Twisted framework. There is an interesting discussion about it at
http://www.apparatusproject.org/blog/2009/04/creating-a-simple-amqp-client-using-txamqp/
...

Specifically the author claims that, "Being event-driven is important
when implementing AMQP in your project, because it creates a
concurrency model using non blocking calls." So he is against using
blocking calls and instead prefers to use callbacks. And apparently
the Twisted "concurrency model" consists of polling for messages and
then going sleep for some amount of time, then polling again .... That
sounds quite inferior to the concurrency model that we have with 0MQ
(i.e. making 0MQ blocking calls, wrapped with Py_BEGIN_ALLOW_THREADS /
Py_END_ALLOW_THREADS).

Aamir


On Thu, Oct 15, 2009 at 11:54 PM, Ben Dyer <ben.dyer at taguchimail.com> wrote:
> Aamir,
>
> I'm using a similar modification to the 0MQ/2.0 Alpha3 libpyzmq to support
> multithreaded operation, and the patch been working as expected for me.
> Although I don't have a complete understanding of the 0MQ threading model,
> my review of the source didn't turn up any means by which disabling the GIL
> could cause issues that wouldn't appear in multithreaded C/C++ apps (or,
> conversely, issues that wouldn't appear in multithreaded Python apps using
> network code other than 0MQ).
>
> Regards,
> Ben
>
> On 16/10/2009, at 03:21 , Aamir M wrote:
>
>> Hello,
>>
>> I am trying to use the 0MQ 1.0.1 Python API to send / recv 0MQ
>> messages in one Python thread while driving a GUI in another Python
>> thread. However, whenever I use the blocking send / recv 0MQ calls,
>> the whole Python process freezes and waits for network I/O (so the GUI
>> becomes non-responsive, etc). It seems that pyzmq.cpp
>> (http://github.com/sustrik/zeromq1/blob/master/libpyzmq/pyzmq.cpp) is
>> not thread friendly. I was able to get around the problem by modifying
>> the pyZMQ_receive and pyZMQ_send functions .... specifically, line 176
>> in pyzmq.cpp reads:
>>
>> sent = self->api_thread->send (exchange, message, block);
>>
>> I replaced this with:
>>
>> Py_BEGIN_ALLOW_THREADS
>> sent = self->api_thread->send (exchange, message, block);
>> Py_END_ALLOW_THREADS
>>
>> Similarly, I changed line 192 to:
>>
>> int queue;
>> Py_BEGIN_ALLOW_THREADS
>> queue = self->api_thread->receive (&message, block);
>> Py_END_ALLOW_THREADS
>>
>>
>> In summary, I just simply the 0MQ C++ blocking calls with the Python
>> macros Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS, which are
>> described in http://docs.python.org/c-api/init.html ... These macros
>> release and reacquire the CPython Global Interpreter Lock.
>>
>>
>> Is this a bug in the 0MQ 1.0.01 library that should be patched? Or is
>> my patch somehow incorrect?
>>
>> Thanks,
>> Aamir
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>



More information about the zeromq-dev mailing list