[zeromq-dev] Multicore Magic

Nicholas Piël nicholas at nichol.as
Mon Apr 26 12:00:31 CEST 2010


On Apr 25, 2010, at 11:56 PM, Joe Holloway wrote:

> Hey guys,
> 
>>> * Python still has the Global interpreter lock that prevents more than
>>> one thread from executing Python code at a time.  This means that
>>> thread based concurrency is not an option for Python.  IOW, we can't
>>> use multiple threads that talk using OMQ.  So, we have to use
>>> processes.  I don't like that, but maybe it is OK...
>> 
>> Well, it's deficiency of Python. Hopefuly Python developers would
>> address the issue in the future.
>> 
>> Btw, maybe it would be worth poking them a little by asking about the
>> behaviour on python dev mailing list...
> 
> Probably preaching to the choir, but the GIL is only a limitation of
> the CPython runtime.  There are other (albeit less mature) Python
> implementations that are not necessarily affected by the GIL, e.g.
> Jython, IronPython, Stackless, PyPy, et al.  Also, there's the Google
> sponsored "Unladen Swallow" project that aims to reinvent the CPython
> runtime so as to improve performance and remove the GIL.

I personally think the GIL is more of a blessing than a curse, as it allows Python to quickly integrate with C-extensions.

From your list:

Stackless and PyPy do in fact have a GIL.

Jython doesn't because it makes use of the Java environment but this also means that you cannot use C extensions. If you want to use Jython and ZMQ you should look at at the Java bindings.

IronPython runs on Mono and does indeed not have a GIL, but again C extensions with IronPython are problematic. You could try IronClad but i suspect this will increase the latency.

Concerning Unladen Swallow, they are no longer optimistic about removing the Python GIL, see here for more info: http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#Global_Interpreter_Lock


Cheers,
Nicholas Piël


More information about the zeromq-dev mailing list