[zeromq-dev] Multicore Magic
nicholas at nichol.as
Mon Apr 26 12:00:31 CEST 2010
On Apr 25, 2010, at 11:56 PM, Joe Holloway wrote:
> Hey guys,
>>> * Python still has the Global interpreter lock that prevents more than
>>> one thread from executing Python code at a time. This means that
>>> thread based concurrency is not an option for Python. IOW, we can't
>>> use multiple threads that talk using OMQ. So, we have to use
>>> processes. I don't like that, but maybe it is OK...
>> Well, it's deficiency of Python. Hopefuly Python developers would
>> address the issue in the future.
>> Btw, maybe it would be worth poking them a little by asking about the
>> behaviour on python dev mailing list...
> Probably preaching to the choir, but the GIL is only a limitation of
> the CPython runtime. There are other (albeit less mature) Python
> implementations that are not necessarily affected by the GIL, e.g.
> Jython, IronPython, Stackless, PyPy, et al. Also, there's the Google
> sponsored "Unladen Swallow" project that aims to reinvent the CPython
> runtime so as to improve performance and remove the GIL.
I personally think the GIL is more of a blessing than a curse, as it allows Python to quickly integrate with C-extensions.
From your list:
Stackless and PyPy do in fact have a GIL.
Jython doesn't because it makes use of the Java environment but this also means that you cannot use C extensions. If you want to use Jython and ZMQ you should look at at the Java bindings.
IronPython runs on Mono and does indeed not have a GIL, but again C extensions with IronPython are problematic. You could try IronClad but i suspect this will increase the latency.
Concerning Unladen Swallow, they are no longer optimistic about removing the Python GIL, see here for more info: http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#Global_Interpreter_Lock
More information about the zeromq-dev