[zeromq-dev] Fwd: Re: New Python bindings

Brian Granger ellisonbg at gmail.com
Tue Feb 16 00:33:56 CET 2010


I implemented the GIL release and Py_INCREF/PyDECREF stuff to try to
do a proper non-copy
send.  It works in some cases, but deadlocks in others (specifically
the latency benchmark).  My
guess is that when two processes are sending message back and forth
and releasing/acquiring the GIL
they end up deadlocking each other in some subtle manner.  I have
removed this for now and
am just doing the copy.  I might revisit this later, but the
performance benefit (possibly 0 or worse)
is not worth the effort at this point.



On Mon, Feb 15, 2010 at 2:22 AM, Jon Dyte <jon at totient.co.uk> wrote:
> On Monday 15 Feb 2010, you wrote:
>> Jon Dyte wrote:
>> > On the other hand it might  be cheaper to just take the hit of the copy,
> as
>> > using the ref count and free function, means the io thread would end up
>> > calling into the python runtime, which might slow things down more and
>> > possibly cause other threading issues.
>> Right. However, it depends on the Python runtime. Would it survive being
>> called from two different threads? If not so, making a copy of the
>> message is much more efficient than wrapping each call in a mutex.
>> Martin
> Python runtime has one big lock called the GIL(global-interpreter-lock).
> This means that python only really runs one python thread(ie started from
> python code ) at a time
> (some might say that's not threaded at all ;-) ) and they get some sort of
> timeslice(number of byte code instructions?) before yielding.
> Note: AFAIK Jython and IronPython (JVM and .Net) threads work more as
> expected.
> When writing a c-module extension calling into a foreign interface, you can
> release the GIL.
> The  io-thread will have to acquire GIL in order to free it up.
> So in this case on balance I'd think I would take the hit of the copy.
> Jon

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com

More information about the zeromq-dev mailing list