[zeromq-dev] Fwd: Re: New Python bindings

Brian Granger ellisonbg at gmail.com
Mon Feb 15 18:31:11 CET 2010


>> > On the other hand it might  be cheaper to just take the hit of the copy,
> as
>> > using the ref count and free function, means the io thread would end up
>> > calling into the python runtime, which might slow things down more and
>> > possibly cause other threading issues.

Yes, you are absolutely right, the deallocation function will have to
reaquire the GIL
decrease the refcount and than release the GIL.  That will completely stop the
io thread.  On CPython though it is actually worse.  If the
application thread is running
non-GIL releasing extension code (we run into this often!), the io
thread won't be able
to aquire the GIL for deallocation and will stop until the application
thread releases the GIL.
We have situations where that time scale could be *hours*.  I think I
will do this:

I will add a copy keyword argument to the send method and set the
default to True.
If False is set, I will use the dealloc. function that mucks with the
GIL and refcount.
But the default will be to copy.

> Python runtime has one big lock called the GIL(global-interpreter-lock).
> This means that python only really runs one python thread(ie started from
> python code ) at a time
> (some might say that's not threaded at all ;-) ) and they get some sort of
> timeslice(number of byte code instructions?) before yielding.

Yes, the situation with the GIL is horrible.  That is the one of the
things that makes 0MQ
look so attractive - all of the io and queuing takes place in a true
OS level thread
and is not affected by the GIL.  This makes me *very* happy.

> Note: AFAIK Jython and IronPython (JVM and .Net) threads work more as
> expected.
> When writing a c-module extension calling into a foreign interface, you can
> release the GIL.
> The  io-thread will have to acquire GIL in order to free it up.
> So in this case on balance I'd think I would take the hit of the copy.

Thanks for the feedback Jon!



> Jon

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com

More information about the zeromq-dev mailing list