[zeromq-dev] Non-copying send/recv in Python bindings

Brian Granger ellisonbg at gmail.com
Thu May 27 23:59:30 CEST 2010


Hi,

With much help from Nicholas Piël., we have figured out how to support
non-copying send and receives in the Python bindings.  The performance
difference is significant, especially on systems with small memory
bandwidth.  I have included a figure of the performance comparisons
that Nicholas did on his machine that compares Python (copy and
non-copy) to C++ (thanks Nicholas for this figure!).  For messages
above 512K, the Python bindings mostly match the speed of C++ on
latency.

We are still tweaking some aspects of this, but we wanted to get this
out so others can begin playing with it.  The logic to get all of this
to work properly was quite subtle, mainly because of the need to
simultaneously manage the Python and 0MQ ref counts.  This logic could
be helpful to other language bindings.  We ended up implementing a
Message object, which allows us to do things like:

>>> m = s1.recv(copy=False)
>>> type(m)
Message
>>> s2.send(copy=False)

Without ever making a copy.

Here is the code:

http://github.com/ellisonbg/pyzmq

Cheers,

Brian


-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: latency.png
Type: image/png
Size: 44839 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20100527/3430a527/attachment.png>


More information about the zeromq-dev mailing list