[zeromq-dev] Oops! I broke pyczmq

Greg Ward greg at gerg.ca
Mon Feb 3 16:42:30 CET 2014


Oops: I submitted a pull request to pyczmq last week which broke
isolated imports. Example:

  $ python -m pyczmq.zauth
  Traceback (most recent call last):
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
      "__main__", fname, loader, pkg_name)
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
      exec code in run_globals
    File "/usr/local/zmq/lib/python2.7/site-packages/pyczmq-0.0.4-py2.7.egg/pyczmq/zauth.py", line 16, in <module>
      @cdef('zauth_t * zauth_new (zctx_t *ctx);')
    File "/usr/local/zmq/lib/python2.7/site-packages/pyczmq-0.0.4-py2.7.egg/pyczmq/_cffi.py", line 17, in cdef
      ffi.cdef(decl)
    File "/usr/local/zmq/lib/python2.7/site-packages/cffi/api.py", line 102, in cdef
      self._parser.parse(csource, override=override)
    File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py", line 154, in parse
      self._internal_parse(csource)
    File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py", line 159, in _internal_parse
      ast, macros = self._parse(csource)
    File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py", line 126, in _parse
      self.convert_pycparser_error(e, csource)
    File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py", line 148, in convert_pycparser_error
      raise api.CDefError(msg)
  cffi.api.CDefError: cannot parse "zauth_t * zauth_new (zctx_t *ctx);"
  :3:29: before: *

I suspected my change was at fault, and git bisect confirms it:

  commit 175520bd3b2562089148f8bd66c9ebc8720e14b2
  Author: Greg Ward <greg at gerg.ca>
  Date:   Thu Jan 16 17:10:54 2014 -0500
  
      Make "import pyczmq" faster (backwards incompatible change!).

Ooops!

The problem is apparently that zauth.py (and several other modules:
zbeacon, zcertstore, ...) implicitly depends on import-time
side-effects in zctx.py. I've found two possible fixes:

1) duplicate the required declaration:

--- a/pyczmq/zauth.py
+++ b/pyczmq/zauth.py
@@ -1,5 +1,6 @@
 from pyczmq._cffi import C, ffi, cdef, ptop
 
+cdef('typedef struct _zctx_t zctx_t;')
 
 CURVE_ALLOW_ANY = "*"

This is yucky because it duplicates code from zctx.py, but it's pretty
minimal.

2) add an import:

--- a/pyczmq/zauth.py
+++ b/pyczmq/zauth.py
@@ -1,4 +1,5 @@
 from pyczmq._cffi import C, ffi, cdef, ptop
+from pyczmq import zctx
 
 
 CURVE_ALLOW_ANY = "*"

And this is yucky because it means we have an unused import which is
necessary for its import-time side effect. Yuck.

Oh yeah: the fix has to be duplicated in all affected modules:
zbeacon, zcertstore, etc.

Also, adding tests to verify the fix is a bit gross. The problem is
that nose seems to import pyczmq.zctx before it gets around to running
any tests, so I can't reproduce the crash without clever trickery.
(Either 1) run a new interpreter in a subprocess or 2) remove pyczmq
modules from sys.modules.)

Anyone have a better idea on how to fix this? Or how to write a test
that reproduces it and verifies the fix?

       Greg



More information about the zeromq-dev mailing list