[zeromq-dev] Oops! I broke pyczmq

Michel Pelletier pelletier.michel at gmail.com
Mon Feb 3 18:26:39 CET 2014


I have a branch which reverts your import changes and switches to
cffi.verify(), which speeds up the import time significantly.

-Michel


On Mon, Feb 3, 2014 at 7:42 AM, Greg Ward <greg at gerg.ca> wrote:

> Oops: I submitted a pull request to pyczmq last week which broke
> isolated imports. Example:
>
>   $ python -m pyczmq.zauth
>   Traceback (most recent call last):
>     File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
>       "__main__", fname, loader, pkg_name)
>     File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
>       exec code in run_globals
>     File
> "/usr/local/zmq/lib/python2.7/site-packages/pyczmq-0.0.4-py2.7.egg/pyczmq/zauth.py",
> line 16, in <module>
>       @cdef('zauth_t * zauth_new (zctx_t *ctx);')
>     File
> "/usr/local/zmq/lib/python2.7/site-packages/pyczmq-0.0.4-py2.7.egg/pyczmq/_cffi.py",
> line 17, in cdef
>       ffi.cdef(decl)
>     File "/usr/local/zmq/lib/python2.7/site-packages/cffi/api.py", line
> 102, in cdef
>       self._parser.parse(csource, override=override)
>     File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py",
> line 154, in parse
>       self._internal_parse(csource)
>     File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py",
> line 159, in _internal_parse
>       ast, macros = self._parse(csource)
>     File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py",
> line 126, in _parse
>       self.convert_pycparser_error(e, csource)
>     File "/usr/local/zmq/lib/python2.7/site-packages/cffi/cparser.py",
> line 148, in convert_pycparser_error
>       raise api.CDefError(msg)
>   cffi.api.CDefError: cannot parse "zauth_t * zauth_new (zctx_t *ctx);"
>   :3:29: before: *
>
> I suspected my change was at fault, and git bisect confirms it:
>
>   commit 175520bd3b2562089148f8bd66c9ebc8720e14b2
>   Author: Greg Ward <greg at gerg.ca>
>   Date:   Thu Jan 16 17:10:54 2014 -0500
>
>       Make "import pyczmq" faster (backwards incompatible change!).
>
> Ooops!
>
> The problem is apparently that zauth.py (and several other modules:
> zbeacon, zcertstore, ...) implicitly depends on import-time
> side-effects in zctx.py. I've found two possible fixes:
>
> 1) duplicate the required declaration:
>
> --- a/pyczmq/zauth.py
> +++ b/pyczmq/zauth.py
> @@ -1,5 +1,6 @@
>  from pyczmq._cffi import C, ffi, cdef, ptop
>
> +cdef('typedef struct _zctx_t zctx_t;')
>
>  CURVE_ALLOW_ANY = "*"
>
> This is yucky because it duplicates code from zctx.py, but it's pretty
> minimal.
>
> 2) add an import:
>
> --- a/pyczmq/zauth.py
> +++ b/pyczmq/zauth.py
> @@ -1,4 +1,5 @@
>  from pyczmq._cffi import C, ffi, cdef, ptop
> +from pyczmq import zctx
>
>
>  CURVE_ALLOW_ANY = "*"
>
> And this is yucky because it means we have an unused import which is
> necessary for its import-time side effect. Yuck.
>
> Oh yeah: the fix has to be duplicated in all affected modules:
> zbeacon, zcertstore, etc.
>
> Also, adding tests to verify the fix is a bit gross. The problem is
> that nose seems to import pyczmq.zctx before it gets around to running
> any tests, so I can't reproduce the crash without clever trickery.
> (Either 1) run a new interpreter in a subprocess or 2) remove pyczmq
> modules from sys.modules.)
>
> Anyone have a better idea on how to fix this? Or how to write a test
> that reproduces it and verifies the fix?
>
>        Greg
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140203/cc722b37/attachment.htm>


More information about the zeromq-dev mailing list