[zeromq-dev] pyczmq import overhead

Greg Ward greg at gerg.ca
Thu Jan 16 18:53:13 CET 2014


Speaking of pyczmq, it is *very* slow to import:

  $ time python -c 'import pyczmq'         
  python -c 'import pyczmq'  1.11s user 0.05s system 100% cpu 1.152 total

Turns out this is because pyczmq/__init__.py imports every submodule,
as well as some identifiers in a couple of submodules. I'm pretty sure
this is considered an anti-pattern. (I dimly recall seeing Guido van
Rossum disparage the practice on python-dev many years ago.)

I've come to agree with him, if only on performance grounds. E.g.
here's the best case:

  $ echo -n > pyczmq/__init__.py
  $ time python -c 'import pyczmq'
  python -c 'import pyczmq'  0.03s user 0.01s system 99% cpu 0.039 total

A more realistic test:

  $ time python -c 'from pyczmq import zmq, zctx, zsocket'
  python -c 'from pyczmq import zmq, zctx, zsocket'  0.46s user 0.05s system 100% cpu 0.508 total

Or, for all the submodules needed to implement Pieter's "ironhouse"
example in Python:

  $ time python -c 'from pyczmq import zmq, zctx, zsocket, zcert, zauth, zstr'
  python -c 'from pyczmq import zmq, zctx, zsocket, zcert, zauth, zstr' 0.61s user 0.03s system 100% cpu 0.640 total

That's not great, but 0.6 sec is still better than 1.1 sec.

Is pyczmq young enough that we can consider breaking backwards
compatibility like this? Anyone currently doing

  import pyczmq

  ctx = pyczmq.zctx.new()
  sock = pyczmq.zsocket.new(...)

would have to change their imports to:

  import pyczmq.zctx
  import pyczmq.zsocket

If that's OK, I'll work up a patch and send it.

       Greg



More information about the zeromq-dev mailing list