[zeromq-dev] Problems with pyczmq.zstr (was Re: pyczmq and ctx.set_linger())

Michel Pelletier pelletier.michel at gmail.com
Fri Jan 17 22:51:09 CET 2014


It wasn't the recv that was truncating the string, it was the send.

For string that contain null chars, you'll have to use zframe send/recv.
 Here's your example working with those functions:

File Edit Options Buffers Tools Python Help

from pyczmq import zmq, zctx, zsocket, zstr, zframe

ctx = zctx.new()
rep = zsocket.new(ctx, zmq.REP)
zsocket.bind(rep, "tcp://127.0.0.1:5253")

req = zsocket.new(ctx, zmq.REQ)
zsocket.connect(req, "tcp://127.0.0.1:5253")

out_data = "blah blah hey what is \0 doing here?"
out_frame = zframe.new(out_data)
zframe.send(out_frame, req, 0)

in_frame = zframe.recv(rep)
in_data = bytearray(zframe.data(in_frame))
assert in_data == out_data, "expected %r, got %r" % (out_data, in_data)












On Fri, Jan 17, 2014 at 1:20 PM, Michel Pelletier <
pelletier.michel at gmail.com> wrote:

> Ah you've got a good catch there, browsing the docs cffi does support
> variadic function calls
> https://cffi.readthedocs.org/en/release-0.8/#variadic-function-calls
>
> I'll study this after work tonight, I'm sure we can address the issue.
>
> As for the null char in the middle of a received string, I'll have to
> think about that one.  I'm open to suggestions.
>
> -Michel
>
>
> On Fri, Jan 17, 2014 at 12:55 PM, Greg Ward <greg at gerg.ca> wrote:
>
>> On 17 January 2014, Michel Pelletier said:
>> > zstr however is how you turn messages into strings.  It is not an
>> > implementation of a string type.  It is critical to the functionality of
>> > pyczmq (assuming you want string output from the library).
>>
>> I was going to start another thread about this... but what the hell,
>> as long as it came up, here are the two problems I've spotted with
>> pyczmq.zstr.send().
>>
>> 1. zstr_send() is printf()-style, pyczmq.zstr.send() is not
>> -----------------------------------------------------------
>>
>> The C API for zstr_send() is
>>
>>   zstr_send(void *zocket, const char *format, ...)
>>
>> which means calling it like
>>
>>   char *data = [...get a string from somewhere...]
>>   zstr_send(mysocket, data);
>>
>> is wrong, because 'data' might happen to contain a "%s" or "%d" by
>> chance. Presumably the innards of zstr_send() will look for further
>> args on the stack, find gibberish, and send gibberish over the wire.
>> Or crash. The right way to do this in C is
>>
>>   zstr_send(mysocket, "%s", data);
>>
>> But the corresponding Python function is
>>
>>   def send(sock, string):
>>       return C.zstr_send(sock, string)
>>
>> so if 'string' happens to contain "%s" or "%d", bad stuff happens. But
>> since pyczmq.zstr.send() doesn't expose a format string, there's no
>> way to do the right thing.
>>
>> Here's an example:
>>
>>   $ cat zstr-bug1.py
>>   from pyczmq import zmq, zctx, zsocket, zstr
>>
>>   ctx = zctx.new()
>>   rep = zsocket.new(ctx, zmq.REP)
>>   zsocket.bind(rep, "tcp://127.0.0.1:5253")
>>
>>   req = zsocket.new(ctx, zmq.REQ)
>>   zsocket.connect(req, "tcp://127.0.0.1:5253")
>>
>>   out_data = "blah blah hey what is %s doing here?"
>>   zstr.send(req, out_data)
>>
>>   in_data = zstr.recv(rep)
>>   assert in_data == out_data, "expected %r, got %r" % (out_data, in_data)
>>
>>   $ python zstr-bug1.py
>>   zsh: segmentation fault (core dumped)  python zstr-bug1.py
>>
>> Here are the innermost stack frames from that segfault:
>>
>> """
>> #0  0x00007f3c06a4bf90 in _IO_vfprintf_internal (s=s at entry
>> =0x7fffd8435470,
>>     format=<optimized out>,
>>     format at entry=0x7f3c07124fa4 "blah blah hey what is %s doing here?",
>>     ap=ap at entry=0x7fffd84355e8) at vfprintf.c:1655
>> #1  0x00007f3c06b0f6c0 in ___vsnprintf_chk (
>>     s=s at entry=0x1120750 "blah blah hey what is ", maxlen=<optimized out>,
>>     maxlen at entry=256, flags=flags at entry=1, slen=slen at entry=256,
>>     format=format at entry=0x7f3c07124fa4 "blah blah hey what is %s doing
>> here?",
>>     args=args at entry=0x7fffd84355e8) at vsnprintf_chk.c:63
>> #2  0x00007f3c04af0d1b in vsnprintf (__ap=0x7fffd84355e8,
>>     __fmt=0x7f3c07124fa4 "blah blah hey what is %s doing here?", __n=256,
>>     __s=0x1120750 "blah blah hey what is ")
>>     at /usr/include/x86_64-linux-gnu/bits/stdio2.h:77
>> #3  zsys_vprintf (
>>     format=0x7f3c07124fa4 "blah blah hey what is %s doing here?",
>>     argptr=argptr at entry=0x7fffd8435638) at zsys.c:386
>> #4  0x00007f3c04af0499 in zstr_send (zocket=0x11c2d30, format=<optimized
>> out>)
>>     at zstr.c:112
>> #5  0x00007f3c05521adc in ffi_call_unix64 ()
>>    from /usr/lib/x86_64-linux-gnu/libffi.so.6
>> """
>>
>> 2. zstr_send() is not binary safe
>> ---------------------------------
>>
>> The fact that CZMQ's zstr_send() is not binary safe is just fine: it
>> says right in the docs that it's a convenience for sending C strings
>> around. So the fact that pyczmq.zstr.send() is not binary safe is not
>> surprising (although it is annoying -- I'm used to Python strings, not
>> C strings). But since you say it's "critical to the functionality of
>> pyczmq", maybe I should worry about it. Anyways, here's an example:
>>
>>   $ cat zstr-bug2.py
>>   from pyczmq import zmq, zctx, zsocket, zstr
>>
>>   ctx = zctx.new()
>>   rep = zsocket.new(ctx, zmq.REP)
>>   zsocket.bind(rep, "tcp://127.0.0.1:5253")
>>
>>   req = zsocket.new(ctx, zmq.REQ)
>>   zsocket.connect(req, "tcp://127.0.0.1:5253")
>>
>>   out_data = "blah blah hey what is \0 doing here?"
>>   zstr.send(req, out_data)
>>
>>   in_data = zstr.recv(rep)
>>   assert in_data == out_data, "expected %r, got %r" % (out_data, in_data)
>>
>>   $ python zstr-bug2.py
>>   Traceback (most recent call last):
>>     File "zstr-bug2.py", line 14, in <module>
>>       assert in_data == out_data, "expected %r, got %r" % (out_data,
>> in_data)
>>   AssertionError: expected 'blah blah hey what is \x00 doing here?', got
>> 'blah blah hey what is '
>>
>> So: until I get to the point of needing secure authentication, pyczmq
>> just gets in my way and makes life harder. Darn.
>>
>>        Greg
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140117/45537749/attachment.htm>


More information about the zeromq-dev mailing list