[zeromq-dev] Question about zmq_recv len and truncate

KIU Shueng Chuan nixchuan at gmail.com
Sat Oct 12 02:07:16 CEST 2013


zmq_recv() doesn't really receive directly into your buffer, it just does
the copying from a zmq_msg_t to your buffer.
It was probably created to mimic the BSD sockets API.

You could prevent out-of-memory with socket option of ZMQ_MAXMSGSIZE.


On Sat, Oct 12, 2013 at 4:45 AM, Bruno D. Rodrigues <
bruno.rodrigues at litux.org> wrote:

> Hello all,
>
> I'm curious about a comment on a certain behaviour of zmq_recv that
> affected me when using it over the Java bindings.
>
> The sentence in question is the following:
> http://api.zeromq.org/3-2:zmq-recv
> "The zmq_recv() function shall return number of bytes in the message if
> successful. Note that the value can exceed the value of the len parameter
> in case the message was truncated."
>
> and on the code, zmq.cpp 493
>     //  At the moment an oversized message is silently truncated.
>
> The context I was trying was a PUB server where I inject random messages,
> which I can't control (JSONs with some hundred bytes to several hundred
> kilobytes), and on the other side a SUB consuming them. This was over the
> Java bindings, using zmq git master (or 4.0.1) and jzmq 3 git master (or
> 3.0.1).
>
> If I use the simpler memcpy heavy versions (send(byte[]) and byte[] x =
> socket.recv()), everything works as expected, but I was trying the other
> alternatives that allows to use direct ByteBuffers and avoid the memcpy
> operations.
> Or explicitly, the recv(byte[] buffer, int len) and the recv(no
> copy)(ByteBuffer but, int len)
>
> What happens is that unless I pre-create a buffer of the maximum expected
> size (like 16MB), messages bigger than my buffer size will not only be
> truncated, but will also break on java when the callback tries to set the
> ByteBuffer position to a value larger than the limit.
> jzmq Socket.cpp
> 703     setByteBufferPosition(env, buffer, rc);
> or
> 728         env->CallVoidMethod(buffer, setPositionHandle, pos + rc);
>
> On top of this, as my C/C++ is quite forgotten since I was forced to move
> to Java, I couldn't follow exactly where the zmq_msg_t content is filled
> up, but a grep by malloc scared me as it seems possible to telnet into the
> server, send the right header prefix, and then announce an incoming message
> of several GB, which is blindly followed by the malloc, to then be
> discarded by the truncate reported above. (this can be mitigated by setting
> a maxMessageSize somewhere depending on the bindings, it seems, but hadn't
> tested it yet)
>
> In simpler words:
> - is it possible to receive a message of unknown size using the methods
> that receive my own buffers? If not, why do those methods exist at all?
> - is it possible to crash a server with out of memory by sending a couple
> message headers with a huge len value?
>
> I'd like to help fixing whatever I can do, but I'm lost now on where to
> pick stuff up, architecturally speaking. could the recv(buffer) return the
> next part of the same message, as I expected?
>
> Thanks in advance
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131012/3e21b7cf/attachment.htm>


More information about the zeromq-dev mailing list