[zeromq-dev] ZMQ_STREAM and incoming message size limit over TCP

Doron Somech somdoron at gmail.com
Wed May 4 16:25:26 CEST 2016


But more important when implementing a stream engine you cannot rely on
receiving the entire message, this is important, if you send block of 20K
the receiving side OS can decide to split it into 20 1k messages.

So my suggestion, don't change anything, process 8KB buffer call receive
again and process the next.

On Wed, May 4, 2016 at 5:22 PM, Doron Somech <somdoron at gmail.com> wrote:

> It is hard coded in the libzmq, you can change it and recompile, change
> in_batch_size in config.hpp.
>
> If you really want to you can make a socket option to change it, maybe
> called ZMQ_STREAM_BUFFER_SIZE.
> If you do make a new socket option, you want to give it to the raw_decoder
> in stream_engine.cpp
>
> [1] https://github.com/zeromq/libzmq/blob/master/src/config.hpp#L60
> [2]
> https://github.com/zeromq/libzmq/blob/master/src/stream_engine.cpp#L198
>
> On Wed, May 4, 2016 at 4:34 PM, Zarko Bulatovic <zb10948 at gmail.com> wrote:
>
>> Hi,
>>
>> My incoming messages to stream socket are being split in < 8kB parts,
>> while I don't have any framing denomination on the ZeroMQ side. As I
>> understand, this is down to OS TCP sockets implementation cuting them to
>> some MTU size, but I don't see it. If I do tcpdump on my channel, I am
>> seeing 20kB packets being relayed from source to destination socket (dest
>> is ZMQ_TCP).
>>
>> I have tried raising the ZMQ_RCVBUF value for the socket, no luck. I have
>> also tried getting the limit from the underlying fd, which gets retreived
>> OK via zmq_getsockopt() with ZMQ_FD, but standard getsockopt() with
>> SO_RCVBUF on that fd yields an "Socket operation on non-socket" error.
>>
>> I would appreciate any idea on how to tackle this issue. I can always
>> implement message integrity on my side but I wanted to utilize one of many
>> great ZMQ functionalities... :)
>>
>> Best Regards,
>> ZB
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160504/db1bd199/attachment.htm>


More information about the zeromq-dev mailing list