[zeromq-dev] ZMQ_STREAM and incoming message size limit over TCP

Osiris Pedroso opedroso at gmail.com
Wed May 4 17:00:24 CEST 2016


IMO, it seems worth having this option when the OS takes advantage of it.

On Wed, May 4, 2016 at 9:25 AM Doron Somech <somdoron at gmail.com> wrote:

> But more important when implementing a stream engine you cannot rely on
> receiving the entire message, this is important, if you send block of 20K
> the receiving side OS can decide to split it into 20 1k messages.
>
> So my suggestion, don't change anything, process 8KB buffer call receive
> again and process the next.
>
> On Wed, May 4, 2016 at 5:22 PM, Doron Somech <somdoron at gmail.com> wrote:
>
>> It is hard coded in the libzmq, you can change it and recompile, change
>> in_batch_size in config.hpp.
>>
>> If you really want to you can make a socket option to change it, maybe
>> called ZMQ_STREAM_BUFFER_SIZE.
>> If you do make a new socket option, you want to give it to the
>> raw_decoder in stream_engine.cpp
>>
>> [1] https://github.com/zeromq/libzmq/blob/master/src/config.hpp#L60
>> [2]
>> https://github.com/zeromq/libzmq/blob/master/src/stream_engine.cpp#L198
>>
>> On Wed, May 4, 2016 at 4:34 PM, Zarko Bulatovic <zb10948 at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> My incoming messages to stream socket are being split in < 8kB parts,
>>> while I don't have any framing denomination on the ZeroMQ side. As I
>>> understand, this is down to OS TCP sockets implementation cuting them to
>>> some MTU size, but I don't see it. If I do tcpdump on my channel, I am
>>> seeing 20kB packets being relayed from source to destination socket (dest
>>> is ZMQ_TCP).
>>>
>>> I have tried raising the ZMQ_RCVBUF value for the socket, no luck. I
>>> have also tried getting the limit from the underlying fd, which gets
>>> retreived OK via zmq_getsockopt() with ZMQ_FD, but standard getsockopt()
>>> with SO_RCVBUF on that fd yields an "Socket operation on non-socket" error.
>>>
>>> I would appreciate any idea on how to tackle this issue. I can always
>>> implement message integrity on my side but I wanted to utilize one of many
>>> great ZMQ functionalities... :)
>>>
>>> Best Regards,
>>> ZB
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160504/e7a1d649/attachment.htm>


More information about the zeromq-dev mailing list