[zeromq-dev] ZMQ allocating all available memory when sending

Pieter Hintjens ph at imatix.com
Sat Nov 19 18:56:16 CET 2011


Clemens,

Set the HWM before doing the connect. Note this line from the
zmq_setsockopt man page:

"Caution: All options, with the exception of ZMQ_SUBSCRIBE,
ZMQ_UNSUBSCRIBE and ZMQ_LINGER, only take effect for subsequent socket
bind/connects."

-Pieter

On Sat, Nov 19, 2011 at 10:45 AM, Clemens Lutz <lutzc at student.ethz.ch> wrote:
> Hi Pieter,
>
> Thanks for the response. Attached is a test case where setting ZMQ_HWM doesn't
> work. Setting ZMQ_HWM to 1024 with 1 KiB messages should limit the queue's
> size to ~1 MiB, right? "top" still reports around one GiB of memory usage when
> sending that amout of data.
> This is with ZMQ v.2.1.10.
>
> Cheers,
> Clemens
>
>
> On Friday 18 November 2011 08.14:49 Pieter Hintjens wrote:
>> On Fri, Nov 18, 2011 at 3:59 AM, Clemens Lutz <lutzc at student.ethz.ch> wrote:
>> > I've been writing a benchmark to verify ZeroMQ's performance is near
>> > line-rate on the machines I'm using. However, I keep running into the
>> > issue of ZMQ allocating memory on the sender's side equal to the amount
>> > I'm sending. If I want to send more data than I have physical memory
>> > the program will exit with an "out of memory" error.
>>
>> zmq_send is asynchronous, so does not send right away but queues the
>> message in memory. Actual sending is limited by the network capacity.
>> So messages will build up. Setting ZMQ_HWM on the sender socket should
>> limit the number of messages. If you have a test case where setting
>> ZMQ_HWM doesn't work, please post it.
>>
>> -Pieter
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>



More information about the zeromq-dev mailing list