[zeromq-dev] Releasing RAM in Chunks

[Pieter Hintjens]

Anatoly,

The standard causes for memory exhaustion are, in order:

* You're forgetting to close messages, or doing it at the wrong time.
* Your consumer is unable to process messages at full speed, so the
queue builds up.

In the second case you need to decide how you want to handle the excess.

-Pieter

On Tue, Jul 3, 2012 at 3:48 AM, Anatoly <tolitius at gmail.com> wrote:
> ØMQ Crowd,
>
>     Reading messages from a socket, placing them on the zmq (PUSH). On the
> other side reading messages of off the queue (PULL) and persisting them in
> to DB.
>
>     If we get millions of messages, ØMQ takes X GB of RAM (since the pushing
> in this case is at much higher speed than pulling that waits for a DB for
> each pull), and does not release these Gigs until ALL of the messages are
> consumed (e.g. pulled).
>
>     Is there a way to configure it to release memory in chunks as the queue
> is being emptied?
>
> Thank you,
> /Anatoly
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>