[zeromq-dev] exiting without missing messages

Chuck Remes cremes.devlist at mac.com
Sat Aug 20 14:40:19 CEST 2011


On Aug 20, 2011, at 3:44 AM, Mathijs Kwik wrote:

> Hi all,
> 
> I would like to be able to scale a part of my application by just
> adding processes/boxes.
> If load is lower again, I would like to remove them again.
> 
> This part of the application uses a PULL socket to get work.
> And just like in the guide/example some kind of ventilator is
> PUSH'ing.
> 
> Now, adding processes is easy, just zmq_connect them and the load
> balancing push socket will do its magic trick automatically.
> 
> However, I couldn't find a way to disconnect again.
> Looking at the manpage for zmq_close, it tells me that it is going to
> drop messages that have been received from the network, but not yet
> handed out to the app itself.
> 
> So zmq_close isn't gonna work for me.
> Even if there was a way to make sure nothing is in the "incoming"
> buffer, I understand that the push socket might already have queued
> messages for a specific pull socket, so if that disappears, messages
> get stuck waiting on the push side.
> So it would be nice if there's a way to tell the push socket to stop
> sending / disconnect a certain puller.

You need another pair of sockets to carry this communication. You can't accomplish your goal with only a PUSH/PULL setup.


> Is there a graceful way to just take out a few workers without
> dropping messages?
> Or do I need to build all kinds of checks around it where the
> ventilator and the sink communicate a lot to detect missing messages
> and use in-app buffers in the ventilator so it can resend missing
> ones?
> I hope this isn't needed, since it will cause latency. This isn't an
> error/exception situation either, just a "planned" change in the
> network.

Take a look at the more advanced patterns in the 0mq guide. You probably want some kind of reliable request/reply so look at the "pirate" patterns in chapter 4.

cr





More information about the zeromq-dev mailing list