[zeromq-dev] RFC: finer control of socket type / behaviour

Goswin von Brederlow goswin-v-b at web.de
Tue Jun 3 09:52:48 CEST 2014


On Mon, Jun 02, 2014 at 01:11:21PM +0200, Pieter Hintjens wrote:
> Returning EAGAIN on a full pipe might be a good improvement, though
> it's unclear how an app could use this. Blocking seems problematic as
> it exposes the app to failure when a single peer stops reading its
> messages.

An app that does multiple things and needs to remain responsive would
want EAGAIN. For example we want to monitor things every host in a
cluster collecting data every minute. Now say a switch dies as they
ocasionally do and the collected data can't be send. The app needs to
keep monitoring so blocking is not an option. Droping is also bad
since that would leave gaps in the monitoring. Instead if send
returned EAGAIN the app could store the data locally for later
submission. Or it could thin out and compress the data. Instead of
keeping every dataset for every minute in memory only keep every
second minute, every 5 minutes, every hour. The amount of memory can
thus be limited without leaving a total blackout in the data.

The same example could also work with blocking mode and send timeout.
In an app with high traffic flow occasionaly hitting the HWM could be
normal and resolve itself after a second or two. Then blocking with a
send timeout would be the better option. Basically a delayed EAGAIN so
the app doesn't have to do anything if it outpaces the available
bandwith for s second.

> I agree that dropping messages is rather brutal in this case. However
> it's also the most robust policy. You should perhaps not be sending
> unlimited messages to a peer. There's suggestions in the Guide for
> credit-based flow control that rates how much gets sent to any single
> peer.
> 
> -Pieter
> 
> On Mon, Jun 2, 2014 at 11:19 AM, Goswin von Brederlow <goswin-v-b at web.de> wrote:
> > On Wed, May 28, 2014 at 10:07:38AM +0200, Pieter Hintjens wrote:
> >> This has been mooted before and I think it's a good idea in some ways.
> >> Certainly to allow experimentation. However the current patterns do
> >> kind of cover the sane use cases. It's hard to see what the point
> >> would be, for instance, of blocking in a ROUTER socket when you can't
> >> send a message. There's no real sense to that.
> >>
> >> If you want to experiment with this, you can build custom socket types
> >> on top of ZMQ_ROUTER and virtualize them, e.g. like the CZMQ zactor
> >> model. Then if you get patterns that work well you've got arguments
> >> for pushing this into libzmq.
> >>
> >> -Pieter
> >
> > Except I can't change the "Action in mute state" frop "Drop" to
> > "Block" or "EAGAIN". That means that messages get silently lost when
> > one sends to fast or there is a temporary hickup in the network
> > connection. E.g. when I reset a switch in the network I want messages
> > to block till the switch is back.
> >
> > MfG
> >         Goswin

MfG
	Goswin



More information about the zeromq-dev mailing list