[zeromq-dev] RFC: finer control of socket type / behaviour

Goswin von Brederlow goswin-v-b at web.de
Thu Jun 5 11:10:04 CEST 2014

On Tue, Jun 03, 2014 at 12:19:53PM +0200, Pieter Hintjens wrote:
> It's a valid use case yet you're describing a one-to-one pattern, so
> perhaps PUSH/PULL or DEALER-DEALER. ROUTER is specifically for servers
> talking to many clients. There is little sense in returning an EAGAIN
> there. Servers can't treat individual clients favorably or they tend
> to crash in unpleasant ways.
> The different socket types do have different semantics, for good reasons.

Sure. I can use PUSH or DEALER. But both of them block when the HWM
hits. That would stop the monitoring. So the problem remains.

For a ROUTER example. How about transfering files to a number of
clients? You can't use (just) PUB/SUB because you need the clients to
ACK to ensure file transmission is complete. So ROUTER/DEALER seems
better than having both PUB/SUB to send files and PUSH/PULL for the
ACKs (assume you aren't doing epgm). So you send out a large file to
each client. If one of the clients stalls you would want to stop
sending it more data until it comes back.

Maybe that example isn't so convincing since due to the ACKs you have
a simple means to do some credit based flow control and avoid hitting
the HWM in the first place.

Anyway. I don't feel quite satisfied with the restriction on the
action in mutet state based on socket type and haven seen other
complain about it too. I guess I should stop bitching and start
writing a patch. Maybe it is such a simple change that it isn't worth
fighting about. After all it wouldn't add any new mute state
behaviour, just more flexibility which behaviour is used.

> On Tue, Jun 3, 2014 at 9:52 AM, Goswin von Brederlow <goswin-v-b at web.de> wrote:
> > On Mon, Jun 02, 2014 at 01:11:21PM +0200, Pieter Hintjens wrote:
> >> Returning EAGAIN on a full pipe might be a good improvement, though
> >> it's unclear how an app could use this. Blocking seems problematic as
> >> it exposes the app to failure when a single peer stops reading its
> >> messages.
> >
> > An app that does multiple things and needs to remain responsive would
> > want EAGAIN. For example we want to monitor things every host in a
> > cluster collecting data every minute. Now say a switch dies as they
> > ocasionally do and the collected data can't be send. The app needs to
> > keep monitoring so blocking is not an option. Droping is also bad
> > since that would leave gaps in the monitoring. Instead if send
> > returned EAGAIN the app could store the data locally for later
> > submission. Or it could thin out and compress the data. Instead of
> > keeping every dataset for every minute in memory only keep every
> > second minute, every 5 minutes, every hour. The amount of memory can
> > thus be limited without leaving a total blackout in the data.
> >
> > The same example could also work with blocking mode and send timeout.
> > In an app with high traffic flow occasionaly hitting the HWM could be
> > normal and resolve itself after a second or two. Then blocking with a
> > send timeout would be the better option. Basically a delayed EAGAIN so
> > the app doesn't have to do anything if it outpaces the available
> > bandwith for s second.
> >
> >> I agree that dropping messages is rather brutal in this case. However
> >> it's also the most robust policy. You should perhaps not be sending
> >> unlimited messages to a peer. There's suggestions in the Guide for
> >> credit-based flow control that rates how much gets sent to any single
> >> peer.
> >>
> >> -Pieter
> >>
> >> On Mon, Jun 2, 2014 at 11:19 AM, Goswin von Brederlow <goswin-v-b at web.de> wrote:
> >> > On Wed, May 28, 2014 at 10:07:38AM +0200, Pieter Hintjens wrote:
> >> >> This has been mooted before and I think it's a good idea in some ways.
> >> >> Certainly to allow experimentation. However the current patterns do
> >> >> kind of cover the sane use cases. It's hard to see what the point
> >> >> would be, for instance, of blocking in a ROUTER socket when you can't
> >> >> send a message. There's no real sense to that.
> >> >>
> >> >> If you want to experiment with this, you can build custom socket types
> >> >> on top of ZMQ_ROUTER and virtualize them, e.g. like the CZMQ zactor
> >> >> model. Then if you get patterns that work well you've got arguments
> >> >> for pushing this into libzmq.
> >> >>
> >> >> -Pieter
> >> >
> >> > Except I can't change the "Action in mute state" frop "Drop" to
> >> > "Block" or "EAGAIN". That means that messages get silently lost when
> >> > one sends to fast or there is a temporary hickup in the network
> >> > connection. E.g. when I reset a switch in the network I want messages
> >> > to block till the switch is back.
> >> >
> >> > MfG
> >> >         Goswin
> >
> > MfG
> >         Goswin
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

More information about the zeromq-dev mailing list