[zeromq-dev] ØMQ 2.1.0 beta release archive

[Martin Sustrik]

Hi,

 >             I think that ZMQ_NOBLOCK is not working properly
 >
 >             I never receive a return code -1 , I always receive 0 but I
 >         see on
 >             LAN that is not used
 >
 >             To try this , I set ZMQ_HWM = 10 and send data to 15 
Mb/sec with
 >             send NOBLOCK.
 >
 >             In this configuration I must receive return code -1, but
 >         this does
 >             not happen
 >
 >
 >     In PUB/SUB model the messages that cannot be pushed to the buffer
 >     are silently dropped. (There's a good reason for that: If the PUB
 >     socket would block instead, a single slow consumer could grind the
 >     whole distribution tree to a halt.) Just use larger, or possibly
 >     infinite (0) HWM.

 > So in PUB/SUB in only makes sense when you have one subscriber with a
 > back channel to implement back pressure.  0MQ however supports multiple
 > subscribers over different transports and as such cannot efficieintly
 > implement an aggregate back pressure.

 >     Thus, the only option is to send the message to the accessible
 >     destination and don't send it to the non-accessible destination
 >     (i.e. drop it).

> That doesn't work with high speed feeds constantly publishing.  The only
> workaround that springs to mind is to allow access to the queue length
> for the connected transports, like a ioctlsocket (FIONREAD) or ioctl
> (SIOCOUTQ).  Then the application can perform, at least, coarse rate
> limiting on the queue depth.

That's just shifting the problem to the application, not solving it. 
What will the application do when 0MQ signals that it can't accept more 
messages? Hm. Store them on disk? But if so, why not simply set 
ZMQ_SWAP. Etc.

The real problem here is that there's more data to process than the 
hardware can accommodate. There's no way to solve that on either 0MQ or 
application level. "Buy more/better hardware" seems to be the only 
reasonable advice.

I'm cc'ing the mailing list btw so that others can join the discussion.

Martin