[zeromq-dev] ZMQ_FLAT_PULL feature discussion

[Michael Haberler]

Daniel,

Am 07.06.2013 um 11:53 schrieb Daniel Krikun <krikun.daniel at gmail.com>:

> Hi all,
> 
> I have a setup, where a server does graphics rendering based on client
> requests, that is, clients send geometric data (position, orientation,
> etc.) and the server runs in cycles: process incoming messages and do
> some rendering.
> Occasionally, the clients might be faster and few messages get
> themselves queued on the server queue. However, only the most recent
> message is of interest as the server does not renders the objects as
> they were a couple of cycles before.
> 
> To solve the problem, I have extended the ZMQ_ROUTER socket (via
> subclassing) so that it has a background thread that empties the
> socket message queue and stores the last message. The message is
> stored in a double-buffer (one for each client, this is why I need a
> router socket type). Then, when the server calls recv() on my extended
> socket, it gets one of the stored messages (if there are any).
> 
> I ask myself whether some similar functionality should be put inside
> zeromq (say, under ZMQ_FLAT_PULL socket type), left as is in a "user
> space" or maybe I should use UDP instead?


having arrived at the point where I could make good use of the feature you proposed like so:

- my scenario happens in a PUB socket (not ROUTER); subscribers present but not necessarily constantly pulling messages
- semantic summary is: 'the last message always wins' as it subsumes all the previous updates

the way I read the documentation on queue full/ high water mark behavior it looks like right now later queue adds are dropped and the old ones retained which is exactly the opposite what I'm looking for

I would think that kind of behavior could be implemented with a new socket option which twists the meaning of high water mark reached (let's call it ZMQ_SNDDROPOLD or so for now):

assuming this option is set and a queue add fails due to HWM reached, the sender would atomically pop entries from the queue head until the queue add succeeds

doesnt this match what you are proposing for ROUTER too? I dont see where a thread is needed here, but I'm not terribly read into ZMQ internals

curious about your progress!

- Michael

> 
> 
> Thank you,
> 
> --
> Daniel Krikun
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev