[zeromq-dev] ZMQ_FLAT_PULL feature discussion
Daniel Krikun
krikun.daniel at gmail.com
Tue Aug 13 19:23:31 CEST 2013
For the sender, the situation is transparent, however, it so happens that
for ZMQ_PUB if hwm is reached it indeed drops the message so it should work
for you, right?
On Tue, Aug 13, 2013 at 6:59 PM, Michael Haberler <mail17 at mah.priv.at>wrote:
>
> Am 13.08.2013 um 17:29 schrieb Daniel Krikun <krikun.daniel at gmail.com>:
>
> > I have almost finished the feature, in the mean time, it evolved as
> follows:
> > -- it will be available for ZMQ_PULL, ZMQ_SUB and ZMQ_DEALER sockets
> > -- it is activated using ZMQ_CONFLATE socket option at the receiver side
> > -- currently, multi-part messages would not be supported
> > -- currently, would not be tested for (e)pgm
> > -- receiver hwm is not considered
> >
> > Does that makes sense to you?
>
> not quite yet, but happy to read through commits
>
> as I said, my key requirement is to have a publish always succeed even if
> it entails dropping old messages on the outbound queue; I dont fully
> understand if this is covered by your patch or not?
>
> - Michael
>
> >
> > I'm currently working on performance issues, hope to submit pull request
> soon :)
> >
> >
> > On Tue, Aug 13, 2013 at 2:40 PM, Brian Knox <briank at talksum.com> wrote:
> > I've been using ZeroMQ to distribute frames from video streams for
> processing and for some of my use cases this feature would be of great
> interest. I'll happily be a test victim of a patch.
> >
> > Brian
> >
> >
> > On Tue, Aug 13, 2013 at 4:27 AM, Michael Haberler <mail17 at mah.priv.at>
> wrote:
> > Daniel,
> >
> > Am 07.06.2013 um 11:53 schrieb Daniel Krikun <krikun.daniel at gmail.com>:
> >
> > > Hi all,
> > >
> > > I have a setup, where a server does graphics rendering based on client
> > > requests, that is, clients send geometric data (position, orientation,
> > > etc.) and the server runs in cycles: process incoming messages and do
> > > some rendering.
> > > Occasionally, the clients might be faster and few messages get
> > > themselves queued on the server queue. However, only the most recent
> > > message is of interest as the server does not renders the objects as
> > > they were a couple of cycles before.
> > >
> > > To solve the problem, I have extended the ZMQ_ROUTER socket (via
> > > subclassing) so that it has a background thread that empties the
> > > socket message queue and stores the last message. The message is
> > > stored in a double-buffer (one for each client, this is why I need a
> > > router socket type). Then, when the server calls recv() on my extended
> > > socket, it gets one of the stored messages (if there are any).
> > >
> > > I ask myself whether some similar functionality should be put inside
> > > zeromq (say, under ZMQ_FLAT_PULL socket type), left as is in a "user
> > > space" or maybe I should use UDP instead?
> >
> >
> > having arrived at the point where I could make good use of the feature
> you proposed like so:
> >
> > - my scenario happens in a PUB socket (not ROUTER); subscribers present
> but not necessarily constantly pulling messages
> > - semantic summary is: 'the last message always wins' as it subsumes all
> the previous updates
> >
> > the way I read the documentation on queue full/ high water mark behavior
> it looks like right now later queue adds are dropped and the old ones
> retained which is exactly the opposite what I'm looking for
> >
> > I would think that kind of behavior could be implemented with a new
> socket option which twists the meaning of high water mark reached (let's
> call it ZMQ_SNDDROPOLD or so for now):
> >
> > assuming this option is set and a queue add fails due to HWM reached,
> the sender would atomically pop entries from the queue head until the queue
> add succeeds
> >
> > doesnt this match what you are proposing for ROUTER too? I dont see
> where a thread is needed here, but I'm not terribly read into ZMQ internals
> >
> > curious about your progress!
> >
> > - Michael
> >
> > >
> > >
> > > Thank you,
> > >
> > > --
> > > Daniel Krikun
> > > _______________________________________________
> > > zeromq-dev mailing list
> > > zeromq-dev at lists.zeromq.org
> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> >
> > --
> > Daniel Krikun
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
--
Daniel Krikun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130813/62e38d87/attachment.htm>
More information about the zeromq-dev
mailing list