[zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)?

artemv zmq artemv.zmq at gmail.com
Sat Dec 28 21:52:01 CET 2013


>> So if the traffic is not relevant, there is nothing IMHO to be done on
ØMQ. Just add a TTL to a part of the multipart message, and have the
receiver look at the time and discard the messages. Have an initial >>
handshake to sync the peer timestamps just in case.

That's what I'm talking about ). Why this can't/shouldn't  be done on 0mq?
 The thing is -- 0mq is queueing solution (after all) and TTL is part of
any queueing. TTL is not concrete business feature, it's very common
and ubiquitous
thing.

Agree?


2013/12/28 Bruno D. Rodrigues <bruno.rodrigues at litux.org>

> I recall apache activemq (jms) had a ttl system that worked like this
> (back in 2008, no idea how it works now):
>
> - messages gets queued individually
> - when a consumer connects, messages gets popped from the queue, and if
> the ttl elapsed, they’d get discarded instead of piped into the consumer.
>
> this means that when no consumer was consuming, the TTL was irrelevant and
> messages would queue up anyway.
>
> having a thread expiring these messages in parallel is quite hard to
> accomplish without some locks in the middle, which kills the performance.
>
>
> Either way, the core problem, as far as I understood, is:
>
> - messages configured with a TTL shall be discarded if the time elapse
>
> The solution can include a second problem to solve:
> - is the amount of traffic relevant? if so, it’d better be dropped on the
> sender, if not it can safely be dropped on the receiver.
>
>
> So if the traffic is not relevant, there is nothing IMHO to be done on
> ØMQ. Just add a TTL to a part of the multipart message, and have the
> receiver look at the time and discard the messages. Have an initial
> handshake to sync the peer timestamps just in case.
>
> If the traffic is relevant and the idea is to drop messages on the sender
> side, good luck implementing it. The simple “discard when piping out”
> should be easy to do, albeit I think ØMQ may already have a tcp buffer
> ready to send and it could be difficult to individualize the messages. The
> harder “discard always”, as I said, it’s quite hard to do without locks and
> killing performance.
>
>
>
> On Dec 28, 2013, at 19:12, Brian Knox <briank at talksum.com> wrote:
>
> I’m a little confused by the wording of your explanation.  You say the
> “receiving peer” should pay attention to the “use-by date” attribute.  If
> the peer is going to receive the message and check this attribute, I’m
> uncertain what a TTL that is checked by the actual zeromq queue is for in
> this use case.
>
> *From:* zeromq-dev-bounces at lists.zeromq.org [
> mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>
> ] *On Behalf Of *artemv zmq
> *Sent:* Saturday, December 28, 2013 1:34 PM
> *To:* ZeroMQ development list
> *Subject:* Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for
> in-mem queue)?
>
> > Perhaps. What problem would you be solving with TTLs, which is an
> > issue today? ("Lack of feature X" is not a valid problem statement for
> > feature X).
>
> There's a need to send certain type of messages which have sort of "use-by
> date" attribute.
> I.e. client may want to define this attribute and receiving peer should
> pay attention to it.  What it would give?  It would give a control over
> queued messages. Say, if I sending messages which
> are "valid" _only_ when peers connected  and  "not valid" when peers
> aren't connected.   Here "valid/not valid" is defined via "use-by date"
> message attribute. Receiving peer may check "use-by date" and recognize if
> gotten message was too long inside somebody's queue, and take some actions:
> log, throw error, silently discard a message, or even collect a message
> inside  so called "dead letter" queue .
>
> Btw, essentially, today 0mq defines "use-by date" as infinite.  And
> proposition is to make this thing configurable.
>
>
>
>
> 2013/12/28 Pieter Hintjens <ph at imatix.com>
>
> On Fri, Dec 27, 2013 at 1:26 PM, artemv zmq <artemv.zmq at gmail.com> wrote:
>
> > And all queueing solutions do have TTL .
> Perhaps. What problem would you be solving with TTLs, which is an
> issue today? ("Lack of feature X" is not a valid problem statement for
> feature X).
>
> -Pieter
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131228/543fbf8c/attachment.htm>


More information about the zeromq-dev mailing list