[zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)?
Laurent Alebarde
l.alebarde at free.fr
Sat Dec 28 22:54:34 CET 2013
An implementation could not lose performance if no TTL is set. And if
some is, the loss of performance may be acceptable. Anyhow, I think a
lasy check through the output queue can be performed in order to have a
very small impact on performance. I mean you can define a memory size
for the output queue and don't check for TTL until it is full.
Le 28/12/2013 21:52, artemv zmq a écrit :
> >> So if the traffic is not relevant, there is nothing IMHO to be done
> on ØMQ. Just add a TTL to a part of the multipart message, and have
> the receiver look at the time and discard the messages. Have an
> initial >> handshake to sync the peer timestamps just in case.
>
> That's what I'm talking about ). Why this can't/shouldn't be done on
> 0mq? The thing is -- 0mq is queueing solution (after all) and TTL is
> part of any queueing. TTL is not concrete business feature, it's very
> common and ubiquitous thing.
>
> Agree?
>
>
> 2013/12/28 Bruno D. Rodrigues <bruno.rodrigues at litux.org
> <mailto:bruno.rodrigues at litux.org>>
>
> I recall apache activemq (jms) had a ttl system that worked like
> this (back in 2008, no idea how it works now):
>
> - messages gets queued individually
> - when a consumer connects, messages gets popped from the queue,
> and if the ttl elapsed, they'd get discarded instead of piped into
> the consumer.
>
> this means that when no consumer was consuming, the TTL was
> irrelevant and messages would queue up anyway.
>
> having a thread expiring these messages in parallel is quite hard
> to accomplish without some locks in the middle, which kills the
> performance.
>
>
> Either way, the core problem, as far as I understood, is:
>
> - messages configured with a TTL shall be discarded if the time elapse
>
> The solution can include a second problem to solve:
> - is the amount of traffic relevant? if so, it'd better be dropped
> on the sender, if not it can safely be dropped on the receiver.
>
>
> So if the traffic is not relevant, there is nothing IMHO to be
> done on ØMQ. Just add a TTL to a part of the multipart message,
> and have the receiver look at the time and discard the messages.
> Have an initial handshake to sync the peer timestamps just in case.
>
> If the traffic is relevant and the idea is to drop messages on the
> sender side, good luck implementing it. The simple "discard when
> piping out" should be easy to do, albeit I think ØMQ may already
> have a tcp buffer ready to send and it could be difficult to
> individualize the messages. The harder "discard always", as I
> said, it's quite hard to do without locks and killing performance.
>
>
> On Dec 28, 2013, at 19:12, Brian Knox <briank at talksum.com
> <mailto:briank at talksum.com>> wrote:
>
>> I'm a little confused by the wording of your explanation. You
>> say the "receiving peer" should pay attention to the "use-by
>> date" attribute. If the peer is going to receive the message and
>> check this attribute, I'm uncertain what a TTL that is checked by
>> the actual zeromq queue is for in this use case.
>> *From:*zeromq-dev-bounces at lists.zeromq.org
>> <mailto:zeromq-dev-bounces at lists.zeromq.org>[mailto:zeromq-dev-bounces at lists.zeromq.org]*On
>> Behalf Of*artemv zmq
>> *Sent:*Saturday, December 28, 2013 1:34 PM
>> *To:*ZeroMQ development list
>> *Subject:*Re: [zeromq-dev] Why 0mq doesn't define TTL for message
>> (for in-mem queue)?
>> > Perhaps. What problem would you be solving with TTLs, which is an
>> > issue today? ("Lack of feature X" is not a valid problem
>> statement for
>> > feature X).
>> There's a need to send certain type of messages which have sort
>> of "use-by date" attribute.
>> I.e. client may want to define this attribute and receiving peer
>> should pay attention to it. What it would give? It would give a
>> control over queued messages. Say, if I sending messages which
>> are "valid" _only_ when peers connected and "not valid" when
>> peers aren't connected. Here "valid/not valid" is defined
>> via "use-by date" message attribute. Receiving peer may
>> check "use-by date" and recognize if gotten message was too long
>> inside somebody's queue, and take some actions: log, throw error,
>> silently discard a message, or even collect a message inside so
>> called "dead letter" queue .
>> Btw, essentially, today 0mq defines "use-by date" as infinite.
>> And proposition is to make this thing configurable.
>>
>> 2013/12/28 Pieter Hintjens <ph at imatix.com <mailto:ph at imatix.com>>
>>
>> On Fri, Dec 27, 2013 at 1:26 PM, artemv zmq
>> <artemv.zmq at gmail.com <mailto:artemv.zmq at gmail.com>> wrote:
>>
>> > And all queueing solutions do have TTL .
>>
>> Perhaps. What problem would you be solving with TTLs, which is an
>> issue today? ("Lack of feature X" is not a valid problem
>> statement for
>> feature X).
>>
>> -Pieter
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131228/e944a87c/attachment.htm>
More information about the zeromq-dev
mailing list