[zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)?

Sun Dec 29 00:19:56 CET 2013

Here here. I start my internal presentations with "zeromq. Forget the MQ.
It's sockets on steroids. And for once this one works" :P

--
Bruno Rodrigues
Sent from my iPhone

No dia 28/12/2013, às 22:35, Brian Knox <briank at talksum.com> escreveu:

  I disagree with the assertion that “ZeroMQ is a queuing solution”, and I
don’t believe that ZeroMQ should try to be one.  “ZeroMQ has and uses
queues” != “ZeroMQ is a queuing solution”, in my opinion.

*From:* zeromq-dev-bounces at lists.zeromq.org [
mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>]
*On Behalf Of *artemv zmq
*Sent:* Saturday, December 28, 2013 3:52 PM
*To:* ZeroMQ development list
*Subject:* Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for
in-mem queue)?

>> So if the traffic is not relevant, there is nothing IMHO to be done on
ØMQ. Just add a TTL to a part of the multipart message, and have the
receiver look at the time and discard the messages. Have an initial >>
handshake to sync the peer timestamps just in case.

That's what I'm talking about ). Why this can't/shouldn't  be done on 0mq?
 The thing is -- 0mq is queueing solution (after all) and TTL is part of
any queueing. TTL is not concrete business feature, it's very common
and ubiquitous
thing.

Agree?

2013/12/28 Bruno D. Rodrigues <bruno.rodrigues at litux.org>

 I recall apache activemq (jms) had a ttl system that worked like this
(back in 2008, no idea how it works now):

- messages gets queued individually

- when a consumer connects, messages gets popped from the queue, and if the
ttl elapsed, they’d get discarded instead of piped into the consumer.

this means that when no consumer was consuming, the TTL was irrelevant and
messages would queue up anyway.

having a thread expiring these messages in parallel is quite hard to
accomplish without some locks in the middle, which kills the performance.

Either way, the core problem, as far as I understood, is:

- messages configured with a TTL shall be discarded if the time elapse

The solution can include a second problem to solve:

- is the amount of traffic relevant? if so, it’d better be dropped on the
sender, if not it can safely be dropped on the receiver.

So if the traffic is not relevant, there is nothing IMHO to be done on ØMQ.
Just add a TTL to a part of the multipart message, and have the receiver
look at the time and discard the messages. Have an initial handshake to
sync the peer timestamps just in case.

If the traffic is relevant and the idea is to drop messages on the sender
side, good luck implementing it. The simple “discard when piping out”
should be easy to do, albeit I think ØMQ may already have a tcp buffer
ready to send and it could be difficult to individualize the messages. The
harder “discard always”, as I said, it’s quite hard to do without locks and
killing performance.

On Dec 28, 2013, at 19:12, Brian Knox <briank at talksum.com> wrote:

   I’m a little confused by the wording of your explanation.  You say the
“receiving peer” should pay attention to the “use-by date” attribute.  If
the peer is going to receive the message and check this attribute, I’m
uncertain what a TTL that is checked by the actual zeromq queue is for in
this use case.

*From:* zeromq-dev-bounces at lists.zeromq.org [
mailto:zeromq-dev-bounces at lists.zeromq.org<zeromq-dev-bounces at lists.zeromq.org>
] *On Behalf Of *artemv zmq
*Sent:* Saturday, December 28, 2013 1:34 PM
*To:* ZeroMQ development list
*Subject:* Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for
in-mem queue)?

> Perhaps. What problem would you be solving with TTLs, which is an
> issue today? ("Lack of feature X" is not a valid problem statement for
> feature X).

There's a need to send certain type of messages which have sort of "use-by
date" attribute.

I.e. client may want to define this attribute and receiving peer should pay
attention to it.  What it would give?  It would give a control over queued
messages. Say, if I sending messages which

are "valid" _only_ when peers connected  and  "not valid" when peers aren't
connected.   Here "valid/not valid" is defined via "use-by date" message
attribute. Receiving peer may check "use-by date" and recognize if gotten
message was too long inside somebody's queue, and take some actions: log,
throw error, silently discard a message, or even collect a message inside
 so called "dead letter" queue .

Btw, essentially, today 0mq defines "use-by date" as infinite.  And
proposition is to make this thing configurable.

2013/12/28 Pieter Hintjens <ph at imatix.com>

 On Fri, Dec 27, 2013 at 1:26 PM, artemv zmq <artemv.zmq at gmail.com> wrote:

> And all queueing solutions do have TTL .

Perhaps. What problem would you be solving with TTLs, which is an
issue today? ("Lack of feature X" is not a valid problem statement for
feature X).

-Pieter

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131228/a3199b9f/attachment.htm>