[zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)?

Brian Knox briank at talksum.com
Sun Dec 29 14:38:42 CET 2013


ZeroMQ's internal queues allow you to send messages asynchronously, and allow ZeroMQ to increase throughput at the cost of a little latency by batching messages.  They aren't in my opinion designed to be a persistent message queue system in the way you describe - and they would need a lot more features added to them than just TTLs in order to meet that requirement.

I still think this is a case of mistakenly assuming that because ZeroMQ has internal queues, a ZeroMQ socket is designed to be a full-fledged persistent message queuing system.

In my use of ZeroMQ, when I need an actual persistent queue, I use something designed to solve that problem in conjunction with ZeroMQ.

From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of artemv zmq
Sent: Sunday, December 29, 2013 5:13 AM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Why 0mq doesn't define TTL for message (for in-mem queue)?

hi Pieter,

> You're still not explaining what problem this solves

Ok. Here's the problem which I faced recently. Here's appl. architecture (to keep discussion focused):

iOS/Android (game ui)  <----ssl----> Tomcat <-------->  bet_service  .

There are three layers (from left to right):  game UI (0),  java webserver (1) and concrete service layer (2).
Layer#0:
  - doesn't host 0mq library.
  - talks to L1 via ssl.
  - blocking(with timeout) on every call to L1.
Layer#1:
  - does host 0mq library.
  - it's a gateway for game ui. It's an async layer between ui and world of services.
  - it's _asynchronous_. It's a sort of "delegator/router/etc" for a call from L0  to  L2.
  - on every call from L0 this layer doesn't wait for response from L2.
Layer#2:
  - does host 0mq library.
  -  a concrete business service layer.
  - L0 and L1 _don't care_ will this layer produce response or not.  If not -- L0 will hit call-timeout, L1 -- simply don't care et al.

The problem is.
When L2 goes down (whatever reason), then L1 will queue messages, and, by turn, L0 will hit call-timeout. After certain amount of time (usually up to 1hr) L2 will be restarted. And messages,
which have been queued on L1, will be flying to L2.  In my case, I don't want getting those messages on L2, because they are "out-of-date" for bet_service.

The solution is -- TTL for a message.
I can implement TTL myself, making it being as a part of message, _but_  I don't want to do that, because, TTL is low level thing, so fixing it at higher level would bring more problems, I suppose.





2013/12/29 Pieter Hintjens <ph at imatix.com<mailto:ph at imatix.com>>
On Sat, Dec 28, 2013 at 9:52 PM, artemv zmq <artemv.zmq at gmail.com<mailto:artemv.zmq at gmail.com>> wrote:
> That's what I'm talking about ). Why this can't/shouldn't  be done on 0mq?
> The thing is -- 0mq is queueing solution (after all) and TTL is part of any
> queueing. TTL is not concrete business feature, it's very common and
> ubiquitous thing.
You're still not explaining what problem this solves. Most "queuing
solutions" offer priorities, acks, persistence, two-phase commits,
etc. ZeroMQ does not.

If you want to contribute to ZeroMQ, I'd advise you to use ZeroMQ to
do do real work, then find areas where you need to make improvements,
then make those improvements carefully and minimally.

-Pieter
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131229/8ee54aa2/attachment.htm>


More information about the zeromq-dev mailing list