[zeromq-dev] Fundamental question on ZMQ. How to determine message fail/success without HWM-arking?

Fri Dec 6 12:19:34 CET 2013

Sorry for confusion.

When I said out-of-control -- I meant they do have ZMQ but they may have
different release cycle and QoS. It's just a service on ZMQ, on a ROUTER.

Our application is aimed to take a message, get its headers, decide on what
service ROUTER to send  and that's it. W/o waiting for reply. Essentially
we are a DEALER.
Replies are important, but as long as they coming back. If they not. Not a
problem. Client application (iPhone game)  by itself  checking replies and
correlation,
and keep watching: "ahha, I didn't receive ack for betting.  hmmm. Let's
try again".  Now it's more clear?

I really don't need PUB/SUB. I need DEALER/ROUTER. Here, in my company, the
only biggest concern so far with ZMQ -- misleading behaviour:
when .send()  returns "true"  that should mean that message "sent",
whatever that means:   left our PID, left our NIC and so on, we have to
guarantee that message is not on us.
I know what's PUB/SUB. And again, telling you that it's not suitable. The
problem statement is simple:

- don't use HWM for DEALER/ROUTER (prohibit message queueing).
- raise immediately if you can't .send()  (don't collect in internal queue)

Is it possible?

BR
-artemv

2013/12/6 Justin Cook <jhcook at gmail.com>

> Ok, this is confusing. If you are sending a message to a service that is
> out of your control, either they use 0MQ or not. I assume they do not. If
> that’s the case, it should not be a part of the use case.
>
> You say you need to know if a message has been received. But, then you say
> no ACKs or timeouts. I’m even more confused. If you are making a request to
> a foreign service over — I assume — HTTP which uses TCP, you are very well
> getting HTTP return codes with the TCP session doing all the hard work. You
> already have what you are looking for there.
>
> As far as your system — going out to mobile devices — using PUB/SUB and
> ACKing messages, this is something you will have to do in another channel
> with 0MQ. Multicast uses UDP; because, it is not feasible to send TCP ACKs
> from every single subscriber. It’s simply not scalable.
>
> You very well may need to develop your own application protocol to send
> ACKs or the publisher retransmits. I highly suggest you have a look at this:
>
>
> http://stackoverflow.com/questions/12956685/what-are-the-retransmission-rules-for-tcp
>
> It may be something you will want to mimic in your implementation. Someone
> else has already suggested a timeout for resending unacknowledged messages.
> As you can see, this is one of the ways TCP retransmissions work. You also
> may have corrupt data that fail a CRC or hash.
>
> I will finish by saying that if you do have a PUB/SUB design using another
> channel for unicast communication, you will need to be very aware of
> scalability issues. You may need to use a lockstep pattern such as REQ/REP
> if you need guarantee of communication.
>
> --
> Justin Cook
>
>
> On Friday, 6 December 2013 at 09:46, artemv zmq wrote:
>
> > Thanks for heads up.
> >
> > 2crocket:
> > No acks. No timeouts. Nothing should be kept. Messages should just
> flowing back and forth. But for every message we have to answer a question:
> "has message left NIC on sending process or not". Let me give example with
> betting: game on iPhone sending us a message "make-a-bet", then we send
> this to BettingService which isn't in our control,
> > so all we have to guarantee -- "make-a-bet" message has left our NIC and
> been "sent" to BettingService. If "make-a-bet" has been droped on a network
> - ok, if BettingService itself drops it - ok.
> >
> > Back to HWM. Let's consider that we send to unavaliable peer.
> > hwm=1. It means you can send 1 message "blindly" and .send() function
> returns success. Of course sending second time will fail. But... the trick
> is -- we need answer first time.
> > hwm=0. It means you can send any number of messages and .send() function
> _always_ returns success :(( Again, isn't this a bug?
> >
> >
> > So let me re-phrase the original question -- how to fail at .send()
> function in ZMQ?
> >
> >
> > BR
> > -artemv
> >
> >
> >
> > 2013/12/6 crocket <crockabiscuit at gmail.com (mailto:
> crockabiscuit at gmail.com)>
> > > Why don't you set a timeout for asynchronous ACKs?
> > > You receive ACKs asynchronously and keep associated messages until
> ACKs come or a timeout occurs.
> > > A timeout of 20 seconds is a reasonable estimate.
> > > After a timeout, if a message doesn't have a corresponding ACK, it is
> determined that the message wasn't delievered, and the message is sent
> again or discarded.
> > >
> > >
> > >
> > > On Fri, Dec 6, 2013 at 3:19 AM, artemv zmq <artemv.zmq at gmail.com(mailto:
> artemv.zmq at gmail.com)> wrote:
> > > > Hi,
> > > >
> > > > My name is Artem. I stay with ZMQ (on java) a year or so. Got a cool
> question for you, ppl!
> > > >
> > > > Here's my story. Recently I entered a new company (gambling games),
> after working few weeks, after getting accustomed with a code, I found that
> they are building very-unnecessarly-complex-distibuted-application ... I
> was unhappy few days, because couldn't even imagine how to support ALL THAT
> CRAP in an upcoming future. So I suggested ZMQ hoping that ZMQ will "open
> eyes" to others. But, as a feedback I got one big fundamental concern (from
> chief architects):
> > > >
> > > > - we have to know only one thing about every message: it has been
> delivered onto remote peer or not
> > > >
> > > > And few additional comments:
> > > > -we don't care if message will get lost on a network
> > > > - we don't need guarantee deliveri
> > > > - no RPC / everything is asynchronous
> > > > - we don't need HWM
> > > >
> > > >
> > > > So I'm here, because I really can't address this question: "for
> every single message how to know : whether it was delivered or not" .
> > > >
> > > > Thanks in advance. And appreciate for your help.
> > > > _______________________________________________
> > > > zeromq-dev mailing list
> > > > zeromq-dev at lists.zeromq.org (mailto:zeromq-dev at lists.zeromq.org)
> > > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > >
> > >
> > >
> > > _______________________________________________
> > > zeromq-dev mailing list
> > > zeromq-dev at lists.zeromq.org (mailto:zeromq-dev at lists.zeromq.org)
> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org (mailto:zeromq-dev at lists.zeromq.org)
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131206/b07b77c4/attachment.htm>