[zeromq-dev] How to get reliability in pub-sub pattern

[Michel Pelletier]

Also check out:

http://zguide.zeromq.org/page:all#A-Shared-Key-Value-Cache-Clone-Pattern

-Michel

On Mon, Jul 30, 2012 at 12:19 PM, Chuck Remes <lists at chuckremes.com> wrote:
> On Jul 30, 2012, at 2:05 PM, diffuser78 wrote:
>
>> Hi,
>>
>> From ZMQ guide I see that pub-sub pattern does not offer reliability, i.e. publisher doesn't know if the subscriber has been consuming messages.
>>
>> What other pattern would you recommend in pub-sub scenario where a publisher must know the details of subscribers and how many messages subscribers have consumed ?
>
> I should write a wiki article about an idea/opinion that I have. I brought it up at the Chicago meetup from a few months ago.
>
> The only socket types that we should encourage people to use are DEALER/ROUTER (and there was some discussion that this could be reduced to just ROUTER sockets). With DEALER/ROUTER, it is (hopefully) obvious how to do reliable communications between any number of peers. The patterns in the guide illustrate this nicely and in most (all?) cases use the DEALER/ROUTER socket types.
>
> For pub/sub, if you need reliability then you absolutely need bidirectional communications at your application level. The PUB/SUB sockets in the library do not offer this, so now you are back to DEALER/ROUTER.
>
> You could mimic reliable pub/sub by using two sets of sockets where the subscribers on the first set would publish the sequence numbers that have been received (or a range). Much like a sliding-window protocol, you could cheaply "ack" lots of messages. But now you need to build in additional intelligence to all subscribers so that they know to ignore duplicate sequence numbers. Plus, you have the complexity of creating a truly unique sequence number in a distributed system; that generator is oftentimes a bottle neck.
>
> Good luck. Let us know what you figure out.
>
> cr
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev