[zeromq-dev] Slow joiner syndrome solution for XSUB/XPUB broker based system

[Tomer Eliyahu]

Hi,



I know this topic was probably discussed before, I couldn't find a proper
solution, so I implemented something a bit different. I'm not sure if this
solves all pitfalls, i'll be greatfull for comments.



We have a system with a XPUB-XSUB broker running as a separate process in
the system (binds frontend to ipc:///tmp/publishers  and backend to
ipc:///tmp/subscribers).



Clients of the broker have both SUB socket for receiving messages, and a
PUB socket for sending messages. When a client boots, it connects both its
PUB and SUB sockets to the broker's endpoints, and subscribes to the topic
of interest.


For the sake of simplicity, lets assume there we have only the broker, a
publisher and a subscriber processes in the system:

We make sure that the broker process starts first, then a subscriber which
connects and subscribes to the topic, and only then start the publisher.
The publisher then sends a single message and terminates.

Obviously, the message is lost due to the slow joiner syndrome - I assume
the reason for that is because the publisher process zmq_connect() call is
asynchronous, therefore the connect is not actually complete by the time we
send the message.



I thought of a possible solution for this - basically we want to
synchronize the connect operation done by the publisher. Having both PUB
and SUB socket, we can simply send a dummy message from PUB to SUB on the
same publisher process until the first message is receieved, and then it is
guarantied that the connect is done and consecutive messages (now to "real"
topics with actual subscribers) will not be lost.



The only part i'm not sure about is the subscriber side - assuming the
subscriber boots, connects and subscribes _before_ we start the publisher -
is it guarantied that no message will be lost (assuming ofcourse the
subscriber doesn't crash / unsubscribe / etc.) ?



Thanks,

Tomer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180528/c062822a/attachment.htm>