[zeromq-dev] how to reliably wait for SUBscribers to connect before sending messages

[Francesco]

Hi Mykola,

>This approach may require quite some design changes, which is not always
welcome. If you need this only for testing purposes, you could try to go
with a socket monitor instead.
Yeah, indeed that's the case: I would need this change just for unit
testing purposes... which seems to be too much invasive to me.
I forgot to say: I already attempted using socket monitor:

> There are ZMQ_EVENT_ACCEPTED and ZMQ_EVENT_HANDSHAKE_SUCCEED events.
Listening for them on PUB socket will give you an idea when client
connects. But I believe there still will be some time window before client
subscription is handled, so you may still need some delay before starting
to send messages on PUB.
You're right: even if I receive on the socket monitor the _ACCEPTED event,
and thus I can consider the PUB connected with the SUB, I still need to
sleep some time after that or otherwise the SUB will loose messages anyway.
That's really bothering.
Question for zmq developers: would be possible to add a new event to the
socket monitor interface that tells you when the SUB side has _really_
connected and is for sure ready to receive messages?

Thanks!!
Francesco







2018-01-05 11:55 GMT+01:00 Mykola Ostrovskyy via zeromq-dev <
zeromq-dev at lists.zeromq.org>:

> Francesco,
>
> This is kind of a classic issue with PUB-SUB. You will have quite some
> results by googling for "zeromq reliable pub-sub".
>
> AFAIK, the proper solution is to have an additional communication channel
> with a stronger message delivery guarantee (e.g. REQ-REP). The PUB server
> can broadcast some dummy messages and wait on REP socket for replies. As
> soon as client connects with SUB, subscribes, and start to receive the
> dummy messages, it sends over REQ socket a request for the actual data.
>
> This approach may require quite some design changes, which is not always
> welcome. If you need this only for testing purposes, you could try to go
> with a socket monitor instead. There are ZMQ_EVENT_ACCEPTED and
> ZMQ_EVENT_HANDSHAKE_SUCCEED events. Listening for them on PUB socket will
> give you an idea when client connects. But I believe there still will be
> some time window before client subscription is handled, so you may still
> need some delay before starting to send messages on PUB.
>
>
> Regards,
> Mykola
>
>
> 2018-01-05 11:03 GMT+02:00 Francesco <francesco.montorsi at gmail.com>:
>
>> Hi all,
>> I have some unit tests around the ZMQ wrappers I wrote for PUB/SUB
>> sockets (these wrappers integrate functionalities specific to my use case).
>>
>> In these unit tests I spawn a thread that creates a PUB socket and
>> another one creating a SUB socket. In the PUB thread I do zmq_bind(), then
>> sleep for a SETTLE_TIME and then I start sending messages with
>> zmq_msg_send().
>> In the SUB I just do zmq_connect() and then immediately zmq_msg_recv().
>>
>> The problem is when I run such unit tests under valgrind. In that case I
>> noticed that randomly my settle time of 1sec is not enough: the unit tests
>> fail because the SUB receives 0 messages instead of N>0.
>> A simple fix would be to increase the settle time. However since I repeat
>> that kind of tests hundreds of times, that means increasing testing time a
>> lot.
>>
>> I think my problem is that I need to wait after the zmq_bind() and before
>> zmq_msg_send() some time to allow ZMQ background threads to actually
>> connect the PUB and SUB sockets together.
>> Is there a better way to test if there are pending connections in ZMQ
>> background threads rather than waiting a (random) amount of time?
>>
>> Thanks,
>> Francesco
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180105/a3200d2b/attachment.htm>