[zeromq-dev] ZMQ reconnect/ephemeral ports

Sat Sep 2 18:02:04 CEST 2017

Thanks again, Luca!

For now, I’m going to go with disabling reconnect on the “data” sockets — that seems to be the best solution for my use case (connecting to endpoints that were returned by the peer binding to an unspecified (“wildcard”) port — e.g., "tcp://<interface>:*" in ZMQ).

This assumes that ZMQ will completely forget about the endpoint if/when it is disconnected, if it is set not to reconnect.  Otherwise I might run afoul of ZMQ’s silently ignoring connections to endpoints that it already knows about:  https://github.com/zeromq/libzmq/issues/788 <https://github.com/zeromq/libzmq/issues/788> (e.g., in the case where another process later happens to be assigned the same ephemeral port).

I’ve done a quick scan of the libzmq code (v4.2.2) and it doesn’t appear that the endpoint is removed in the case of a (terminal) disconnect.  If you can confirm/deny this behavior, that would be helpful.  Failing that, I guess I’ll need to test this in the debugger — any hints on how best to do this would also be much appreciated.

Regards,

Bill

> On Sep 1, 2017, at 6:19 PM, Luca Boccassi <luca.boccassi at gmail.com> wrote:
> 
> On Fri, 2017-09-01 at 18:03 -0400, Bill Torpey wrote:
>> Thanks Luca!  That was very helpful.
>> 
>> Although it leads to a couple of other questions:
>> 
>> - Can I assume that a ZMQ disconnect of a tcp endpoint would only
>> occur if the underlying TCP socket is closed by the OS? Or are there
>> conditions in which ZMQ will proactively disconnect the TCP socket
>> and try to reconnect?
> 
> Normally that's the case - you can set up heartbeating with the
> appropriate options and that will kill a connection if there's no
> answer
> 
>> - I see that there is a sockopt (ZMQ_RECONNECT_IVL) that can be set
>> to -1 to disable reconnection entirely.  In my case, the the “data”
>> socket pair will *always* connect to an ephemeral port, so I *never*
>> want to reconnect.  Would this be a reasonable option in my case, do
>> you think?
> 
> If that makes sense for your application, go for it - in these cases
> the only way to be sure is to test it and see how it works
> 
>> - Would there be any interest in a patch that would disable
>> reconnects (controlled by sockopt) for ephemeral ports only?  I’m
>> guessing that reconnecting mostly makes sense with well-known ports,
>> so something like this may be of general interest?
> 
> If by ephemeral port you mean anything over 1024, then actually in most
> applications I've seen it's always useful to reconnect, and the
> existing option should be enough for those cases where it's not desired
> - we don't want to duplicate functionality
> 
>> Thanks again!
>> 
>> Bill 
>> 
>>> On Sep 1, 2017, at 5:30 PM, Luca Boccassi <luca.boccassi at gmail.com>
>>> wrote:
>>> 
>>> On Fri, 2017-09-01 at 16:59 -0400, Bill Torpey wrote:
>>>> I'm curious about how ZMQ handles re-connection.  I understand
>>>> that
>>>> re-connection is supposed to happen "automagically" under the
>>>> covers,
>>>> but that poses an interesting question.
>>>> 
>>>> To make a long story short, the application I'm working on uses
>>>> pub/sub sockets over TCP. and works like follows:
>>>> 
>>>> At startup:
>>>> 1.  connects to a proxy/broker at a well-known address, using a
>>>> pub/sub socket pair ("discovery");
>>>> 2.  subscribes to a well-known topic using the "discovery" sub
>>>> socket;
>>>> 3.  binds a different pub/sub socket pair ("data") and retrieves
>>>> the
>>>> actual endpoints assigned;
>>>> 4.  publishes the "data" endpoints from step 3 on the "discovery"
>>>> pub
>>>> socket; 
>>>> 
>>>> When the application receives a message on the "discovery" sub
>>>> socket, it connects the "data" socket pair to the endpoints
>>>> specified
>>>> in the "discovery" message.
>>>> 
>>>> So far, this seems to be working relatively well, and allows the
>>>> high-volume, low-latency "data" messages to be sent/received
>>>> directly
>>>> between peers, avoiding the extra hop caused by a proxy/broker
>>>> connection.  The discovery messages use the proxy/broker, but
>>>> since
>>>> these are (very) low-volume the extra hop doesn't matter.  The
>>>> use of
>>>> the proxy also eliminates the "slow joiner" problem that can
>>>> happen
>>>> with other configurations.
>>>> 
>>>> My question is what happens when one of the "data" peer sockets
>>>> disconnects.  Since ZMQ (apparently) keeps trying to reconnect,
>>>> what
>>>> would prevent another process from binding to the same ephemeral
>>>> port?  
>>>> 
>>>> - Can I assume that if the new application at that port is not a
>>>> ZMQ
>>>> application, that the reconnect will (silently) fail, and
>>>> continue to
>>>> be retried?
>>> 
>>> The ZMTP handshake would fail, so yes.
>>> 
>>>> - What if the new application at that port *IS* a ZMQ
>>>> application?  Would the reconnect succeed?  And if so, what would
>>>> happen if it's a *DIFFERENT* ZMQ application, and the messages
>>>> that
>>>> it's sending/receiving don't match what the original application
>>>> expects?
>>> 
>>> Depends on how you handle it in your application. If you have
>>> security
>>> concerns, then use CURVE with authentication so that only
>>> authorised
>>> peers can connect.
>>> 
>>>> It's reasonable for the application to publish a disconnect
>>>> message
>>>> when it terminates normally, and the connected peers can
>>>> disconnect
>>>> that endpoint.  But, applications don't always terminate normally
>>>> ;-)
>>> 
>>> That's a common pattern. But the application needs to handle
>>> unexpected
>>> data somewhat gracefully. What that means is entirely up to the
>>> application - as far as the library is concerned, if the handshake
>>> succeeds then it's all good (hence the use case for CURVE).
>>> 
>>>> Any guidance, hints or tips would be much appreciated -- thanks
>>>> in
>>>> advance!
>>> 
>>> -- 
>>> Kind regards,
>>> Luca Boccassi_______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org> <mailto:zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>>
>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev <https://lists.zeromq.org/mailman/listinfo/zeromq-dev>
>>> <https://lists.zeromq.org/mailman/listinfo/zeromq-dev <https://lists.zeromq.org/mailman/listinfo/zeromq-dev>>
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev <https://lists.zeromq.org/mailman/listinfo/zeromq-dev>
> 
> -- 
> Kind regards,
> Luca Boccassi_______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev <https://lists.zeromq.org/mailman/listinfo/zeromq-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20170902/f29b72f3/attachment.htm>