[zeromq-dev] How to detect if a remote socket has closed

Tue Aug 17 15:15:51 CEST 2010

LOL. things never "just work" !!!

I'm finding that I'm hanging on socket creation a lot. Is that usual?

I'm running the zmq loop in a different thread and queuing the creation to that thread and waiting until creation is done (do while notCreated(): sleep 0.01 in python). I'm using the call back api in IOLoop to do the queuing.

My recovery strategy is:

Once I know there's a failure search for a remote connection that provides the same service, if I can't find one report back to the UI. this is more general than having a bunch of servers because a) the user might need to do the restart (in this case the crashed excel), b) the topology might not be known before hand (the user might be writing a "server" in excel that only needs to be around for a few minutes to get the particular job done).

-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Tue 17-Aug-2010 13:55
To: 0MQ development list
Subject: Re: [zeromq-dev] How to detect if a remote socket has closed

On Tue, Aug 17, 2010 at 2:47 PM,  <David.Briant at ubs.com> wrote:

> The challenge here is not one-to-many or many-to-many communication
> between sockets but the setup of network topology. I'm looking for a
> dynamic topology with mechanisms to inform me as part of the topology
> needs to change (e.g. due to crashes, shutdowns, new peers becoming
> available, etc).

Right.  0MQ aims to hide all this from your applications so that
things "just work", and can then be scaled transparently.  That can
make it frustrating if you want manual control over network
connections.

Imagine you want to manage not 2 but 2,000 Excel instances across a
thousand boxes and you see what the upside is.

But, OK, how to handle just the two instances... what you've not
explained is your recovery strategy.  If B fails, is there a C and
where?  Or do you want A to wait until B restarts?

What I think you need here is simply a reliable request-reply that is
robust against peer crashes, something I'm planning to design for the
user guide.

-Pieter

>
> Traditionally messaging can't do this. Creating brokers is adding
> another component to the system.
>
>
> I'm getting errors if I start up a client socket (REQ) before the server
> socket (REP). - "Assertion failed: fetched (z:\my
> documents\development\c++\zeromq-2.0.7\src\rep.cpp:232)" happens in my
> server process as I do the bind.
>
>
> -----Original Message-----
> From: zeromq-dev-bounces at lists.zeromq.org
> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
> Hintjens
> Sent: Tue 17-Aug-2010 13:15
> To: 0MQ development list
> Subject: Re: [zeromq-dev] How to detect if a remote socket has closed
>
> On Tue, Aug 17, 2010 at 1:59 PM,  <David.Briant at ubs.com> wrote:
>
>> I have a bunch of applications and services that I want to be able to
> communicate on an adhoc / peer basis. E.g. AppA can initiate connection
> and send messages to AppB or vice versa. AppA discovers AppB (or the
> other way round) using a directory (similar to zeroconf). If AppB
> crashes, the port (which is randomly assigned) is no longer connected to
> AppB, and potentially AppC could reuse it. So AppA needs to know that
> AppB is no longer available and needs to inform the user and potentially
> search for another peer that can be called instead of AppB.
>
> Consider using a broker as in
> http://www.zeromq.org/docs:user-guide#toc23.
>
>> If messages are lost (transparently) whilst AppA is down or the pairs
> can't reconnect automatically and have the queued messages sent on
> restart of AppA then I'm going to have to implement reliability at the
> application level - which seems a bit of a shame.
>
> It's a bit of a shame if you're used to traditional messaging, but 0MQ
> does not do any recovery in case of crashed nodes.  It's something I
> want to provide some reusable answers for in the user guide.
>
>> The implication of requiring a bind and connect means something needs
> to know / decide / coordinate who is connecting and who is binding. I.e.
> the application level needs to know who is listening for connection
> requests. If that is the case then the application level also needs to
> know if the listener is no longer available because it needs to find
> another listener (in my case using a directory for address lookup).
>
> You are IMO still at the learning stage where your knowledge of TCP
> sockets is messing with how you understand 0MQ sockets.
> http://www.zeromq.org/docs:user-guide#toc9
>
> Not that you can't do this at the application level, but it won't
> happen as you expect.
>
>> I'm also wanting something a bit more asynchronous than the REQ/REP
> socket. I.e. I'd like to be able to fire off several requests and wait
> for the responses, rather than have to wait.
>
> The ideal pattern for asynchronous task distribution is PUSH/PULL (aka
> DOWNSTREAM/UPSTREAM).  There is a worked example in the guide now.
> http://www.zeromq.org/docs:user-guide#toc6
>
> -
> Pieter Hintjens
> iMatix - www.imatix.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> Visit our website at http://www.ubs.com
>
> This message contains confidential information and is intended only
> for the individual named.  If you are not the named addressee you
> should not disseminate, distribute or copy this e-mail.  Please
> notify the sender immediately by e-mail if you have received this
> e-mail by mistake and delete this e-mail from your system.
>
> E-mails are not encrypted and cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost,
> destroyed, arrive late or incomplete, or contain viruses.  The sender
> therefore does not accept liability for any errors or omissions in the
> contents of this message which arise as a result of e-mail transmission.
> If verification is required please request a hard-copy version.  This
> message is provided for informational purposes and should not be
> construed as a solicitation or offer to buy or sell any securities
> or related financial instruments.
>
> UBS Limited is a company registered in England & Wales under company
> number 2035362, whose registered office is at 1 Finsbury Avenue,
> London, EC2M 2PP, United Kingdom.
>
> UBS AG (London Branch) is registered as a branch of a foreign company
> under number BR004507, whose registered office is at
> 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom.
>
> UBS Clearing and Execution Services Limited is a company registered
> in England & Wales under company number 03123037, whose registered
> office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom.
>
> UBS reserves the right to retain all messages. Messages are protected
> and accessed only in legally justified cases.
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>

-- 
-
Pieter Hintjens
iMatix - www.imatix.com
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Visit our website at http://www.ubs.com

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mails are not encrypted and cannot be guaranteed to be secure or 
error-free as information could be intercepted, corrupted, lost, 
destroyed, arrive late or incomplete, or contain viruses.  The sender 
therefore does not accept liability for any errors or omissions in the 
contents of this message which arise as a result of e-mail transmission.  
If verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities 
or related financial instruments.

UBS Limited is a company registered in England & Wales under company
number 2035362, whose registered office is at 1 Finsbury Avenue,
London, EC2M 2PP, United Kingdom.

UBS AG (London Branch) is registered as a branch of a foreign company
under number BR004507, whose registered office is at
1 Finsbury Avenue, London, EC2M 2PP, United Kingdom.

UBS Clearing and Execution Services Limited is a company registered
in England & Wales under company number 03123037, whose registered
office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom.

UBS reserves the right to retain all messages. Messages are protected
and accessed only in legally justified cases.