[zeromq-dev] multi-connect REQ socket send routing

Jim Melton jim at melton.space
Sun Jun 7 05:09:49 CEST 2020


Carl,

ZeroMQ is more of a communications toolkit. It’s up to you to clearly understand your requirements and build the infrastructure that meets your requirements from the provided building blocks.

REQ/REP works best for a synchronous communication. As you note, it will block if the server is not ready (that’s the definition of synchronous, after all).

If you goal is to detect failed servers, you can accomplish with by setting LINGER to 0 and adding a timeout to your REQ socket. See https://github.com/zeromq/libzmq/issues/3914#issue-617731189 for a potentially similar design/problem observed using REQ/REP. In my case, however, I used a separate REQ socket for each “server”.

If you want reliable delivery, but don’t care about which server responds to the request, then you need to have a broker, who will select an “appropriate” server to handle the request. That’s the definition of a broker, and it is not built into the REQ socket.

FWIW, in the abstract client/server REQ/REP pattern, I tend to view the clients as ephemeral, while the servers are long-running (think web server as a simple analogy). Given that any miscommunication with between the REQ/REP can leave one or both of the sockets in an indeterminate state, I find it better to create a new REQ socket for each “conversation”. This allows for a sequence of related requests to be grouped and not have to establish a new socket for each request. In my usage, however, I make my request complex enough that everything I need is returned in a single reply, using multi-part messages to handle the details.
--
Jim Melton
(303) 829-0447
http://blogs.melton.space/pharisee/
jim at melton.space




> On Jun 6, 2020, at 12:57 PM, Carl Jenkins <cjenkins5614 at gmail.com> wrote:
> 
> Hi. I posted this last weekend, and further posted this on the pyzmq GitHub Issues, as well as the IRC channel. But I was not able to get any response.
> 
> I was able to reproduce this with Cppzmq as well, so it’s not a Python question.
> 
> What I really would like to know is, how do people do multiple server req-rep without a broker? Zeromq seems to offer this with one single socket connecting to multiple servers, judging by the standard doc, but in reality it hangs forever if any one of all servers disappears. It must be a common use case at scale, and how do people resolve it? Do they create a socket for each server, and cycle through them manually, or I’m missing something obvious here.
> 
> Best,
> Carl
> 
> On Sat, May 30, 2020 at 11:52 AM Carl Jenkins <cjenkins5614 at gmail.com <mailto:cjenkins5614 at gmail.com>> wrote:
> Hi. Just to follow up on the question. I think detecting dead server during REQ is covered in Chapter 4 Reliable Req-Rep Patterns, and the lazy pirate pattern deals with this case. In my situation, I ended up having another SUB socket receive heartbeats from servers and reconnect to the healthy ones when recv() times out.
> 
> This is still a bit involved (one more socket on client side, and some communication between the health checker and main thread) and led me into testing some simple cases. What I found was that if the binding servers are REP, and client is REQ, when one server dies, client won’t automatically choose a different connected server to send the request. However, if the binding server is REQ, and clients are REP, when one client dies, the server is able to fall back to other healthy clients.
> 
> The code is at 
> https://pastebin.com/RSJfdef1 <https://pastebin.com/RSJfdef1>.
> 
> Best,
> Carl
> 
> On Mon, May 25, 2020 at 3:23 PM Carl Jenkins <cjenkins5614 at gmail.com <mailto:cjenkins5614 at gmail.com>> wrote:
> Hello,
> 
> I started using zeromq for a project and have bee reading the guide (almost done with chapter 3. There is one occasion where I want to have a REQ client talk to multiple servers. I don’t plan to use the routing/load balancing pattern, with the intent to avoid single point of failure. If a server dies, I’d like the client to switch to a different server to do send-recv.
> 
> I have realized recv would block if that very host it’s talking to dies, and my solution is to set RCVTIMEO to a finite number (and LINGER to a finite number as well when socket terminates). However, for the send part, I can see send always seems to go through (I think due to buffering), and the subsequent recv would time out. This means send is still doing simple round-robin, otherwise recv would have been smooth, given other servers are healthy. I’m OK with buffering and send not reporting one server dying, but how can I ask zeromq to neglect the dead server from the round robin list?
> 
> This seems to be the documented behavior from https://zeromq.org/socket-api/ <https://zeromq.org/socket-api/>, but my tests above didn’t reproduce it:
> 
> If no services are available, then any send operation on the socket will block until at least one service becomes available. The REQ socket will not discard any messages.
> 
> Thanks,
> Carl
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20200606/066399dc/attachment.htm>


More information about the zeromq-dev mailing list