[zeromq-dev] Preserving order of replies in a multi threaded server

[Pieter Hintjens]

XRAP is in draft stage, and you're welcome to submit patches to the RFC.

On Sun, Jan 31, 2016 at 8:13 AM, Tom Quarendon
<tom.quarendon at teamwpc.co.uk> wrote:
> Well when there’s XRAP 2.0, I can consider that. In the meantime, I’m
> constrained by the definition of the protocol I’m trying to implement…
>
>
>
> From: zeromq-dev-bounces at lists.zeromq.org
> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Greg Young
> Sent: 30 January 2016 22:39
>
>
> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> Subject: Re: [zeromq-dev] Preserving order of replies in a multi threaded
> server
>
>
>
> It's worth mentioning this also leads to head of the line problems...
>
>
>
> Http2 removes the constraint of same order of responses... This is a major
> selling point of 2.0
>
> On Saturday, January 30, 2016, Tom Quarendon <tom.quarendon at teamwpc.co.uk>
> wrote:
>
> A number of things.
>
> HTTP1.1 has pipelining. Server has to return requests in same order as they
> were received. Simplest method to implement this is with thread per
> connection. Over a low latency connection there is little benefit to this
> though. Benefit comes in high latency situations.
>
> You can get serious performance benefits and still have the results returned
> in the same order as the requests. Consider that each request takes 1 minute
> to process. Client sends ten such requests to a multithreaded server. Server
> then processes them all concurrently, orders the results and sends them all
> back, total time taken 1 minute. If the client sends request, waits for
> response, sends request waits for response etc, total time taken 10 minutes.
>
> With HTTP there is no option but for the server to return the responses in
> the same order, there is no way that the client can otherwise correlate the
> responses, there is no unique ID to correlate by. For the client to do the
> correlation you need a unique request ID.
>
> The same is true in XRAP. There is no request ID for the client to correlate
> with. Hence the server has to return the requests in the same order, indeed
> the spec says as much.
>
>
>
> With zeromq there is no way to implement thread-per-connection as far as I
> known. So you need to find another way of giving rise to the same behaviour
> whilst ideally being able to process multiple requests at once from the same
> client. It complicates the server rather, but I think that the server has to
> manufacture a unique request number per client ID and then correlate the
> replies in order, before sending them back. Complicated.
>
>
>
>
>
> From: zeromq-dev-bounces at lists.zeromq.org
> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Doron Somech
> Sent: 30 January 2016 18:46
> To: ZeroMQ development list <zeromq-dev at lists.zeromq.org>
> Subject: Re: [zeromq-dev] Preserving order of replies in a multi threaded
> server
>
>
>
> IMO it is completely up to the client. Your solution for server is like
> saying don't send multiple request and will gave same performance. If you
> prefer client simplicity over performance in the client just send request
> synchronously, if you performance client need to know how to handle this.
>
> In HTTP server doesn't handle this and its up to the browser. Actually only
> HTTP 2.0 introduced multiple request on same connection.
>
> On Jan 30, 2016 18:52, "Tom Quarendon" <tom.quarendon at teamwpc.co.uk> wrote:
>
> I’m having difficulty understanding how I would preserve the relative order
> of replies to clients when I’m implementing a multithreaded server and when
> clients are allowed to pipeline requests (i.e send multiple requests without
> waiting for a response each time). So for example the XRAP protocol.
>
>
>
> So the obvious way of implementing the server seems to be to have a ROUTER
> socket that you read the requests from, then pass the requests to a pool of
> threads for processing. On completion the threads pass back the responses
> and then get forwarded back to the original requester.
>
> However if you implement that, then if clients can pipeline requests, so
> send multiple requests without waiting for responses, then there’s no
> guarantee that the responses get sent back in the same order. So client
> sends request1, then request2. Server receives request1, gives it to thread
> 1. Then server receives request2, gives it to thread2. Processing for
> request1 takes longer than processing for request2, so the response for
> request2 gets sent to the client before the response for request1. Client
> gets confused.
>
>
>
> It’s very unclear how you solve this. I think if I were doing this with
> normal TCP sockets, then my main loop would do a “select” on all the current
> connected client sockets. What I’d probably do is having received a request
> from a client, remove that socket from the poll list until the response is
> sent. That would reasonably simply give rise to the right behaviour. Haven’t
> actually done this, don’t know how HTTP servers actually do it, allow
> request pipelining, maybe there are better ways.
>
>
>
> Anyway, with zeromq sockets I don’t have that control, I don’t think. It’s
> not at all obvious what I would do.
>
> Can I somehow tell a zeromq socket to ignore one of the internal queues for
> a bit? I don’t think I can, but I may just not know what I’m looking for.
>
> I did wonder about having some kind of internal loopback queue, and having a
> list of “active” clients, and if I get a new message from one of those
> clients put it on that “loopback” queue. However I would appear to need one
> for each client, which would seem excessive, and also duplicating the
> internal zeromq buffers. If you only have one, then you’re going to go into
> a tight loop reading a message from it, then putting it back because you
> aren’t free to process another message for that client.
>
>
>
> If there was a sequence number in the messages, then I could push the
> problem to the client. So leave it up to the client to sort out messages
> coming back in the wrong order. However that complicates the client code and
> means you can’t use a “simple” zeromq API, and that code then has to be
> implemented in multiple ways if there are multiple client types. So I’m not
> sure that really helps.
>
>
>
> Any ideas?
>
>
>
> Thanks.
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> --
>
> Studying for the Turing test
>
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>