[zeromq-dev] Preserving order of replies in a multi threaded server

[Greg Young]

It's worth mentioning this also leads to head of the line problems...

Http2 removes the constraint of same order of responses... This is a major
selling point of 2.0

On Saturday, January 30, 2016, Tom Quarendon <tom.quarendon at teamwpc.co.uk>
wrote:

> A number of things.
>
> HTTP1.1 has pipelining. Server has to return requests in same order as
> they were received. Simplest method to implement this is with thread per
> connection. Over a low latency connection there is little benefit to this
> though. Benefit comes in high latency situations.
>
> You can get serious performance benefits and still have the results
> returned in the same order as the requests. Consider that each request
> takes 1 minute to process. Client sends ten such requests to a
> multithreaded server. Server then processes them all concurrently, orders
> the results and sends them all back, total time taken 1 minute. If the
> client sends request, waits for response, sends request waits for response
> etc, total time taken 10 minutes.
>
> With HTTP there is no option but for the server to return the responses in
> the same order, there is no way that the client can otherwise correlate the
> responses, there is no unique ID to correlate by. For the client to do the
> correlation you need a unique request ID.
>
> The same is true in XRAP. There is no request ID for the client to
> correlate with. Hence the server has to return the requests in the same
> order, indeed the spec says as much.
>
>
>
> With zeromq there is no way to implement thread-per-connection as far as I
> known. So you need to find another way of giving rise to the same behaviour
> whilst ideally being able to process multiple requests at once from the
> same client. It complicates the server rather, but I think that the server
> has to manufacture a unique request number per client ID and then correlate
> the replies in order, before sending them back. Complicated.
>
>
>
>
>
> *From:* zeromq-dev-bounces at lists.zeromq.org
> <javascript:_e(%7B%7D,'cvml','zeromq-dev-bounces at lists.zeromq.org');>
> [mailto:zeromq-dev-bounces at lists.zeromq.org
> <javascript:_e(%7B%7D,'cvml','zeromq-dev-bounces at lists.zeromq.org');>] *On
> Behalf Of *Doron Somech
> *Sent:* 30 January 2016 18:46
> *To:* ZeroMQ development list <zeromq-dev at lists.zeromq.org
> <javascript:_e(%7B%7D,'cvml','zeromq-dev at lists.zeromq.org');>>
> *Subject:* Re: [zeromq-dev] Preserving order of replies in a multi
> threaded server
>
>
>
> IMO it is completely up to the client. Your solution for server is like
> saying don't send multiple request and will gave same performance. If you
> prefer client simplicity over performance in the client just send request
> synchronously, if you performance client need to know how to handle this.
>
> In HTTP server doesn't handle this and its up to the browser. Actually
> only  HTTP 2.0 introduced multiple request on same connection.
>
> On Jan 30, 2016 18:52, "Tom Quarendon" <tom.quarendon at teamwpc.co.uk
> <javascript:_e(%7B%7D,'cvml','tom.quarendon at teamwpc.co.uk');>> wrote:
>
> I’m having difficulty understanding how I would preserve the relative
> order of replies to clients when I’m implementing a multithreaded server
> and when clients are allowed to pipeline requests (i.e send multiple
> requests without waiting for a response each time). So for example the XRAP
> protocol.
>
>
>
> So the obvious way of implementing the server seems to be to have a ROUTER
> socket that you read the requests from, then pass the requests to a pool of
> threads for processing. On completion the threads pass back the responses
> and then get forwarded back to the original requester.
>
> However if you implement that, then if clients can pipeline requests, so
> send multiple requests without waiting for responses, then there’s no
> guarantee that the responses get sent back in the same order. So client
> sends request1, then request2. Server receives request1, gives it to thread
> 1. Then server receives request2, gives it to thread2. Processing for
> request1 takes longer than processing for request2, so the response for
> request2 gets sent to the client before the response for request1. Client
> gets confused.
>
>
>
> It’s very unclear how you solve this. I think if I were doing this with
> normal TCP sockets, then my main loop would do a “select” on all the
> current connected client sockets. What I’d probably do is having received a
> request from a client, remove that socket from the poll list until the
> response is sent. That would reasonably simply give rise to the right
> behaviour. Haven’t actually done this, don’t know how HTTP servers actually
> do it, allow request pipelining, maybe there are better ways.
>
>
>
> Anyway, with zeromq sockets I don’t have that control, I don’t think. It’s
> not at all obvious what I would do.
>
> Can I somehow tell a zeromq socket to ignore one of the internal queues
> for a bit? I don’t think I can, but I may just not know what I’m looking
> for.
>
> I did wonder about having some kind of internal loopback queue, and having
> a list of “active” clients, and if I get a new message from one of those
> clients put it on that “loopback” queue. However I would appear to need one
> for each client, which would seem excessive, and also duplicating the
> internal zeromq buffers. If you only have one, then you’re going to go into
> a tight loop reading a message from it, then putting it back because you
> aren’t free to process another message for that client.
>
>
>
> If there was a sequence number in the messages, then I could push the
> problem to the client. So leave it up to the client to sort out messages
> coming back in the wrong order. However that complicates the client code
> and means you can’t use a “simple” zeromq API, and that code then has to be
> implemented in multiple ways if there are multiple client types. So I’m not
> sure that really helps.
>
>
>
> Any ideas?
>
>
>
> Thanks.
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> <javascript:_e(%7B%7D,'cvml','zeromq-dev at lists.zeromq.org');>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>

-- 
Studying for the Turing test
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160130/11c69aeb/attachment.htm>