[zeromq-dev] Is there already native "high level" connection pool?

Stefan de Konink stefan at konink.de
Tue Nov 27 14:33:37 CET 2012


On 11/27/12 13:49, Pieter Hintjens wrote:
> Please explain what you are actually trying to achieve, not in terms
> of 0MQ but in terms of the actual overall problem, and we'll be able
> to help more.

I would like to send an incomming HTTP POST to an undefined number of
clients on the internet, having an envelope (the HTTP path) and data the
HTTP POST. The two constraint of this process is minimizing latency and
guaranteed delivery.

In order to handle large HTTP POST each incomming HTTP request is
compressed using gzip. When the request comes in, an envelope is send,
with SND_MORE. The HTTP POST is compressed on the fly, each interation
the webserver receives data on the socket for that request, gzip is
called, and the partial result is send over a ZeroMQ socket using
SND_MORE. When the HTTP POST is completely processed no SND_MORE flag is
append.

In order to handle concurrent requests the above process creates a new
socket for each HTTP POST, in any other way the concurrency could
already mess up the envelope and first data packet after it which are
already two calls. The creation of new sockets per request doesn't scale
- it does solve my problem but it is not elegant.

A potential solution is buffering the compressed request, and send both
envelope and data after eachother. Still the zeromq socket should be
locked for that, or a per thread socket should be created. The solution
I would prefer is reusing used sockets, based on a pool. It seems that
this functionality is not available yet as an abstract solution.


Stefan



More information about the zeromq-dev mailing list