[zeromq-dev] Paranoid Pirate pattern with priority queuing

[Pieter Hintjens]

We moved the resource functionality to RFC 37 (draft), as it didn't
get into ZeroMQ 4.0.

On Mon, Apr 27, 2015 at 3:29 PM, James & Veri <verimart at hotmail.com> wrote:
> Assume you have a requirement to support reliable priority queues whereby,
> if there is ever a message sent by a client with a higher priority (say 3
> queues, high/medium/low) workers will always see those requests before they
> see lower priority requests. I'm currently working with jeromq and netmq, so
> the solution should be based on zmq 3.2.5.
>
> I have thought of doing this in the proxy with an endpoint per priority on
> the front end and one endpoint for the back end. I'm thinking the
> prioritization would come from the proxy by organizing its poller's inputs
> such that each iteration through its polling loop checks for input in
> descending order from highest to lowest and delivers from the first frontend
> socket available for reading. Does this sound right? I assume the logic at
> the top of the poller loop still hold (only poll for input on all of the
> priority endpoints if there's at least one 'ready' worker), right?
>
> Is this the simplest approach? It seems that it gets pretty port-hungry at
> the proxy as the number of such prioritized queues grows ( (# of priorities
> + 1) x (# of distinct prioritized queues) ). I'm thinking one poller thread
> per distinct prioritized queue.
>
> Also, I seem to remember somewhere that versions of zeromq supporting zmtp
> 4.0 were expected to alleviate this by supporting ipc/tcp endpoints that can
> specify different resources at the same endpoint, ala
> tcp://*:5555/queue-name/priority-value.  Looking through the 'what's new'
> notes for the latest zmq 4.0 builds though, I didn't seem to find anything
> like this. Is this still planned (or already available)? If so, is dev work
> in progress for the managed implementations (jeromq/netmq)?
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>