[zeromq-dev] QoS support

[Erich Heine]

Hi Martin,

In general I was providing a case for end-to-end qos, as opposed to Steve's
assertion that it only makes sense to do QoS starting at the gateway level
and that QoS is only meaningful on routers.  (in that regard I think he and
I are using differing definitions of QoS).

More inline below:

On Tue, Nov 10, 2009 at 4:32 PM, Martin Sustrik <sustrik at 250bpm.com> wrote:

>
> Hi Erich,
>
> AFAIU what you are saying is that there's sense in tagging packets by
> different TOSes depending on the message contained in the packet, right?
>
> Absolutely.


> If so, the problem is that many messages can be sent in a single packet
> to gain performance boost. In that case we would have to use highest
> priority for such a packet thus giving other messages unfair preference.
>

That is one potetntial solution. I see no problem giving the higher priority
to the piggy backers. Another solution would be to limit aggregation to
messages of similar priority. Yet another would be priority queueing within
the middleware itself.


>

As for 1. and 2. can you give a concrete example? My feeling is that our
> thoughts are basically in line just that we are expressing them in a
> different way.
>
>
Sorry I should have revised my statements better, I hope this clears them
up.

For 1. imagine a complex distributed control system. there are multiple
hosts on the network, each with sensor inputs.  Values may be considered
critical, important, or nice to have. The application(s) which run on each
host deal with raw sensor input and determine how important a value is, as
do "centralized" monitoring applications (control rooms, or wide area
algorithms). The general domain modeling relies on sensor names in such
cases.  Subscriptions to individual sensors come and go, and as mentioned
above the priority of those may be in flux as well.

In such a case app writing becomes much easier if the application can do
subscription and publishing based on sensor names, and do priority with a
simple modification to "send message" parameters. An alternative would be to
have 3 connections corresponding to importance, and embed the sensor name in
the message. The former case is harder to administer with external QoS
parameter tuning, as there would need to be all sorts of deep packet
inspection, on many connections. The latter would require much more
consideration from the application writers, potentially including their own
priority queueing.  Basically my position is that since this is a control
network, and therefore the hosts are pretty well trusted, we can make a
smart middleware to do the QoS params, and trust it at the routers (i.e. the
first solution).

Point 2, is a slightly different take on the same notion. It assumes that
some applications will not partake in the system QoS of their own accord. In
such cases, some QoS mechanisms on the host will go a long way. Say in the
example above there is an app that aggregates security video feeds and sends
them along the network. For whatever reason this app runs on the same box as
the control apps (its a beefy box at a small plant or something). The video
system doesn't do QoS with the control system. So we need to have host based
QoS here, with priority queueing making the video a lower priority via
system intervention. The video may or may not be tagged on egress, that is a
separate issue, what we are doing here is providing a guarantee to the the
control system that it will be able to do its own thing correctly.  Further
say each individual sensor and camera steam comes in on the LAN. We need
ingress controls to prioritize handling of sensor data over video data.

Regards,
Erich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20091111/7eb0ac40/attachment.htm>