[zeromq-dev] Middleware du jour

[Len Holgate]

Steven McCoy <steven.mccoy <at> miru.hk> writes:

> I'd like to see a more portable ServerFramework but more important would be a
> complete suite of performance tests and comparisons between available options.
> The actual figures tend to be irrelevant, it is the relative differences
> between each sampling that is of interest.  I also realize for most users the
> present implementation is fast enough and correctness is of higher regard for
> which I am impressed with the attention given.
 
That's not really the market I'm in. I'd love to have the time to work on a
portable version but I tend to get driven by client demand and I have limited
time. 

The Server Framework isn't attempting to be a cross platform solution, neither
is it suggesting that the Windows approach is in any way better (from either a
performance or design perspective) than any other approaches. I hope that the
website doesn't give any other impression, but from your initial, rather
scathing, comments I guess it must be less clear than I had hoped (marketing
was never my favourite endeavour).

The point is simply that IOCP is the most efficient way to do this stuff on
a Windows platform and The Server Framework is designed to make this a 
little easier for people. 

In fact there has always been a trade off within The Server Framework between
performance and simplicity of development. I often struggle with blending
things just right, and, in general, simplicity wins. The underlying idea is
that if the framework is flexible enough to allow you to develop your
application quickly and then tune the code to obtain more performance if 
necessary. There are LOTS of little things that can bite you with Windows IOCP
solutions and preventing these issues from getting in the way often has
performance costs. I tend to favour an approach whereby the resulting
application is safe and correct and wont bring your box down due to
unconstrained non-paged pool usage (etc) and then if you want or need to
go faster you have a stable base to work from and you can remove some of the
abstractions or protections to get more speed. For most of my clients this
has worked well; we normally hit people's performance targets without any
need to heavily customise the code and simply having a system that will scale
well is often enough as it's often cheaper for my clients to buy bigger
hardware than to spend the development time needed to eek out the absolute best
performance. It's interesting that recently my game company clients have been
the ones driving the performance angle most, it always used to be the finance
guys. Annoyingly for me, there is no one true way, the applications that use
The Server Framework are so diverse that trying to provide the best
performance for all of the use cases at once simply isn't possible - which
leads to trade offs, compile time tuning and complexity that I'd prefer to do
without.

I agree that a set of performance comparisons would be useful but the problem
here is the risk of comparing apples with oranges.

I run a suite of performance tests with each new release so that I understand
how the various options available during compilation affect the end performance
and so that I can spot the simple mistakes that often destroy performance.
It's harder to come up with viable comparisons to alternative systems. For
one thing I'm not actually that interested in how we perform compared to other
operating systems as my clients are generally tied to Windows for other 
reasons and, as I mentioned above, I don't have the time or demand to support
other operating systems at present. Cross platform solutions often tend
not to have Windows as their primary target which means that lots of the
Windows specific issues that we address in The Server Framework are ignored
by cross platform systems; these things can and do impose a performance
penalty and whilst we can tune things to remove some of these items if required
I'd rather not be first in a 'runs very fast but crashes' competition. Finally
there's the time issue again; I don't have the time available to produce a set
of performance comparisons and I don't have the knowledge of all of the other
systems out there to be able to build realistic and idiomatic applications in
them to compare to The Server Framework. I would, of course, be more than
happy to work with someone who did have the time and provide a suitable
benchmark application written using The Server Framework.
  
For my users the important things tend to be time to market, correctness
and performance. For me they're correctness, performance and time to market - 
we juggle it around a bit but there's no point having the other two if the
code is going to break...

Len
http://www.lenholgate.com