[zeromq-dev] LMAX/Distruptor code project

Steven McCoy steven.mccoy at miru.hk
Thu Jul 28 15:09:04 CEST 2011

On 28 July 2011 19:43, Ian Barber <ian.barber at gmail.com> wrote:

>> Memory management and NUMA-awareness is very difficult, ptmalloc3 is
>> incredibly good that you don't need your own SLAB manager like GLib has;
>> Windows equivalent is the Concurrency Runtime.  The fight is between
>> pre-allocating a large arena and optimising that for one fast consumer or
>> producer, and making a multi-thread friendly architecture that allows
>> multiple threads to publish or consume.  With PGM there is a big design
>> issue with how to handle new publishers, whether you have a bus topology or
>> limit to one sender at a time and recycle the receive window.  It is very
>> expensive to allocate large chunks of memory for each new receiver.
> From what I understand the dispruptor basically acts as a bus with one
> large allocated big of memory, but once a writing process has a slot it can
> happily write into that part of the buffer without clashing with other
> publishers, as they attempt to pad their slots to be as cache independent as
> possible. I believe that most of the actual entry structures are
> pre-allocated as well, in part of avoid GC by making them very long lived,
> again I think that reflects the nature of events they're processing.
It's all on Java which makes the figures even more impressive, although I'm
not aware whether this is Java 7 or a real time variant running on a real
time optimised Linux like SuSe Enterprise.

Java 7 is required for NIO multicast which is why I've been holding off.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110728/bb9abc48/attachment.htm>

More information about the zeromq-dev mailing list