[zeromq-dev] LMAX/Distruptor code project
Steven McCoy
steven.mccoy at miru.hk
Thu Jul 28 15:09:04 CEST 2011
On 28 July 2011 19:43, Ian Barber <ian.barber at gmail.com> wrote:
>
>> Memory management and NUMA-awareness is very difficult, ptmalloc3 is
>> incredibly good that you don't need your own SLAB manager like GLib has;
>> Windows equivalent is the Concurrency Runtime. The fight is between
>> pre-allocating a large arena and optimising that for one fast consumer or
>> producer, and making a multi-thread friendly architecture that allows
>> multiple threads to publish or consume. With PGM there is a big design
>> issue with how to handle new publishers, whether you have a bus topology or
>> limit to one sender at a time and recycle the receive window. It is very
>> expensive to allocate large chunks of memory for each new receiver.
>>
>
> From what I understand the dispruptor basically acts as a bus with one
> large allocated big of memory, but once a writing process has a slot it can
> happily write into that part of the buffer without clashing with other
> publishers, as they attempt to pad their slots to be as cache independent as
> possible. I believe that most of the actual entry structures are
> pre-allocated as well, in part of avoid GC by making them very long lived,
> again I think that reflects the nature of events they're processing.
>
>
It's all on Java which makes the figures even more impressive, although I'm
not aware whether this is Java 7 or a real time variant running on a real
time optimised Linux like SuSe Enterprise.
Java 7 is required for NIO multicast which is why I've been holding off.
--
Steve-o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110728/bb9abc48/attachment.htm>
More information about the zeromq-dev
mailing list