[zeromq-dev] malamut matching engine & disappearing subscription

[Benoit Chesneau]

Sorry for the late answer.

Thanks for the explanations anyway. I will have a  closer look at the
OpenAMQ implementation as well. Anyway the design of malamut is very
interesting :)

- benoît

On Wed, Aug 26, 2015 at 12:09 PM Pieter Hintjens <ph at imatix.com> wrote:

> The mlm_stream_simple engine doesn't do the bitmap magic. It uses
> regexps and matches every message against every pattern, so it's
> simple (hence the name) yet O(n2) slow.
>
> The bitmap design comes from OpenAMQ and it proven, and fast. The
> trouble is it's quite delicate machinery (so takes time) and I'm
> waiting for an opportunity to make it. It recalculates matches as it
> gets new message topics, and new/canceled subscriptions, which
> introduces latency peaks. However these are minimal in practice since
> the sets of topics and subscriptions are stable over time. We can in
> theory cache these to reduce runtime recalculation. In OpenAMQ
> deployments we never had to do this.
>
> -Pieter
>
> On Wed, Aug 26, 2015 at 11:42 AM, Benoit Chesneau <bchesneau at gmail.com>
> wrote:
> > hi all,
> >
> > I was reading the malamut white paper about the matching engine, and I am
> > trying to understand what I miss in using an inverted bitmap index to
> match
> > consumers subscriptions. I tried to look at the source code but it seems
> > that zbits is not used right now.
> >
> > From what I understand, bitmaps key are the subscription fields with
> their
> > value (ex. `market=forex`) and the bitmap  allows to match the
> subscriptions
> > (each subscription item is a bit set to 1 for a field). If I read the
> given
> > example, the size of the bitmap equals to the number of subscriptions.
> >
> > My question is, what happen when a subscription is closed? Are all the
> > bitmaps for the fields recalculated ? Or just the bit removed for all the
> > fields of the subscription? Is the bitmap size dynamic? Or did I miss
> > something?
> >
> > In that case is this designed for the case where we can have a lot of
> > subscription created and removed over the time?
> >
> >
> > - benoît
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150908/0f7f8910/attachment.htm>