[zeromq-dev] Persistence with ZeroMQ

[Pieter Hintjens]

Mohit,

I've started on a new broker project,
https://github.com/Malamute/malamute, which aims to do what you're
asking for.

If you would like to read the description and tell me how closely it
matches, that would be helpful.

-Pieter

On Wed, Oct 1, 2014 at 9:19 PM, Trevor Bernard <trevor.bernard at gmail.com> wrote:
>> Thanks for the pointer. Could you please describe high level architecture of
>> how you are using it with zeromq?
>
> At a high level, my services flowed data in a pipeline. So for each
> process, there would be an upstream and downstream zeromq socket each
> running on their own thread. I choose push/pull but it could very
> easily work with pub/sub and use xpub,xsub to extend it.
>
> So on the upstream side, I would loop forever reading messages from
> the socket and publishing onto a disruptor.
>
> The first disruptor handler would serialize the message onto disk via
> Chronicle queue. The second handler would do the business logic,
> whatever that might be.. Then after it was done processing, it would
> publish on the downstream socket to whoever was listening.
>
> This worked pretty well and could achieve very high throughput with low latency.
>
> -Trev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev