[zeromq-dev] Persistence with ZeroMQ

[Trevor Bernard]

> Thanks for the pointer. Could you please describe high level architecture of
> how you are using it with zeromq?

At a high level, my services flowed data in a pipeline. So for each
process, there would be an upstream and downstream zeromq socket each
running on their own thread. I choose push/pull but it could very
easily work with pub/sub and use xpub,xsub to extend it.

So on the upstream side, I would loop forever reading messages from
the socket and publishing onto a disruptor.

The first disruptor handler would serialize the message onto disk via
Chronicle queue. The second handler would do the business logic,
whatever that might be.. Then after it was done processing, it would
publish on the downstream socket to whoever was listening.

This worked pretty well and could achieve very high throughput with low latency.

-Trev