[zeromq-dev] Can a Trading Platform rely on ØMQ?

Anatoly tolitius at gmail.com
Tue Sep 6 18:06:31 CEST 2011


    I have a financial exchange/trading client where (market) data needs to
be passed from a core trading system to a warehouse over TCP/IP. What I keep
seeing in the Guide is that "*In some rare cases ØMQ drops messages silently,
if there is no obvious strategy for recovering from the error*". Obviously
we can not allow any messages being dropped / ignored discarded silently. I
am going through the guide, and playing with several patterns to have a
better feel for ØMQ API and philosophy, but I would really appreciate
knowledge of somebody who already feels quite competent with the product to
save me some of the "midnight oil":

     1. Is there a built in mechanism to e.g. start off loading messages to
disk, in case a queue is close to be overflowed? Or to just STOP instead of
discard messages silently?

     2. Mad Black Box is something that looks the closest to what we need,
however we already have publishers themselves sharded. Would sharding them
further into different Subscriber's "PUSH"ers be necessary to avoid a
"subscriber's overflow"? ( e.g. processing side would be slower, as it
deserializes [probably google protobufs] and persists messages to disk )

     3. In case the answer to 3 is YES, does not an additional internal
sharding introduce a new point of failure ( e.g. a thread dies, etc.. ), in
which case is there a some kind of built in recovery / retry mechanism (
similar to PGM, but a bit of a higher level, since we are dealing with ZMTP
messages )?

Thank you,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110906/6e32adbc/attachment.htm>

More information about the zeromq-dev mailing list