[zeromq-dev] Several Edge Case Questions

[Sam Stein]

We're running on the majordomo model of ZMQ 4.0.4. The primary language used is Delphi. The use case is a logging system. Clients are sending data to the Broker, which distributes it to Workers to process into a database. The time it takes to get the data into the archive is relatively unimportant, but it's imperative that all data gets there eventually, and that Clients be able to run other tasks. We have been running various tests, and feel reasonably comfortable with the core ZMQ framework, bindings, and usage within applications. What we seem to need is some guidance regarding the capabilities and limitations involved with ZMQ.


Most of the issues we have are based on what happens if/when various components break. Some questions that have surfaced:


What happens if all the worker threads disappear? It looks like the data backs up inside the broker until it maxes its allocated RAM and crashes. Is there a 'standard' way to offload the queued information when no workers are available?

What happens if the clients outrun what the broker is capable of processing? It looks like the clients block until the broker becomes available again, but that becomes an issue for the client's other functions. Is there an alternative that moves the data to a holding zone?

What happens if the broker disappears altogether? It looks like the clients keep throwing their data at it if it isn't there. Is there a way to let them continue their work and queue the data until the broker comes back?


Any insight you can give us would be greatly appreciated.


Thank you,
Sam Stein

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20171030/953d8a8d/attachment.htm>