[zeromq-dev] Persistence with ZeroMQ

Holger Joukl Holger.Joukl at LBBW.de
Thu Oct 16 16:04:34 CEST 2014


thanks for your clarifications. Some follow-up coments/questions:

> Von: Pieter Hintjens
> A stream is a convenient top-level isolation. Typically one stream
> serves one application. Within streams, readers choose subjects they
> are interested in, using pattern matching.

Ok. Still more or less the same to me: A receiver subscribes to a stream
or a subset of the stream.

I quite like the analogon of tuning in to a radio frequency for
broadcast/multicast pub-sub. I guess it then depends on what you'd describe
a pub-sub "channel", i.e. (protocol, port) or (protocol, port,

So technically "streams" would then be discriminated by ports or you'd need
to have some kind or root topic/subject.

> > Would each malamute broker record the data all its connected consumer
> > clients are interested in or would the publisher's malamute broker
> record the data?
> Yes, the broker. Writers do not see readers.

I think my wording was a bit vague here: Suppose a topology of one malamute
broker per host and each client connecting to its local broker. Sender
and receivers are all on different hosts.

Will the sender's broker or the (multiple) receivers' broker(s) store
the messages?

> > How will be made sure that every published message will indeed get
> > persisted?
> There's no guarantee of this. We assume (hah!) that the broker is
> reliable, and that the failures we expect and have to solve are in
> clients. In practice we may use redundant brokers, for instance. To be
> seen.

Isn't it entirely possibly that the sending client works just fine but
and/or storing the data fails? Depends in part on how the client
with its broker I suppose but still sounds like some kind of acknowledge
would be necessary. I don't think redundant brokers would help here.

I think e.g. Apache Kafka uses an optional acknowledgement for producers.

> > When a consumer demands delivery of missed (by him) or historic
> > is the consumer responsible for its own delivery state and for sorting
> > duplicates?
> Yes. This would be hidden in the client stack.

I take it that this would then be functionality in a malamute client
that reveivers link/import/include.

> > - Where will the (PGM or NORM) multicast communication come into play?
> As an optional output socket for any given stream. Configurable, I guess.

I still don't get it. Would the clients listen to a multicast address
where a broker sends out data? Or would the multicast communication be
inter-brokers, the clients then getting the data from their brokers
over IPC or TCP?


Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

More information about the zeromq-dev mailing list