[zeromq-dev] distributed message bus on top of / built with zeromq?

[Holger Joukl]

Hi Sabri,

zeromq-dev-bounces at lists.zeromq.org schrieb am 18.09.2014 19:07:02:

> Von: Sabri Skhiri <sabri.skhiri at gmail.com>
> We detected the necessity
> to create a lightweight and elastic bus on top of ZeroMQ in the end of
> 2010. At that time we started an internal research project and we
> published our first conclusion in 2011 at the IEEE CloudCom conference
> (http://euranova.eu/download?type=publications&title=EQS%3A+AN
> +ELASTIC+AND+SCALABLE+MESSAGE+QUEUE+FOR+THE+CLOUD).
> The paper explains how we designed an elastic message bus on top of
> ZMQ.
> After the publication, the compmunity asked for shaing the code and we
> open Sourced RoQ (http://roq-messaging.org/). Today, this is still a
> proof of concept and not a product yet, however we still use it within
> EURA NOVA for going further in the understanding of all the technical
> aspects of a distributed message bus (HA, reliability, scalability,
> etc.).

Thanks for sharing this information! This certainly looks interesting.

I've now read your EQS paper
(http://roq-messaging.org/docs/EQS.pdf) and looked at your future plans
(https://github.com/roq-messaging/RoQ/wiki/Roadmap-%26-Milestones)

...and I do have some questions :-):

- Does one EQS queue cover multiple topics (or subjects)?

- Does one EQS exchange cover multiple topics/subjects?

- Is a producer of 1 topic/subject connected to one exchange?

- Are 2 listeners on the same topic/subject always connected to the same
exchange?

- During relocation of a producer, I take it that listeners don't lose
messages (producer probably
stops sending messages during relocation)?

- Did you define any wire protocol for RoQ?

- With regard to your elastic scaling algorithm:
What happens if one single producer overloads an exchange all by itself?

- I understand RoQ uses TCP/IP for transport. Do you have any plans on
using UDP multicast? How
would you scale for a great number of listeners for one topic/subject?

- What kind of QoS or delivery guarantee are you targetting for your future
plans on persistence and
reliability, e.g. at-least-once, exactly-once,...?

- Is my understanding correct that the RoQ management components are single
points of failure of
the RoQ infrastructure and needed to be clustered / hot standby?

- Citing your future plans:
"[...] UC3 - Reconnected subscriber: if a subscriber disconnect from the
topic it must be able to either (1)
receive all the events from the beginning of the topic, (2) receive all the
events from the last connection or
(3) receive no historical events and just starts listening new ones. For
the case (2) we take as assumption
that the client must keep the state of its last connection (last message
sequence ID for instance).[...]"
Do you plan on implementing this in the subscriber client API layer or will
this be the responsibility of the
subscriber application?

- Is my understanding correct that RoQ is currently Java-only? Are the API
calls and interactions
formally specified for development of non-Java clients?

Best regards
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart