[zeromq-dev] Broadcasting Requests

[Fabien Ninoles]

Hello,

I'm quite new to zeromq but already have implement it in a couple of complex clustering setup.  So far, so good, performance is awesome and thanks a lot for this wonderful library.

My current challenge is to be able to broadcast a request to multiple node and collect back the results of all nodes.  At start with a pub/sub pipe for request and ROUTER/DEALER pipe for collecting the results, and, since it doesn't work (ROUTER need to be handshake before knowing the recipient, which make the full setup too complex for my taste), I hand up with a mix of two pub/sub channel, acting a little bit like the ROUTER/DEALER pair (frontends stack its identity, backends filter on top address and drop it).

Here a diagram of the resulting channel
 
+------+    +------+-----+    +-----+-----+    +-----+------+    +-----+
| XREQ | -> | XREP | PUB | -> | SUB | PUB | -> | SUB | XREQ | -> | REP |
|      | <- |      | SUB | <- | PUB | SUB | <- | PUB |      | <- |     |
+------+    +------+-----+    +-----+-----+    +-----+------+    +-----+

PUB/SUB pair add their "address" to the msg stack and subscribe to this address for efficiency.  SUB/PUB pair subscribe to everything and forward everything.  All devices are simple queue, forward msgs from the frontend to the backend and vice-versa.

For now, it's work well, and given enough intermediate queues, can even scale, IMHO.  However, I doesn't like some behaviour:

- It uses two ports for bidirectional input;
- Since the address is added by the frontend, it can't used its own transient identity.  To avoid using permanent identity, I add to add a separate identity to the frontend.
- Filtering is also done on the frontend, which means that it must discard some reply that it doesn't event need to see (and that the backend could have been already discard since it knows that the request didn't come from there).
- I need to put an additional socket on the frontend/backend to have a nice XREP/XREQ pipe (which have a better contract for this kind of messaging).

Moreover, since the behaviour of the PUBSUB/SUBPUB pair is exactly the same as an XREQ/XREP pair with simply the XREQ broadcasting the message instead of load-balancing it, I would like to know if there is feasible to have something resembling this in the future ?  May be a sockopt to allow XREQ to broadcast instead of load balancing ?

Thanks for your advices,

Fabien