[zeromq-dev] Just installed zmq and have a question about wiring and running a broker.

[Martin Sustrik]

Vineet Jain wrote:
> I currently have a large number of python applications on a compute
> farm. I was going to try use 0mq for logging and also for
> communication between each other. I would like my messaging
> architecture to be simple in the beginning. All my python apps would
> communicate to one messaging server which would host a global exchange
> and multiple queues.
> 
> I was hoping that all of my apps could connect to zmq_server which
> intern would host the global exchange and queues. From your reply, it
> seems like that is not possible.

Well, that's how it works currently. Have a look a chatroom example: 
There's a chatroom application, say 100 display applications and 100 
prompt applications. All these 201 applications share a common 
configuration repository (zmq_server). You can add new chatrooms easily 
(just run a new instance of a chatroom executable with a different 
name). This way you can scale up to 1000 chatrooms and millions of 
displays/prompts. The important thing is that chatrooms are different 
processes so you can place 500 of them on one server and another 500 on 
a different server, thus avoiding performance bottlenecks (both CPU 
power and network bandwidth).

You can of course create a single mega-chatroom application that would 
host all the chatrooms, however, it's hard to scale that kind of thing 
over a certain threshold.

> How would I go about creating a wiring where this is one centralized
> messaging server which just creates a global exchange and the clients
> are all responsible for the business logic, creating the queues and
> pushing and consuming messages.

Simply create N global exchanges/queues in a single application. The 
application can find out which queue the message was received from by 
looking at the queue ID and it can send message to particular exchange 
based on exchange ID.

Martin