[zeromq-dev] If not Majordomo for asynchronous workers, then what?

[Ivan]

Pieter,

Pieter Hintjens <ph <at> imatix.com> writes:

> 
> Hi Ivan,
> 
> Majordomo is based on a 1-to-1 request-reply model for workers, yes.
> You should be able to quite easily modify MDP to allow multiple
> replies, perhaps by adding a "there's more coming" indicator to
> replies that the broker uses to route several replies back to the same
> client.

It seems like I will have to map a clientOrderID to a particular client. That 
way, when the FIX engine sends back more messages, I can look up this order ID, 
get the client, and send it back to that particular client.

> Modifying MDP and the implementations will be simpler than writing
> your own from scratch.

Right. I believe what I describe above is the correct modification to the 
Broker. I have to understand the magic by which the broker knows how to route a 
message back to the client. It would be a great addition to MDP if I could call 
a function that, given a message, would break the multipart message for me. This 
way I could hash the route to the client based on some key, in my case, OrderID.

> You can also use other architectures; MDP is specifically aimed at
> allowing multiple redundant workers per service type. If you have a
> single well-known gateway per exchange you can just connect each
> client to each exchange gateway directly and work fully
> asynchronously.

Having the clients know of the routes would break encapsulation, to say nothing 
of the fact that there can be all sort of different protocols like FIX, ITCH, 
proprietary binary, etc. The idea is that a client can send normalized/generic 
order to a broker, who then fills in the appropriate details as to how to get 
that order to a particular exchange. On the way back from the exchange, the 
reverse process occurs, where a given message from an exchange is normalized 
back into a simpler internal message. So complexity is hidden from the client.


> -Pieter

Thanks again Peiter for your suggestions. Is there any way I can contact you 
directly?

Ivan


> On Thu, Jul 26, 2012 at 10:29 PM, Ivan Figueredo <ivan <at> blackpx.com> 
wrote:
> > I am writing a distributed trading manager. There are many different
> > liquidity providers, and the client would like to route to a given exchange.
> > With this in mind, I thought that using the Majordomo pattern with a broker
> > in between clients and workers asking a particular [named] worker to do its
> > work. However at least as I understand it, Majordomo won't work in this
> > scenario.
> >
> > Let's take the FIX protocol. Since FIX is asynchronous, I believe I can't
> > use the Majordomo. You may send an order to a FIX engine, and that one order
> > may send back many different messages. For example, that one order may see
> > the following responses: Insert, Pending, New, Partial Fill, then finally
> > Fill.
> >
> > In the Majordomo pattern, a section of code in the worker looks like this
> >
> > mdwrk session ("tcp://localhost:5555", sourceStr.c_str(), verbose);
> >
> > zmsg *reply = 0;
> > while (1) {
> >     zmsg *request = session.recv (reply);
> >     if (request == 0) {
> >         break;              //  Worker was interrupted
> >     }
> >
> >     //reply = request;        //  Echo is complex... 
> >     reply = new zmsg(sourceStr.c_str());
> > }
> >
> >
> > This assumes one request, and one reply. Since my application is not
> > strictly a single request, that gets a single response as in the code above,
> > do I have to drop Majordomo ?
> >
> >
> > Thanks!
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev <at> lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>