[zeromq-dev] content based routing for a thread pool

[Lei Jiang]

Thanks a lot for the quick and detailed reply Brett!

The MDP pattern is very interesting but I can't use it in my case. I am
aiming to share data and resources inside a process so can't go
inter-process. I'm also trying to maximize concurrency and minimize latency
so ideally as few hops as possible. The other thing is the clients don't
know the worker IDs in advance. The workers are of the same type of
service. The reason I want to route messages by key(s) is that I must
maintain sequence of messages of the same key so the only possible way
seems to me to be to hash the key and send the messages to certain workers
based on the hash.

The server/client socket looks very promising. But there are 2 difficulties
for me. First they are in draft so I have to get them from an unstable
version, small hurdle. Second by reading the code they do not seem to
support muti-part messages, a deal breaker.

It looks like I might have to replicate the pipe, and
fair-queue/load-balancer to achieve my goal.

Cheers,
Lei

On Sat, Oct 21, 2023 at 11:58 PM Brett Viren <bv at bnl.gov> wrote:

> Hi Lei,
>
> I think what you describe matches the majordomo pattern (MDP).
>
>
> https://zguide.zeromq.org/docs/chapter4/#Service-Oriented-Reliable-Queuing-Majordomo-Pattern
>
> In that description of MDP, the request from a client includes a
> "service name".  The broker uses that name to route the request to one
> worker in a subset of workers that had previously registered that
> service name when they requested work from the broker.
>
> If you can translate your requirement to "route the requests to certain
> threads based on the contents" into this "service name" mechanism then I
> think you can adopt MDP just as it is described in the guide.
>
> Alternatively, you can keep the "broker" unaware of types of service, as
> in the preceding PPP description, and instead develop a single type of
> worker that internally implements every type of service.  That worker
> must then interpret the request to determine the type of service prior
> to applying the functionality that actually implements the service.  Of
> course, this "multi-service-worker" pattern can only work if the broader
> architecture allows all worker instances to access all resources
> required to satisfy all supported types of service.  Eg, if one type of
> service requires some backend DB access then the entire architecture
> must be prepared for some time where all workers simultaneously try to
> access the backend DB.  That is, multi-service-worker PPP would not have
> any load balancing features that MDP has (other than simply controlling
> the total number of workers).
>
>
> Note, for whatever pattern you end up with, if your sockets are
> themselves running on threads in a pool in a way that you can not
> guarantee each socket is accessed from exactly the same thread over its
> lifetime then you may run afoul of the thread non-safety of the
> (non-draft) ZeroMQ sockets.  This problem can be solved by switching
> from ROUTER/DEALER based MDP to one that uses the draft SERVER/CLIENT
> sockets (which are safe call from multiple threads).  If this problem
> indeed strikes you, you can re-implement MDP on SERVER/CLIENT.  I have
> an example of this in my "generaldomo" package on github.
>
> -Brett.
>
> Lei Jiang <lei.jiang.29 at gmail.com> writes:
>
> > Hi,
> >
> > I hope someone can kindly give me a point in the right direction.
> >
> > What I want to achieve is to have a thread pool with a bunch of REP
> sockets in each thread to handle requests. A
> > ROUTER socket binds to the server address:port. A DEALER socket binds to
> an inproc address. All REP sockets connect
> > to the inproc address. And finally a proxy() call will bridge the ROUTER
> and DEALER sockets.
> >
> > This pattern works as it is. But I need more. The pool must route the
> requests to certain threads based on the
> > contents. For that, I am ready to write my own proxy() function. I read
> from the guide that putting an identity
> > frame followed by a blank frame and then the message body can route the
> message to a certain peer. But my bigger
> > problem is it seems I can't use DEALER any more. It has to be ROUTER <=>
> ROUTER, and I will have to maintain the
> > sessions myself. I guess largely it's what's in the pipe_t, which is not
> exposed.
> >
> > Am I right about this? Or is there a better way?
> >
> > Thanks a lot!
> >
> > Lei
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> >
> https://urldefense.com/v3/__https://lists.zeromq.org/mailman/listinfo/zeromq-dev__;!!P4SdNyxKAPE!G_ci73VSJTnWAZt5Xszv_uJ1IxqygfqNgOdFgBMDTMHyy59koUsHkTTnzeRN6VxpuyRm3zq2cBXkc22HXQ$
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20231022/415a564f/attachment.htm>