[zeromq-dev] content based routing for a thread pool

Lei Jiang lei.jiang.29 at gmail.com
Mon Nov 13 13:02:01 CET 2023

Hi Brett,

Sorry for the late reply!

On Fri, Nov 3, 2023 at 12:40 AM Brett Viren <bv at bnl.gov> wrote:

> > A long multi-part message could potentially lead to embarrassing
> > situations quite easily. Even when the proxy code is fast, if
> > something gets stuck in either the request or the response, for
> > example a slow DB query, a long delay can be easily introduced.
> I may not follow.
> All parts of a message are transmitted atomically and asynchronously
> from the point of view of the application.  This happens in "the
> background" by ZeroMQ code running in its "I/O" threads.
> So, certainly a proxy may take a "nap" in the middle of recv()'ing
> message parts, but that does not impact the ongoing socket activity on
> the transmission side.  And, a slow marshalling inside the proxy is not
> an issue that relates to using multi-part messages as once the first
> part is available the proxy application can recv() the remaining parts
> immediately.
> It is true that if the proxy "naps" are required and they are long
> enough then they may limit overall latency and/or throughput.  But,
> again, that can be solved by having the proxy farm tasks out to workers
> in their own threads, allowing the proxy to go back to marshaling for a
> while.  Ie, the proxy becomes the broker in the MDP/PPP pattern.

What I meant was, ZMQ seems to keep multipart message frames together. I
wrote some test code, in the same thread, sending interleaving frames using
2 sockets like this.

ZMQFrame("frame 1-1").Send(client1, ZMQ_SNDMORE);
ZMQFrame("frame 2-1").Send(client2, ZMQ_SNDMORE);
ZMQFrame("frame 1-2").Send(client1, ZMQ_SNDMORE);
ZMQFrame("frame 2-2").Send(client2, ZMQ_SNDMORE);
ZMQFrame("frame 1-3").Send(client1, ZMQ_SNDMORE);
ZMQFrame("frame 2-3").Send(client2, ZMQ_SNDMORE);

After my proxy code I got complete messages in the workers separately in
the right order. I think it suggests that as long as the "active" message
has not finished, meaning it's still sending the "MORE" flag, a ROUTER
socket will not receive() a frame from a different client. Given that the
proxy loop is single threaded, that means it won't tend to other clients
until the current one is finished. If some of the message frames have
delays, they will surely cause delays in the processing of frames from
other clients as well.

I think this means the multipart is not meant to be used to carry really
long repeating parts but rather just one part of the message with different

> BTW, I explored this symmetry between ROUTER/DEALER and SERVER/CLIENT in
> my "generaldomo" implementation of MDP.

I have been reading the patterns in more detail recently. Some are so long
I did not read carefully earlier. MDP is very cool but I realized what I
did with my thread pool is actually a modified load balancing pattern. Now
that I have done it, things look stupidly simple.

The key really is the understanding of the format of the routing part of
the envelope, as well as the behavior of the ROUTER socket when it sends
and receives messages. It turns out what I needed to do is simply
prepending the worker ID, followed by an empty frame, to the requests. If
the worker keeps the routing part of the request, there isn't even the need
to do any mapping in the proxy, because the routing already has all
necessary information to return to the sender. The only thing I need to do
is to remove the first 2 frames as that will be the worker ID and an empty
frame added by the backend ROUTER.

But here I have a new question.

If I want to improve my threadpool to have better handling of queuing of
the workers, also improved reliability by bringing in heartbeats, I will
probably need to bring in the PPP. But the example in the guide seems to be
having the proxy, ppqueue, pushing tasks to the workers. If I want to have
the queue in the proxy, then I should have the workers "pulling" tasks. In
the earlier part of the guide, it says this is achieved by having worker
send a "READY" message to proxy and then proxy should reply with the task.
So the core of my question here is, the worker can not know how long the
task queue is in the proxy so it's possible queue is empty. In this case
what should the proxy do? My guess is not replying to worker but do all the
other things so that the worker should keep polling and that will be most
efficient, right?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20231113/74747b06/attachment.htm>

More information about the zeromq-dev mailing list