[zeromq-dev] Alternative efficiency design of lbbroker

[Kenneth Adam Miller]

No, I don't want dealer router combination.

Basically I have two mutually requiring parties that are each doing a lot
of processing and work, distributed over a lot of machines. They all run
concurrently, so there has to be a reliable way to facilitate an exchange
of a work task from a producer to one that is ready to receive, but
sometimes the workers also need to send messages back to the work producer
to inform them of something.

So what I've done is I've made the lb broker example, only except currently
I've got it modified so that it:
1) facilitates a pairing
2) ferries message from *each* side to the other

Currently, the manual offers work-work producer example with lb broker that
facilitates an exchange first and then ferries a message from the work
producer to the worker. So it's just a little bit different.

So the way I understand it is, the load balancing broker in the example is
somewhat of a bottle neck. What I was thinking though is, instead of
receiving the opposite side's address on each side like I do in order that
I can send that *back through the broker to be routed to the other side*,
why not immediately use the address in order to make a send on a local
router socket send to the correct destination on the side of the worker.

In this way, the broker actually facilitates connection setup (no
sock->connect, just direct relation pairing). In the previous example, if
this works then individual 1 & 2 first talk to broker to find out what they
need to know in order to talk directly to one another. So this is better
use of resources; if there are lots of data being sent back and forth
through the broker the manual example, you could potentially add too many
workers and work producers on each side; then workers starve (somewhat) and
managers await them (somewhat).

What I didn't know was, if you have the address on each side first, and you
have a router object in the worker/work producer, and you take the address
received from the opposite side of the broker and empty delimit it on the
local router socket in order for it to go directly to individual 1/2, will
that work in zmq?

On Mon, Jan 12, 2015 at 10:28 AM, Benjamin <benjamin.l.cordes at gmail.com>
wrote:

> >>  But I want to do simply a request to the broker and receive the
> information I need in order to send directly to the other entity.
>
> I don't follow the question. You're talking about name-resolution?
> Chain of servers is how DNS works for instance (router => X =>
> router). Some server knows the answer to a lookup, so one node can
> request information from a second node, which the client did not know
> about. Perhaps you're looking for some dealer/router combination
> described in the ZMQ guide.
>
> On Mon, Jan 12, 2015 at 4:09 PM, Pieter Hintjens <ph at imatix.com> wrote:
> > No apologies, I wasn't aiming to be critical. It's just that the
> > thinking process happens best by making working code and then
> > improving it. If you have specific questions others can help. With
> > larger architectural questions, you won't often get answers here.
> >
> > On Mon, Jan 12, 2015 at 3:43 PM, Kenneth Adam Miller
> > <kennethadammiller at gmail.com> wrote:
> >> Oh I apologize. I was just asking if what I was thinking of was possible
> >> with zeromq.
> >>
> >> On Mon, Jan 12, 2015 at 4:29 AM, Pieter Hintjens <ph at imatix.com> wrote:
> >>>
> >>> > In any case, does what I am thinking about work?
> >>>
> >>> It's usually a good idea to make small pieces, and grow your
> >>> understanding of your problem and alternative solutions like that.
> >>> No-one here can do that for you.
> >>>
> >>> On Mon, Jan 12, 2015 at 7:31 AM, Kenneth Adam Miller
> >>> <kennethadammiller at gmail.com> wrote:
> >>> > Instead of routing all information through the broker and requiring
> an
> >>> > intermediary hop, I'd like to consider an approach where the address
> >>> > information that the req socket parses out on the side that first
> sends
> >>> > ready is used in order to manage a simple mutual connection
> facilitator
> >>> > in
> >>> > ZMQ...
> >>> >
> >>> > Just like with load balancing broker from the manual, I have two
> parties
> >>> > that mutually require one another. But I want to do simply a request
> to
> >>> > the
> >>> > broker and receive the information I need in order to send directly
> to
> >>> > the
> >>> > other entity.
> >>> >
> >>> > With router sockets, I get the address in a empty delimited set of
> >>> > message
> >>> > sending convention. What I don't know is if I add a router socket to
> >>> > each
> >>> > mutually requiring party, and -even if party A is not connected even
> to
> >>> > party B- use the address information attained from the broker to
> send on
> >>> > party A's router socket to party B and vice versa. Is this the best
> way
> >>> > to
> >>> > do this? I don't want to manage a set of connected sockets.
> >>> >
> >>> > I realize that party A may have individual 1 and party B an
> individual 2
> >>> > where 1 & 2 repeatedly make connection over broker-I want to keep
> using
> >>> > broker anyway, because broker allows each party to know when an
> >>> > individual
> >>> > is ready in each. So the 1& 2 scenario is coincidental.
> >>> >
> >>> > But what I don't think is a good design idea is having a socket that
> >>> > makes
> >>> > connect/close calls between each iteration to the broker or the idea
> of
> >>> > having each endpoint know it's hostname or something (possibly I have
> >>> > premature misgivings).
> >>> >
> >>> > In any case, does what I am thinking about work? Reading the target
> >>> > address
> >>> > information gained from a router object and feeding it to a
> completely
> >>> > different router object (where individual 1 may not have ever had his
> >>> > router
> >>> > connected to individual 2 before)?
> >>> >
> >>> > _______________________________________________
> >>> > zeromq-dev mailing list
> >>> > zeromq-dev at lists.zeromq.org
> >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>> >
> >>> _______________________________________________
> >>> zeromq-dev mailing list
> >>> zeromq-dev at lists.zeromq.org
> >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150112/3587c9cf/attachment.htm>