[zeromq-dev] How fast inproc socket REQ/DEALER create/destroy lifecycle?
Artem Vysochyn
artem.vysochyn at gmail.com
Fri Sep 20 19:00:56 CEST 2013
Ok. Understood. Thanks for help Pieter.
2013/9/20 Pieter Hintjens <ph at imatix.com>
> Creating new sockets over and over isn't a perfect answer. It's a
> terrible answer. The right answer is to keep sockets connected and put
> the semantics of your operations into the message, where they belong.
> We have dozens of examples in the Guide that show how to do this.
>
> My advice is to work through the Guide examples (don't just read them)
> until the end of Ch4.
>
> Then, you may better understand how to construct the your operations
> as multipart messages on a single socket, rather than trying to avoid
> this by creating new sockets over and over. Your design is mixing
> layers in a way that will not work, and goes against the basic zen of
> how to use ZeroMQ.
>
> Hope that helps.
>
> -Pieter
>
>
> On Fri, Sep 20, 2013 at 6:20 PM, Artem Vysochyn
> <artem.vysochyn at gmail.com> wrote:
> > I' afraid I use ROUTER/DEALER correctly.
> >
> > Let me make it more clear.
> >
> > What is needed -- differentiate operations emitted by client side.
> Speaking
> > in terms of zmq -- I need different socket identity per "operation".
> > The operation in turn may be "one request / one reply", or "N requests
> / M
> > replies".
> >
> > I feel like I ca't re-use same DEALER for several operations. I.e. can't
> go
> > with DEALER per thread. Simply imagine the case:
> >
> > You doing simple request-reply via DEALER and you timeouts:
> >
> > send a message: [thread-0][dealer-0][send]: sent()
> > message(correlation#0)
> > recv reply: [thread-0][dealer-0][recv]: recv() ...
> > timedout on recv(): [thread-0][dealer-0][recv]: timedout doing recv().
> >
> > This is fine to not receive reply in specified timeout, still DEALER can
> be
> > used for further operations, and now we want to issue batch-of-requests
> and
> > get batch-of-replies:
> >
> > send first message: [thread-0][dealer-0][send]: sent()
> > message(correlation#1)
> > send second message: [thread-0][dealer-0][send]: sent()
> > message(correlation#2)
> > send third message: [thread-0][dealer-0][send]: sent()
> > message(correlation#3)
> > recv first time: [thread-0][dealer-0][recv]: recv() ...
> > got reply recv(): [thread-0][dealer-0][recv]: recv()
> > message(correlation#0)
> >
> > Here's real problem -- I have got reply message(correlation#0) which
> has
> > been from previous "operation".
> > In second operation I sent correlation#1, correlation#2, correlation#3
> so I
> > expect them back, and not the correlation#0.
> >
> > Now it's more clear?
> >
> > So having new DEALER per operation (instead per thread) would perfectly
> be
> > a way to go, because new DEALER is essentially new identity. That would
> be
> > great. Because ROUTER would be able to reply to "operation" identified
> by
> > identity.
> >
> > But the thing is I can't create zmq socket per operation because this is
> > exhausting OS from file descriptors... 8(((((( That's why I'm here and
> > asking for help.
> >
> >
> > 2013/9/20 Pieter Hintjens <ph at imatix.com>
> >>
> >> I guess the point about router-dealer is the intelligence sits on the
> >> router side, not the dealer side. One router, many dealers. Perhaps
> >> you are using these the wrong way around?
> >>
> >> On Fri, Sep 20, 2013 at 5:45 PM, Artem Vysochyn
> >> <artem.vysochyn at gmail.com> wrote:
> >> > Appreciate for prompt response, Pieter.
> >> >
> >> > Yes, I use this classic pattern: DEALER on client / ROUTER on server,
> >> > exactly. The question still remains: how to make correlation between
> >> > requests and replies on client side if I use DEALER. What is on the
> >> > surface -- create new DEALER upon every operation and send messages
> over
> >> > it,
> >> > block on getting replies, and after that -- dispose DEALER.
> >> > This approach would allow me to send new socket identity (since I
> create
> >> > new
> >> > DEALER) upon every operation, so ROUTER side can use the newly
> created
> >> > identity and reply.
> >> >
> >> >
> >> >
> >> > 2013/9/20 Pieter Hintjens <ph at imatix.com>
> >> >>
> >> >> The classic pattern is a router server that tracks client identities
> >> >> and state per dealer client.
> >> >>
> >> >> On Fri, Sep 20, 2013 at 5:21 PM, Artem Vysochyn
> >> >> <artem.vysochyn at gmail.com> wrote:
> >> >> > Have a next problem:
> >> >> >
> >> >> > I'm working on generic RPC java library on the jzmq/zmq base. At
> the
> >> >> > core
> >> >> > will be following arch:
> >> >> > clients own inproc DEALER/REQ and send to local inproc
> >> >> > ROUTER-DEALER.
> >> >> > The
> >> >> > latter DEALER side will be tcp, and it will propagate messages
> >> >> > further
> >> >> > to
> >> >> > the services which are tcp ROUTER-s somewhere. Pretty classical:
> >> >> > N-inproc-clients talking to M-tcp-ROUTER-s on the network. Here's
> >> >> > pastebin'
> >> >> > diagram: http://pastebin.com/kR4y1Fhx
> >> >> >
> >> >> > The matter in that I want some synchronous nature on client side.
> >> >> > Clients
> >> >> > may emit one-request and should wait for one-reply, or clients may
> >> >> > emit
> >> >> > batch-of-requests and should wait for batch-of-replies. Formally
> like
> >> >> > this:
> >> >> > req_i <--> rep_i and [req_i, ..., req_n] <--> [rep_i,
> >> >> > ...,
> >> >> > rep_m].
> >> >> > I found that DEALER is almost perfect for this. But here's a
> problem:
> >> >> > how
> >> >> > to track correlation between what I sent and what received? In REQ
> >> >> > socket
> >> >> > this is done via some internal state machine, but in DEALER I'm
> free
> >> >> > to
> >> >> > keep
> >> >> > sending and keep receiving w/o knowing correlation between requests
> >> >> > and
> >> >> > replies. So how to do this correlation? Write some code on client
> >> >> > side
> >> >> > which
> >> >> > would keep a list of sent message' ids?
> >> >> >
> >> >> > Another question related to what I described above -- how to
> achieve
> >> >> > this
> >> >> > correlation w/o writing this custom code on client side? I
> >> >> > really-really
> >> >> > don't want to write it. And keeping in mind that, can be the
> >> >> > following
> >> >> > approach as a solution: create new inproc socket upon client
> >> >> > operation,
> >> >> > and
> >> >> > upon completion dispose socket? I.e. one inproc socket per
> operation
> >> >> > (opposite to one per thread). And since this will be an inproc
> socket
> >> >> > (the
> >> >> > guide says it's not real tcp and it's cheap), so can I follow this
> >> >> > approach:
> >> >> > "inproc socket per operation"?
> >> >> >
> >> >> >
> >> >> > Thank you in advance.
> >> >> >
> >> >> > _______________________________________________
> >> >> > zeromq-dev mailing list
> >> >> > zeromq-dev at lists.zeromq.org
> >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >> >
> >> >> _______________________________________________
> >> >> zeromq-dev mailing list
> >> >> zeromq-dev at lists.zeromq.org
> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130920/e5b63ed5/attachment.htm>
More information about the zeromq-dev
mailing list