[zeromq-dev] which pattern to use

Thu Nov 8 07:04:13 CET 2012

Hi,

I think a disadvantage to the pipeline is that  push and pull can only work
with each other and so I can't have a broker in the middle to distribute to
the following nodes. 

This means that the client needs to push to the workers/service in one
socket for each group of workers/services.

And also I can't have a broker in the middle for dynamic configuration.

Is this right?

Thanks 

From: Michal Singer [mailto:Michal at leadspace.com] 
Sent: Thursday, November 08, 2012 7:47 AM
To: 'ZeroMQ development list'
Subject: RE: [zeromq-dev] which pattern to use

Hi,

I read this thoroughly. I implemented the client part which receives
messages in round robin as was suggested by this thread.

However, I am still wondering if this is the right solution or is the
push-pull(pipeline) right. Maybe you can help me with this.

Each pattern is not exactly what I need.

The pipeline is only downstream but if I do pipeline in both directions that
I would get what I need (each worker will have two sockets, one push and one
pull).

And the Majordomo can be used but basically this is a request-reply pattern
when I don't really care for two points to request-reply together since each
side can send to another node.

(I also still need to change this pattern to support  sending messages to
workers even if they didn't complete their job since this is what I need. If
all workers are working, they can still get more requests - which is more
like the push-pull pattern)

Another small question: Is there anywhere I can see good examples with all
reliability considerations on the pipeline, like there is on request-reply
patterns on chapter 3?

thanks

From: zeromq-dev-bounces at lists.zeromq.org
[mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Monday, November 05, 2012 9:17 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] which pattern to use

I'm rewriting Chapter 3 of the Guide, which is a bit messy. But the concepts
are there and you should learn them, then this problem will be trivial for
you.

-Pieter

On Mon, Nov 5, 2012 at 4:47 PM, Charles Remes <lists at chuckremes.com> wrote:

The first graphic is correct.

You need to run the code and dive into it a bit more to understand it. You
are trying to do something that isn't very complicated, but your lack of
experience and comfort with the library is making it seem harder than it
should. Experiment a bit. There's no substitute. Once you grok it better, my
last answer will make more sense.

cr

On Nov 5, 2012, at 9:35 AM, Michal Singer <michal at leadspace.com> wrote:

Do you mean that I should change in the broker code (mdbroker- in java
implementation) the wait to send a response from worker (method
processWorker handle of MDP.W_REPLY)

to dispatch to one of the clients like as defined for the workers?

Does it mean that I should use the same pattern:

<image001.png>

Theoretically I would prefer to use a dealer as the broker in between since
I don't need the ROUTER's knowledge of who to response to. But according to
the man, a DEALER can only talk to a ROUTER,REQ, REP.

Is this correct? Should I use dealer to router to dealer in this case? Or is
this legal?

<image002.png>

Thanks,

From: zeromq-dev-bounces at lists.zeromq.org
[mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Charles Remes
Sent: Monday, November 05, 2012 4:05 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] which pattern to use

Read this wiki page about DEALER and ROUTER sockets:
<http://www.zeromq.org/tutorials:dealer-and-router>
http://www.zeromq.org/tutorials:dealer-and-router

That page should help clarify how you can modify the Majordomo pattern to
accomplish your task. The broker in the middle could easily round-robin (or
whatever scheme you choose) the replies back to the request group.

Right now the majordomo broker assumes a single request will be mapped to a
group of workers. You want the request to also come from a group. In the
current broker examples, the original requestor is looked up so the reply
can be sent back. You would modify that to lookup the group that the request
came from and pick the next available worker to send the reply back to.

Does that make sense?

cr

On Nov 5, 2012, at 7:48 AM, Michal Singer < <mailto:michal at leadspace.com>
michal at leadspace.com> wrote:

Can I do the Majordomo pattern with two sockets, one in each direction to
get the feature of "return to different clients with the answer" ? Or is it
better to use the push-pull?

The push-pull is not explained a lot in the guide so I am not sure how to
implement:

1.       LRU

2.       Service discovery

3.       Reliability

Whereas the Majordomo is explained a lot with all the examples.

Thanks.

From: Michal Singer [mailto:Michal@ <http://leadspace.com> leadspace.com] 
Sent: Monday, November 05, 2012 1:58 PM
To: 'ZeroMQ development list'
Subject: RE: [zeromq-dev] which pattern to use

Thanks for the fast replyJ

The problem with this that it uses ROUTER as broker as this is a
request-reply pattern and I don't want the reply to necessarily return to
the sender but instead to anyone which is listening in the same group of
senders.

From:  <mailto:zeromq-dev-bounces at lists.zeromq.org>
zeromq-dev-bounces at lists.zeromq.org [
<mailto:zeromq-dev-bounces at lists.zeromq.org>
mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Monday, November 05, 2012 12:14 PM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] which pattern to use

You can probably base this off the Majordomo pattern.

On Mon, Nov 5, 2012 at 10:48 AM, Michal Singer <
<mailto:michal at leadspace.com> michal at leadspace.com> wrote:

Hi.

I am new with ZMQ. I am checking it out to use for a distributed system. I
am reading the guide but I can't find an exactly suited pattern for our
needs.

I need a pattern for:

.         a services oriented system: one which consists of groups of
service, let's say group A (A1, A2, A3,.), group B (B1,B2,B3,..), group
C(C1,C2,C3,.)

.         These services are like 'workers'

.         Each service from each group can send a request to one service
from each of the other groups

.         The reply from the destination service can go to any of the other
services from the sending group

Something like this:

A1 (send request to)->B2 (send reply to)->A2 (send request to )->C1 (send
reply to )->A1

It is some sort of a push-pull where any service should have a 'PUSH' and
'PULL' socket.

It is also some kind of a dealer to dealer type of a pattern, though it is
important that the reply can return to any of the sending group.

I got to chapter 6 and couldn't find any good examples of the PUSH-PULL
pattern and it seems like I need one to fully cover all scenarios of
reliability and so on.

What should I do?

Thanks in advance, Michal

_______________________________________________
zeromq-dev mailing list
 <mailto:zeromq-dev at lists.zeromq.org> zeromq-dev at lists.zeromq.org
 <http://lists.zeromq.org/mailman/listinfo/zeromq-dev>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
 <mailto:zeromq-dev at lists.zeromq.org> zeromq-dev at lists.zeromq.org
 <http://lists.zeromq.org/mailman/listinfo/zeromq-dev>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20121108/d8f19275/attachment.htm>