[zeromq-dev] Can't make router router work will use Malamute
Douglas Petican
dpetican at gmail.com
Sun Jun 12 21:29:42 CEST 2016
I suppose that eventually I may use malamute, but I'm doing this on an
embedded system so I want to keep the overhead to a minimum and I want
to understand how zmq works so I'm not just implementing someone's black
box solution.
The key point I think is that this piece of code does not do what I
would expect/want it to do in a router-dealer (my original) broker:
message[0] = backend_socket_id
print ("Broker, outgoing message to backend: %s" % message)
backend.send_multipart(message)
Where backend_socket_id equals the pid of the worker. zmq does not
always send the message to the specified socket id even though I have an
already spawned worker with the correct identity set. zmq is always
trying to load balance and seems to ignore the message[0] field. This
is counter-intuitive to me.
Reading the guide extremely closely, I now understand why this doesn't
work. The router (frontend) socket uses the address while the dealer
(backend) does/may not. This explains why the frontend always works and
the first answer I got what to use a router-router broker. However, I
couldn't make this work because the router-rep socket pair is not valid
for the worker connection.
So my only valid choice for the backend worker is to use a router-router
pair. I did this and it now works the way I would want/expect. I have
some more I want to discuss about this concept and the pattern I've
created, but I will update my code on pastebin first and start a new
discussion.
Thanks to everyone who responded. It all helped to get me to my solution.
<-Douglas Petican->
On 2016-06-11 2:26 PM, Michal Vyskocil wrote:
>
> Btw if you problem is how to offload some task to thread, you might
> want to take a look on zactor class from czmq. It makes spawning the
> thread and communicating with it really easy.
>
> Dne 11. 6. 2016 8:08 PM napsal uživatel "Michal Vyskocil"
> <michal.vyskocil at gmail.com <mailto:michal.vyskocil at gmail.com>>:
>
> Hi,
>
> Did you take a look on a SERVICE pattern (mlm_client_sendfor)? The
> proposal smells like that to me. In this pattern workers are not
> (usually) spawned by client making request. The pool is usually
> created before.
>
> A1 yes, uuid can be used. We have used prosess id and thread id to
> create unique mailbox addresses. Whatever will work for you is fine.
>
> A2 mlm_client_recv is blocking call, so you must use zpoller_t
> from czmq (or anything which will work for zsock_t)
>
> Q3 malamute does maintain the state in memory, so it will always
> start fresh. However I'm a bit confused by mentioning crash
> together with different signals, so I might misunderstood the
> question.
>
> If this is about stopping sequences, you can run malamute as a
> thread, so it will have the same lifecycle as your system.
>
> Dne 11. 6. 2016 5:12 PM napsal uživatel "Douglas Petican"
> <dpetican at gmail.com <mailto:dpetican at gmail.com>>:
>
> Thanks. Malamute looks really cool. Nice doggy. And I don't
> have to keep trying to reinvent the wheel ;) I also like the
> idea of persistent storage. Could be useful for storing
> logfiles and data coming from a datalogger attached to the system.
>
> So, I've read the whitepaper and it seems like direct
> messaging is what I want to do. This is how I think I should
> proceed. Note that all my "servers/workers" process only one
> request and respond with one reply. Either success and data,
> or error and error message, then terminate normally.
>
> 1) Client spawns a worker and passes the worker its identity
> as a parameter
>
> 2) Client sends a direct message to the broker using its own
> identity addressed to the worker identity
>
> 3) When worker spins up it checks its mailbox and receives the
> message (request)
>
> 4) Worker does its work, sends a message (response) addressed
> to the client and terminates
>
> 5) Client receives the message (response) and does what it has
> to do
>
> Q1) Should I be using something like uuids so that mailboxes
> have unique names?
>
> Q2) Unlike req-rep which blocks will I have to keep polling
> the broker to see if there is a message?
>
> Q3) If malamute dies can I request that when it starts up
> again say using a watchdog like monit that it starts fresh.
> This is because the INT and TERM signals are processed by all
> my clients and workers as well. So if one process dies. They
> all die.
>
>
> <-Douglas Petican->
>
> On 2016-06-10 8:32 AM, Kevin Sapper wrote:
>> Sorry messed up the link: https://github.com/zeromq/malamute
>>
>> 2016-06-10 14:32 GMT+02:00 Kevin Sapper
>> <kevinsapper88 at gmail.com <mailto:kevinsapper88 at gmail.com>>:
>>
>> Hi Douglas,
>>
>> have a look a malamute
>> (https://github.com/zeromq/malamulte) which should work
>> for your use case.
>>
>> //Kevin
>>
>> 2016-06-08 16:07 GMT+02:00 Douglas Petican
>> <dpetican at gmail.com <mailto:dpetican at gmail.com>>:
>>
>> I want to enforce a one to one relationship between
>> client and dynamically spawned worker. I asked
>> before on the list and was told to use a router
>> router broker. However, I can't get my code to work
>> in anyway with a router on both the frontend and
>> backend of the broker. Nothing happens. It only
>> works (almost) with a dealer backend.
>>
>> The first client connects okay and the worker is
>> spawned dynamically. When the second client connects
>> a new worker is spawned and the second client and
>> worker talk to each other exclusively, but the first
>> client talks to both workers which is not what I want.
>>
>> My broker code is here: http://pastebin.com/eKVQ7VwR
>>
>> The client/worker code is here:
>> http://pastebin.com/5tU9BZRn
>>
>> --
>> <-Douglas Petican->
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> <mailto:zeromq-dev at lists.zeromq.org>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org <mailto:zeromq-dev at lists.zeromq.org>
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160612/37b3ba7a/attachment.htm>
More information about the zeromq-dev
mailing list