[zeromq-dev] IoT pattern work in progress

[Pieter Hintjens]

Have a look at Zyre, it deals with a lot of this.

On Thu, Jun 16, 2016 at 2:35 AM, Douglas Petican <dpetican at gmail.com> wrote:
> I've been working on req-broker-rep messaging pattern for an application and
> so far its working great.   Thanks for the help. Right now it does what I
> need it to do for the current application, but I've realized there is a lot
> more potential if I extend this pattern.  By the way I'm running zmq on
> raspberry pi3's.  The broker enforces a 1:1 relationship between client and
> worker.
>
> My broker has a special command that allows it to create the requested
> service.  It responds okay when the service is created. The client can then
> send messages to the newly apwned service. This means that the broker could
> be sitting on a remote machine with no application processes running, get a
> message to start a service, then pass requests along to that service.  The
> broker enforces a 1:1 relationship between client and worker.  The last
> request will usually be for the service to self-terminate.  But doesn't have
> to be.  Its worthwhile to note that in my applications none of the modules
> store state.  So I think that makes them naturally restful.  However,
> there's nothing to prevent a service from always running and storing state
> if that's whats required.
>
> What if each client was also a worker?  Then I would have:
> req/rep---broker---rep/req.  Nice.   That's the reason for the one-one
> relationship and the service self-terminating.  The calling client (in my
> application) is itself a worker to some other client.  So I'm creating a
> dynamic, distributed, on demand pipe-lining system.  This is pretty tasty
> sounding to me.  I've already done it, but I want to go further.  Lets make
> it scalable.
>
> How do you discover services that are not running?  The broker could be
> pre-populated with information about the services it can start.  Clients
> could query the broker about available services. Wait... what about if a new
> service popped into existence only long enough to register itself with the
> broker then die and wait to be spawned again on command.  Cool.  Hidden
> services?  Sure, just pass the name to the broker and it will happily start
> it. That way the modules in question are not public knowledge to other
> modules.
>
> Where are these brokers, and how do I find them?  Brokers are always running
> (well maybe not) so they should broadcast messages at regular intervals to
> other brokers notifying them of their existence on the network.  Then the
> other brokers who receive this message will broadcast back their location
> and the services they know about.   A client need only know of the existence
> of one broker to get in on the game.  So really each machine with clients
> could run its own broker on a known socket for the express reason of
> discovering other unknown brokers even though it may not have any services
> available locally.  To cut down on traffic you could have gateway
> super-brokers who sit at network access points and collect information about
> brokers on the upstream side.
>
> I'm not enforcing any specific way for clients and workers to communicate
> with respect to their particular application except that the only
> requirement is for two parts in a multi-part message.  The first part is the
> command which could be a command to the broker or a worker.  And the second
> is the data (if any) that corresponds to the command.  Like filename or
> module name or any kind of data that the worker needs to do its thing.  If a
> command to a worker is something like read, then the data in the request
> could be blank but be filled in when the response comes back. The first part
> which was a command field in the request becomes a status field in the reply
> (ex okay, good, error etc). In the case of error, the data part would be the
> error message. Its up to the application to decide what the commands should
> be (apart from broker commands) and what the data part of the message
> actually means.  Just adhere to the contract that all requests must receive
> a reply.
>
> Sorry this was long winded.  Maybe it was just a way to get the thoughts out
> of my head and onto virtual paper.  Anybody have any thoughts on this?  ZMQ
> really is radioactive eh?  I will publish this part of application to a
> public github soon.  There's nothing proprietary in it.  Our secret sauce is
> weird serial protocols that emulate tape cassette and bubble memory readers
> that replaced punch tape machines.  Extending it this way is an after work
> project for now.  I have to work on the secret sauces.
>
>
> --
> <-Douglas Petican->
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev