[zeromq-dev] Kivix: Beacon like service discovery

[Andrew Hume]

this also fits in nicely with promise theory (mark burgess, uni of oslo)
which gives a different take on how to reason about cooperating nodes.

On Dec 19, 2013, at 3:01 AM, Pieter Hintjens <ph at imatix.com> wrote:

> One thing you could look at is how we did distributed log collection
> in Zyre. Any node can be a log collector. When nodes discover each
> other, and interconnect, they indicate if they collect logs, and then
> their peers connect back, over ZeroMQ, and stream log events to them.
> 
> -Pieter
> 
> On Wed, Dec 18, 2013 at 8:27 PM, Robert Gallas <gallas.robert at gmail.com> wrote:
>> Thanks Pieter,
>> 
>> For PoC purpose only node discovery is implemented. ZeoMQ (JeroMQ) will be
>> used as core node interconnection protocol. Other protocols are present just
>> as PoC of hexagonal application design possibilities.
>> 
>> So far I can live with unreliable Beacon style heartbeat for monitoring
>> purposes.
>> 
>> Node management is more interesting topic. To be honest I have not very
>> clear vision yet. Kivix is meant to be set of autonomous nodes. But nodes
>> form functional topology only if they cooperate with each other. Autonomous
>> nodes design is therefore in direct contradiction to cooperating nodes. In
>> traditional style of system management there are agents installed on every
>> node where we need to support manageability. There are three basic types of
>> node functionality failures:
>> 
>> 1. managed functionality fails, agent runs and we can contact the agent
>> 2. management agent fails
>> 3. failure at network level
>> 
>> Only in first scenario management agent can help to resolve failure
>> situation. And there is of course topic of who discovers failed node.
>> Basically it's the same situation as with central broker vs brokerless
>> architecture. Having central management console is something I would like to
>> avoid.
>> 
>> One possible way of solving node manageability is to have functionality
>> requesting node to report failure. Then along with listening to service urn
>> to show up on network, node can start to broadcast resource starvation
>> message. Then only really required functionality is reported to be failed.
>> Basically there is no need for urn:com.example.kivix.mathapp to be reported
>> as failed if there is no node requesting that functionality.
>> 
>> Then there is question how to respond to node failure. There can be latent
>> nodes on network which can implement more than one capability. Then in case
>> of resource starvation broadcast message, node can takeover the
>> responsibility and start to provide required capability. This kind of
>> behaviour can be then used in self-load-balancing nodes. If there is too
>> much messages waiting in dealer queue, dealer can broadcast resource
>> starvation message and if there is possibility, some nodes can switch to
>> worker mode with required capability.
>> 
>> But again there are categories of applications where there are very
>> different requirements about level of manageability.
>> 
>> At this time I'm not decided which approach to begin with. Kivix is only
>> sideproduct of another application. I'm implementing features only if
>> requirement is driven by that application or if there is some interesting
>> idea like Beacon worth to try out as PoC.
>> 
>> If you can point me to some resource where this topic is discussed I would
>> be glad. I'd like to avoid reinventig the wheel.
>> 
>> Robert
>> 
>> 
>> 2013/12/18 Pieter Hintjens <ph at imatix.com>
>>> 
>>> Hi Robert,
>>> 
>>> This is neat! Have you used ZeroMQ for instance for monitoring and
>>> managing nodes?
>>> 
>>> -Pieter
>>> 
>>> On Tue, Dec 17, 2013 at 10:24 PM, Robert Gallas <gallas.robert at gmail.com>
>>> wrote:
>>>> Hello,
>>>> 
>>>> I'd like to announce simple beacon implementation in Kivix framework.
>>>> Inspiration comes from beacon implementation in czmq. So far as PoC
>>>> only.
>>>> There is no API or protocol similarities implemented yet. Example is
>>>> given
>>>> at http://gabert.github.io/index.html?art=exno5_6
>>>> 
>>>> Many Thanks
>>>> Robert
>>>> 
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>> 
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
>> 
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


-----------------------
Andrew Hume
949-707-1964 (VO and best)
732-420-0907 (NJ)
andrew at research.att.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131219/88b0bcfa/attachment.htm>