[zeromq-dev] Kivix: Beacon like service discovery

[Pieter Hintjens]

One thing you could look at is how we did distributed log collection
in Zyre. Any node can be a log collector. When nodes discover each
other, and interconnect, they indicate if they collect logs, and then
their peers connect back, over ZeroMQ, and stream log events to them.

-Pieter

On Wed, Dec 18, 2013 at 8:27 PM, Robert Gallas <gallas.robert at gmail.com> wrote:
> Thanks Pieter,
>
> For PoC purpose only node discovery is implemented. ZeoMQ (JeroMQ) will be
> used as core node interconnection protocol. Other protocols are present just
> as PoC of hexagonal application design possibilities.
>
> So far I can live with unreliable Beacon style heartbeat for monitoring
> purposes.
>
> Node management is more interesting topic. To be honest I have not very
> clear vision yet. Kivix is meant to be set of autonomous nodes. But nodes
> form functional topology only if they cooperate with each other. Autonomous
> nodes design is therefore in direct contradiction to cooperating nodes. In
> traditional style of system management there are agents installed on every
> node where we need to support manageability. There are three basic types of
> node functionality failures:
>
> 1. managed functionality fails, agent runs and we can contact the agent
> 2. management agent fails
> 3. failure at network level
>
> Only in first scenario management agent can help to resolve failure
> situation. And there is of course topic of who discovers failed node.
> Basically it's the same situation as with central broker vs brokerless
> architecture. Having central management console is something I would like to
> avoid.
>
> One possible way of solving node manageability is to have functionality
> requesting node to report failure. Then along with listening to service urn
> to show up on network, node can start to broadcast resource starvation
> message. Then only really required functionality is reported to be failed.
> Basically there is no need for urn:com.example.kivix.mathapp to be reported
> as failed if there is no node requesting that functionality.
>
> Then there is question how to respond to node failure. There can be latent
> nodes on network which can implement more than one capability. Then in case
> of resource starvation broadcast message, node can takeover the
> responsibility and start to provide required capability. This kind of
> behaviour can be then used in self-load-balancing nodes. If there is too
> much messages waiting in dealer queue, dealer can broadcast resource
> starvation message and if there is possibility, some nodes can switch to
> worker mode with required capability.
>
> But again there are categories of applications where there are very
> different requirements about level of manageability.
>
> At this time I'm not decided which approach to begin with. Kivix is only
> sideproduct of another application. I'm implementing features only if
> requirement is driven by that application or if there is some interesting
> idea like Beacon worth to try out as PoC.
>
> If you can point me to some resource where this topic is discussed I would
> be glad. I'd like to avoid reinventig the wheel.
>
> Robert
>
>
> 2013/12/18 Pieter Hintjens <ph at imatix.com>
>>
>> Hi Robert,
>>
>> This is neat! Have you used ZeroMQ for instance for monitoring and
>> managing nodes?
>>
>> -Pieter
>>
>> On Tue, Dec 17, 2013 at 10:24 PM, Robert Gallas <gallas.robert at gmail.com>
>> wrote:
>> > Hello,
>> >
>> > I'd like to announce simple beacon implementation in Kivix framework.
>> > Inspiration comes from beacon implementation in czmq. So far as PoC
>> > only.
>> > There is no API or protocol similarities implemented yet. Example is
>> > given
>> > at http://gabert.github.io/index.html?art=exno5_6
>> >
>> > Many Thanks
>> > Robert
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>