[zeromq-dev] Kivix: Beacon like service discovery

[Robert Gallas]

Pieter, Andrew


Thanks for pointers. I'm not the best C source reader, but beacon was well
written and not that much difficult to understand. I'll try to look around
in Zyre as well.

I have run quickly through http://markburgess.org/BookOfPromises.pdf and it
seems that Kivix is targeting same area. By coincidence it seems that what
Promise Theory calls "promise" in Kivix is implemented as "capability". And
has a form of URN urn:kivix:demo:mathapp.

Bundles of promises seems like failover functionality. Which brinks me to
question why have bundle of promises and not just start more nodes with
same capability / promise. But that is not question for this thread and
probably not for ZeroMQ community.

I'll look at Zyre and will read Promise Theory draft. Hope I'll be able to
come up with simple "just enough" API or else I'd rather drop this failover
feature.

Many thanks again
Robert



2013/12/19 Andrew Hume <andrew at research.att.com>

> this also fits in nicely with promise theory (mark burgess, uni of oslo)
> which gives a different take on how to reason about cooperating nodes.
>
> On Dec 19, 2013, at 3:01 AM, Pieter Hintjens <ph at imatix.com> wrote:
>
> One thing you could look at is how we did distributed log collection
> in Zyre. Any node can be a log collector. When nodes discover each
> other, and interconnect, they indicate if they collect logs, and then
> their peers connect back, over ZeroMQ, and stream log events to them.
>
> -Pieter
>
> On Wed, Dec 18, 2013 at 8:27 PM, Robert Gallas <gallas.robert at gmail.com>
> wrote:
>
> Thanks Pieter,
>
> For PoC purpose only node discovery is implemented. ZeoMQ (JeroMQ) will be
> used as core node interconnection protocol. Other protocols are present
> just
> as PoC of hexagonal application design possibilities.
>
> So far I can live with unreliable Beacon style heartbeat for monitoring
> purposes.
>
> Node management is more interesting topic. To be honest I have not very
> clear vision yet. Kivix is meant to be set of autonomous nodes. But nodes
> form functional topology only if they cooperate with each other. Autonomous
> nodes design is therefore in direct contradiction to cooperating nodes. In
> traditional style of system management there are agents installed on every
> node where we need to support manageability. There are three basic types of
> node functionality failures:
>
> 1. managed functionality fails, agent runs and we can contact the agent
> 2. management agent fails
> 3. failure at network level
>
> Only in first scenario management agent can help to resolve failure
> situation. And there is of course topic of who discovers failed node.
> Basically it's the same situation as with central broker vs brokerless
> architecture. Having central management console is something I would like
> to
> avoid.
>
> One possible way of solving node manageability is to have functionality
> requesting node to report failure. Then along with listening to service urn
> to show up on network, node can start to broadcast resource starvation
> message. Then only really required functionality is reported to be failed.
> Basically there is no need for urn:com.example.kivix.mathapp to be reported
> as failed if there is no node requesting that functionality.
>
> Then there is question how to respond to node failure. There can be latent
> nodes on network which can implement more than one capability. Then in case
> of resource starvation broadcast message, node can takeover the
> responsibility and start to provide required capability. This kind of
> behaviour can be then used in self-load-balancing nodes. If there is too
> much messages waiting in dealer queue, dealer can broadcast resource
> starvation message and if there is possibility, some nodes can switch to
> worker mode with required capability.
>
> But again there are categories of applications where there are very
> different requirements about level of manageability.
>
> At this time I'm not decided which approach to begin with. Kivix is only
> sideproduct of another application. I'm implementing features only if
> requirement is driven by that application or if there is some interesting
> idea like Beacon worth to try out as PoC.
>
> If you can point me to some resource where this topic is discussed I would
> be glad. I'd like to avoid reinventig the wheel.
>
> Robert
>
>
> 2013/12/18 Pieter Hintjens <ph at imatix.com>
>
>
> Hi Robert,
>
> This is neat! Have you used ZeroMQ for instance for monitoring and
> managing nodes?
>
> -Pieter
>
> On Tue, Dec 17, 2013 at 10:24 PM, Robert Gallas <gallas.robert at gmail.com>
> wrote:
>
> Hello,
>
> I'd like to announce simple beacon implementation in Kivix framework.
> Inspiration comes from beacon implementation in czmq. So far as PoC
> only.
> There is no API or protocol similarities implemented yet. Example is
> given
> at http://gabert.github.io/index.html?art=exno5_6
>
> Many Thanks
> Robert
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> -----------------------
> Andrew Hume
> 949-707-1964 (VO and best)
> 732-420-0907 (NJ)
> andrew at research.att.com
>
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131219/a8015795/attachment.htm>