[zeromq-dev] Kivix: Beacon like service discovery

Thu Dec 19 21:19:46 CET 2013

Robert,

You might even look at the Jyre project, which is a (slightly out of
date) Java implementation of the Zyre protocols.

-Pieter

On Thu, Dec 19, 2013 at 9:08 PM, Robert Gallas <gallas.robert at gmail.com> wrote:
> Pieter, Andrew
>
>
> Thanks for pointers. I'm not the best C source reader, but beacon was well
> written and not that much difficult to understand. I'll try to look around
> in Zyre as well.
>
> I have run quickly through http://markburgess.org/BookOfPromises.pdf and it
> seems that Kivix is targeting same area. By coincidence it seems that what
> Promise Theory calls "promise" in Kivix is implemented as "capability". And
> has a form of URN urn:kivix:demo:mathapp.
>
> Bundles of promises seems like failover functionality. Which brinks me to
> question why have bundle of promises and not just start more nodes with same
> capability / promise. But that is not question for this thread and probably
> not for ZeroMQ community.
>
> I'll look at Zyre and will read Promise Theory draft. Hope I'll be able to
> come up with simple "just enough" API or else I'd rather drop this failover
> feature.
>
> Many thanks again
> Robert
>
>
>
> 2013/12/19 Andrew Hume <andrew at research.att.com>
>>
>> this also fits in nicely with promise theory (mark burgess, uni of oslo)
>> which gives a different take on how to reason about cooperating nodes.
>>
>> On Dec 19, 2013, at 3:01 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>> One thing you could look at is how we did distributed log collection
>> in Zyre. Any node can be a log collector. When nodes discover each
>> other, and interconnect, they indicate if they collect logs, and then
>> their peers connect back, over ZeroMQ, and stream log events to them.
>>
>> -Pieter
>>
>> On Wed, Dec 18, 2013 at 8:27 PM, Robert Gallas <gallas.robert at gmail.com>
>> wrote:
>>
>> Thanks Pieter,
>>
>> For PoC purpose only node discovery is implemented. ZeoMQ (JeroMQ) will be
>> used as core node interconnection protocol. Other protocols are present
>> just
>> as PoC of hexagonal application design possibilities.
>>
>> So far I can live with unreliable Beacon style heartbeat for monitoring
>> purposes.
>>
>> Node management is more interesting topic. To be honest I have not very
>> clear vision yet. Kivix is meant to be set of autonomous nodes. But nodes
>> form functional topology only if they cooperate with each other.
>> Autonomous
>> nodes design is therefore in direct contradiction to cooperating nodes. In
>> traditional style of system management there are agents installed on every
>> node where we need to support manageability. There are three basic types
>> of
>> node functionality failures:
>>
>> 1. managed functionality fails, agent runs and we can contact the agent
>> 2. management agent fails
>> 3. failure at network level
>>
>> Only in first scenario management agent can help to resolve failure
>> situation. And there is of course topic of who discovers failed node.
>> Basically it's the same situation as with central broker vs brokerless
>> architecture. Having central management console is something I would like
>> to
>> avoid.
>>
>> One possible way of solving node manageability is to have functionality
>> requesting node to report failure. Then along with listening to service
>> urn
>> to show up on network, node can start to broadcast resource starvation
>> message. Then only really required functionality is reported to be failed.
>> Basically there is no need for urn:com.example.kivix.mathapp to be
>> reported
>> as failed if there is no node requesting that functionality.
>>
>> Then there is question how to respond to node failure. There can be latent
>> nodes on network which can implement more than one capability. Then in
>> case
>> of resource starvation broadcast message, node can takeover the
>> responsibility and start to provide required capability. This kind of
>> behaviour can be then used in self-load-balancing nodes. If there is too
>> much messages waiting in dealer queue, dealer can broadcast resource
>> starvation message and if there is possibility, some nodes can switch to
>> worker mode with required capability.
>>
>> But again there are categories of applications where there are very
>> different requirements about level of manageability.
>>
>> At this time I'm not decided which approach to begin with. Kivix is only
>> sideproduct of another application. I'm implementing features only if
>> requirement is driven by that application or if there is some interesting
>> idea like Beacon worth to try out as PoC.
>>
>> If you can point me to some resource where this topic is discussed I would
>> be glad. I'd like to avoid reinventig the wheel.
>>
>> Robert
>>
>>
>> 2013/12/18 Pieter Hintjens <ph at imatix.com>
>>
>>
>> Hi Robert,
>>
>> This is neat! Have you used ZeroMQ for instance for monitoring and
>> managing nodes?
>>
>> -Pieter
>>
>> On Tue, Dec 17, 2013 at 10:24 PM, Robert Gallas <gallas.robert at gmail.com>
>> wrote:
>>
>> Hello,
>>
>> I'd like to announce simple beacon implementation in Kivix framework.
>> Inspiration comes from beacon implementation in czmq. So far as PoC
>> only.
>> There is no API or protocol similarities implemented yet. Example is
>> given
>> at http://gabert.github.io/index.html?art=exno5_6
>>
>> Many Thanks
>> Robert
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>> -----------------------
>> Andrew Hume
>> 949-707-1964 (VO and best)
>> 732-420-0907 (NJ)
>> andrew at research.att.com
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>