[zeromq-dev] adaptable net

[Pieter Hintjens]

Too many questions at once :)

- Async request-reply already works, using service requests, and
mailboxes for replies.

- For reliability, there is already a "tracker" field in messages. My
intention was/is that recipients can send CONFIRMs back for specific
messages. These flow asynchronously, and allow senders to retry as
needed. See mlm_proto.xml.

- If you need additional reliability layers I'd suggest writing this
in an application on top of Malamute rather than in the
protocol/server/client itself. Since you can embed mlm_server as a
thread, and talk to the broker over inproc:, you can embed routing
applications in the same process.

- As always, try to make small, incremental changes that can be proven
independently. Be wary, even paranoid, of your own ability to make
large-scale architectures. If you want to make large experiments, you
can certainly do this, yet it will always be hard to bring such
changes back into master.

-Pieter

On Sat, Apr 9, 2016 at 1:57 AM, Matjaž Ostroveršnik
<matjaz.ostroversnik at gmail.com> wrote:
> Hi all,
>
> In last days I am studying the mlm server architecture. Nice and clean
> design. Congrats to the architects.
>
> My objective is set to prepare adaptable net of nodes with client and worker
> functionality (i.e. service pattern in mlm semantics) that
> can communicate among themselves. It is always client (consumer of the
> results of the service)-worker(provider of the service) communication, but
> nodes have both client and worker functionality.
>
> There is/are one or more brokers between clients and workers. Communication
> goes client-broker-worker-broker-client. Brokers should be as simple as
> possible.
>
> A. Broker:
>
> There are two or more brokers in the net.
> Brokers hold no vital information. If one broker goes down (regular /
> irregular shutdown) net must readapt. No payload is lost.
> broker is go between clients and workers from the security reasons (clients
> must not have direct access to the workers and vice versa)
>
> brokers allow only connection of registered clients/workers
>
> brokers distribute work between workers offering the same service (load
> balancing)
>
> e.g. round-robin
>
> provide alternative paths between clients and workers.  (resistance to
> failure)
>
> C1->B1->W1->B1-C1 if there are several brokers the same pair C1/W1 can
> communicate via different brokers, assuming that all  workers and all
> clients are connected to all brokers.
>
> source of information about other brokers in the net
>
> i.e. client/worker connects to one predefined broker, periodically it
> distributes the list of other brokers to its workers/clients
> brokers have periodically "quorum" when they exchange information about them
> selves (i.e. list of all active brokers)
> initially each node is configured with at least one broker (others are
> provided dynamically)
> initially each broker is configured with at least one peer broker (is there
> mechanism to make broadcast on WAN?)
>
> B. Clients:
>
> connected to one broker initially, gradually they "see" whole set of brokers
> if one broker fails, client uses different path (via retransmitting the
> message)
> clients distribute work between brokers
>
> C. Workers:
>
> - each worker provides one service type
> - there can be several workers offering the same service
>
> D. Messages:
>
> payload is opaque (size cca 4KB)
> meta-data : sender, receiver, unique-msg id, possible duplicate, original
> message id (for replys), retry cnt
> if they are expired and until there is an option for retry client retries
> them
>
> E. Message exchange:
>
> it is always initiated by the client part of the node and goes to the
> service worker, which replys
> if message is not return in predefined period of time (msg exchange timeout)
> it is retried (predefined number of retries)
> client can select the broker to send the message through, but worker always
> returns message via arriving path(broker)
> message exchange C-B-W-B-C is fast (sub seccond)
>
> Current mlm server is by my opinion platform for this task. Am I too
> optimistic?
> Existing functionality:
>   - A.1-2
>   - A.4 (?) believe so
>   - C.*
>   - E.1 (how to do reply?)
>   - E.3-4
> Already added functionality:
>   - A.3.1 (curve)
>   - D.1  (blobs)
> TODO functionality:
>   - A.5
>   - A.6 (the toughest stuff)
>   - B.1-3
>   - D.2-3
>   - E.2
>
> Questions:
>
> Async request-reply pattern. Is it already supported? Code talks about
> replys, but I am unsure. Hints?
> D.2-3, E.2: Is current message (mlm_msg_t) suitable for expansion (e.g.
> unique-id, original-unique-id, retry-cnt)?Or should I do it in some other
> way?
> A.5: clients / workers should connect to different brokers (e.g. array of
> mlm_client actors )
> A.6: Currently I do not have solution for A.6. Is zgossip the right
> solution?
> B.1-3 - Strictly related to A.6 solution
>
> What is your opinion on this?
> Would you do something in a different way?
> How you would tackle A.6?
>
> Thanks in advance
>
> Matjaž
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>