[zeromq-dev] Malamute broker project

Kenneth Adam Miller kennethadammiller at gmail.com
Mon Mar 2 17:30:21 CET 2015


Oh you mean with mlm_client_set_worker! Do I do set_worker on each side
with different service names? How does a client get a specific service?

On Mon, Mar 2, 2015 at 11:26 AM, Kenneth Adam Miller <
kennethadammiller at gmail.com> wrote:

> Service semantics? I don't know what those are...
> I read what tutorials I think that there are. I have some questions about
> the how things are forwarded-I want only one to one pairing... I'm not sure
> if what I'm doing is setting up for publishing and subscriptions. There was
> a lot of talk about some of the other features in the malamute
> manual/whitepaper, and it's kind of confusing. Basically, I just want FCFS
> exchange of information for mutually requiring parties.
>
> On Mon, Mar 2, 2015 at 4:13 AM, Pieter Hintjens <ph at imatix.com> wrote:
>
>> The simplest way to make a lookup service is using the service
>> semantics, and the lookup service can talk to the broker over inproc
>> or tcp as it wants (it could be a thread in the same process, or a
>> separate process).
>>
>> On Sun, Mar 1, 2015 at 9:00 PM, Kenneth Adam Miller
>> <kennethadammiller at gmail.com> wrote:
>> > So, in order to manage a mutual exchange of address between two
>> concurrent
>> > parties, I thought that on each side I would have a producer produce to
>> a
>> > topic that the opposite side was subscribed to. That means that each
>> side is
>> > both a producer and a consumer.
>> >
>> > I have the two entities running in parallel. The front end client
>> connects
>> > to the malamute broker, and subscribes to the backendEndpoints topic,
>> and
>> > then producing it's endpoint to the frontendEndpoints topic.
>> >
>> > The opposite side does the same thing, with the back end subscribing to
>> the
>> > frontendEndpoints and producing to backendEndpoints.
>> >
>> >
>> > The problem is that if the front end and back end are in their own
>> threads
>> > then only the thread that completes the mlm_set_producer and
>> > mlm_set_consumer call proceed. The one that didn't make it that far will
>> > hang at that mlm_set_x pair point...
>> >
>> > code:
>> >
>> >   std::cout << "connectToFrontEnd" << std::endl;
>> >   mlm_client_t *frontend_reader = mlm_client_new();
>> >   assert(frontend_reader);
>> >   mlm_client_t *frontend_writer = mlm_client_new();
>> >   assert(frontend_writer);
>> >   int rc=mlm_client_connect (frontend_reader,  "tcp://127.0.0.1:9999" ,
>> > 1000, "reader/secret");
>> >   assert(rc==0);
>> >   rc=mlm_client_connect (frontend_writer,  "tcp://127.0.0.1:9999" ,
>> 1000,
>> > "writer/secret");
>> >   assert(rc==0);
>> >   std::cout << "frontend mlm clients connected" << std::endl;
>> >
>> >   mlm_client_set_consumer(frontend_reader, "backendEndpoints", "*");
>> >   mlm_client_set_producer(frontend_writer, "frontendEndpoints");
>> >   std::cout << "frontend client producers and consumers set" <<
>> std::endl;
>> >
>> >
>> > The code looks exactly* the same for the backend, but with some
>> variable and
>> > other changes.
>> >
>> >   std::cout << "connectToBackEnd" << std::endl;
>> >   mlm_client_t *backend_reader = mlm_client_new();
>> >   assert(backend_reader);
>> >   mlm_client_t *backend_writer = mlm_client_new();
>> >   assert(backend_writer);
>> >   int rc=mlm_client_connect(backend_reader,"tcp://127.0.0.1:9999",
>> 1000,
>> > "reader/secret");
>> >   assert(rc==0);
>> >   rc=mlm_client_connect(backend_writer,"tcp://127.0.0.1:9999", 1000,
>> > "writer/secret");
>> >   assert(rc==0);
>> >   std::cout << "backend mlm clients connected" << std::endl;
>> >
>> >   mlm_client_set_consumer(backend_reader, "frontendEndpoints", "*");
>> >   mlm_client_set_producer(backend_writer, "backendEndpoints");
>> >   std::cout << "backend client producers and consumers set" <<
>> std::endl;
>> >
>> > I only ever will see either "frontend client produces and consumers
>> set" or
>> > "backend client producers and consumers set".
>> >
>> > On Sun, Mar 1, 2015 at 2:00 PM, Pieter Hintjens <ph at imatix.com> wrote:
>> >>
>> >> My assumption is that a broker that's doing a lot of service requests
>> >> won't be showing costs of regular expression matching, compared to the
>> >> workload.
>> >>
>> >>
>> >> On Sun, Mar 1, 2015 at 7:49 PM, Doron Somech <somdoron at gmail.com>
>> wrote:
>> >> >> I did it in actors and then moved it back into the main server as it
>> >> >> was complexity for nothing (at that stage). I'd rather design
>> against
>> >> >> real use than against theory.
>> >> >
>> >> > Don't you worry about the matching performance which will happen on
>> the
>> >> > main
>> >> > thread? Also a usage I can see is to use exact matching (string
>> >> > comparison)
>> >> > over regular expression (I usually use exact matching), this is way I
>> >> > think
>> >> > the plugin model fits the service as well.
>> >> >
>> >> > On Sun, Mar 1, 2015 at 8:09 PM, Pieter Hintjens <ph at imatix.com>
>> wrote:
>> >> >>
>> >> >> On Sun, Mar 1, 2015 at 5:52 PM, Doron Somech <somdoron at gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >> > So I went over the code, really liked it. Very simple.
>> >> >>
>> >> >> Thanks. I like the plugin model, especially neat using CZMQ actors.
>> >> >>
>> >> >> > I have a question regarding services, for each stream you are
>> using a
>> >> >> > dedicate thread (actors) and one thread for managing mailboxes.
>> >> >> > However
>> >> >> > (if
>> >> >> > I understood correctly) for services you are doing the processing
>> >> >> > inside
>> >> >> > the
>> >> >> > server thread, why didn't you use an actor for each service or
>> actor
>> >> >> > to
>> >> >> > manage all services? I think the matching of services can be
>> >> >> > expensive
>> >> >> > and
>> >> >> > block the main thread.
>> >> >>
>> >> >> I did it in actors and then moved it back into the main server as it
>> >> >> was complexity for nothing (at that stage). I'd rather design
>> against
>> >> >> real use than against theory.
>> >> >>
>> >> >> -Pieter
>> >> >> _______________________________________________
>> >> >> zeromq-dev mailing list
>> >> >> zeromq-dev at lists.zeromq.org
>> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > zeromq-dev mailing list
>> >> > zeromq-dev at lists.zeromq.org
>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150302/ad0a1081/attachment.htm>


More information about the zeromq-dev mailing list