[zeromq-dev] [pyzmq]: Asynchronous client api design

Yassine Lamgarchal bouceka78 at gmail.com
Mon Mar 9 12:12:01 CET 2015


Hi Kenneth,

Thank you for your quick response !

* I use a DEALER socket because the client must be able to send multiple
commands without
receiving the response:

    - async_result1 = client.command("data1")
    - async_result2 = client.command("data2")
    - .... do other stufs ....
    - result1 = async_result1.get()
    - result2 = async_result2.get()

* Indeed, it looks like a future object :) !

* Yea, in the schema it's a thread which run the event loop and listen for
commands to send to the server, i
think i can easily spawn a process instead of a thread. The consumer simply
connect to the socket, on
which the thread listen on, and send commands.

* Thanks, i realize that a REQ socket is better in this case because a PAIR
will restrict to only one client
connected to the "framework thread".

Q1. Well, for now there is a fixed number of servers known by each client.
When the cluster start it elects a server as
a leader, the command are only sent to that particular server. So for now,
there is no need to load balance the requests.

Q2. Yea, the thing is as a user i don't like when a library create threads
in the background, this could lead to bugs hard
to troubleshoot. But yea with message passing this could be acceptable.

Actually, i thought if i can have the same behavior (then having a single
threaded client application) and letting the user
create the event loop.  If i do so, the client application must be fully
event driven (the client event loop receive events
and send commands) but i don't want this. In my use case i think that
"framework thread" is mandatory, does my reasoning
make sense ? :)


--
Yassine


2015-03-09 3:16 GMT+01:00 Kenneth Adam Miller <kennethadammiller at gmail.com>:

> Replied inline
>
> On Sun, Mar 8, 2015 at 7:55 PM, Yassine Lamgarchal <bouceka78 at gmail.com>
> wrote:
>
>> Hello all o/,
>>
>> I'm currently designing a distributed application which is composed of
>> clients
>> and a remote server.
>>
>> I struggle with the asynchronous client api... I want it to be the most
>> simple
>> to use and to implement.
>>
>> Here is some background: the clients can send commands, when the server
>> receive the
>> command it does some work and response back to the client. For that, i
>> use a DEALER
>> socket in the client side and a ROUTER socket in the server side because
>> i want the
>> client and the server to be asynchronous. In addition, the client can
>> receive some
>> unexpected event messages.
>>
>>
> Why not a request socket in the client?
>
>
>> An UUID is associated to each command.
>>
>>
>> What i want to do, from a user point of view, is something like that:
>>
>>     - async_result = client.command("some_data") # send the command to
>> the server
>>     - ... do other stufs ...
>>     - result = async_result.get() # block until the response is available
>>
>>
> Sounds a lot like a future/promise relationship.
>
>
>> Here is a schema of what i imagined so far:
>> http://postimg.org/image/qxyazo2st
>>
>
> Nice schema. Have you thought about adding the possibility of this
> asynchronous command being made into a library? Possibly even it's own
> process and therefore daemon? Would be interesting to do so, with python's
> first class modules, you could easily implement the same send and receive
> API, so any consumer could just as easily take advantage of the library
> features by simply importing a different module.-
>
>>
>>
>> In this design, there is a thread which run an event loop, this event
>> loop will listen
>> to a PAIR socket which make the client able to interact with it
>> and tells it to send commands.
>>
>
> Req socket would do this, but ok.
>
>
>>
>> When the server send the response, the event loop handler will put the
>> response on
>> a data structure with the command UUID. The client can retrieve the
>> response thanks
>> to the command UUID.
>>
>> So there is a synchronization to do between the main application thread
>> and the
>> "framework thread".
>>
>> My questions are the following:
>>
>>     1. Do you think its a simple design for this purpose ?
>>
>
> Well, have you thought about the possibility of adding additional servers?
> This would easily get hard in that each client would never every server's
> address/config info... you could easily put a load balancing broker in here
> for easy scalability. It could do nothing but facilitate connection
> information exchange for direct direct communication. In this way, you
> preserve the entire communication model you've already designed, but also
> get more scalability.
>
>
>>     2. Is it possible to avoid the use of a "framework thread"(and then
>> avoid threads synchronization) ?
>>
>>
> Thread synchronization shouldn't be so hard with message passing semantics
> under the hood, if you just have the API hide this complexity.
>
> By the way, while I'm writing everything, I'm wondering if it's actually
> already been done... *hint :)
>
>
>>
>> Thank you for your help in advance :) !
>>
>> --
>> Yassine
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150309/14138354/attachment.htm>


More information about the zeromq-dev mailing list