[zeromq-dev] Async access to a single socket - send and recv without reducing throughput

Lindley French lindleyf at gmail.com
Thu Jan 2 22:27:07 CET 2014


Did the inproc solution work well for you?

I believe this is a very common problem (how do I send and receive
asynchronously on the same zmq socket without touching it from multiple
threads?), and there ought to be a best practice defined for it.


On Tue, Dec 31, 2013 at 11:26 AM, Amir Taaki <zgenjix at yahoo.com> wrote:

> thanks! I went with inproc sockets... hope performance is good. will
> benchmark this.
>
>
> https://github.com/spesmilo/obelisk/commit/c0882a9b74bcce3cca41e0cf6dab4cb93552ad39
>
>
>
>
>
> On Tuesday, December 31, 2013 12:44 PM, Bruno D. Rodrigues <
> bruno.rodrigues at litux.org> wrote:
> poll on an additional pair of zmq inproc:// sockets ;)
>
>
> On Dec 31, 2013, at 12:41, Matt Connolly <matt.connolly at me.com> wrote:
>
> > Zmq poller can also wake on standard file descriptors (eg unix socket).
> If your custom event can write to a unix socket or pipe you might be in
> luck.
> >
> > Cheers
> > Matt.
> >
> >> On 31 Dec 2013, at 4:35 pm, Amir Taaki <zgenjix at yahoo.com> wrote:
> >>
> >>
> >>
> >> I was reading the docs and saw:
> >>
> >> "How can I integrate ØMQ sockets with normal sockets? Or with a GUI
> event loop?
> >> You can use the zmq_poll() function to poll for events on both ØMQ and
> normal sockets. The zmq_poll() function accepts a timeout so if you need to
> poll and process GUI
> >> events in the same application thread you can set a timeout and
> >> periodically poll for GUI events. See also the reference documentation."
> >>
> >>
> >> How can I wake up zmq_poll() with a custom event? This seems to solve
> my problems if possible. Then I can create a wakeup to process sends in the
> polling loop.
> >>
> >>
> >> On Tuesday, December 31, 2013 6:14 AM, Amir Taaki <zgenjix at yahoo.com>
> wrote:
> >>
> >> Hi!
> >>
> >> I have a bit of a design issue. I want to achieve this:
> >>
> >> - Software receives a request on a DEALER socket from a ROUTER socket
> (think the worker in paranoid pirate pattern).
> >> - Request is executed asynchronously in the software. Result is ready
> inside another thread.
> >>
> >> - Send it back over the same socket as the receive.
> >>
> >> In an ideal world, I'd be polling the socket to receive in one thread,
> and then performing the sends from another. But we cannot use sockets from
> multiple threads - sockets can only be used from one thread.
> >>
> >> Currently I have a multi-producer single-consumer lockless queue for
> sends, and then there's an std::condition_variable that's waiting for a
> signal to wake-up and batch process the send queue. Also there's a polling
> loop for receives. All the options I can think of have downsides:
> >>
> >> * Use a separate receive and send socket. I'd need some mechanism so
> that the broker (ppqueue) is aware of receive/send socket pairs. Maybe a
> special message for the broker that isn't relayed, but indicates the
> identity of the receive socket.
> >>
> >> * Polling then sending reduces the throughput of sends. I've
> benchmarked performance and it's a significant hit. You're essentially
> penalising sends by T microsecs. Sleeping for 0 is not a good idea since
> CPU usage hits 100%.
> >>
> >> * Using the same socket but synchronising access from the send/recv
> threads - ZMQ crashes infrequently when I do this because of triggered
> asserts. It'd be great if I know how to do this safely (maybe by calling
> some method on the socket).
> >>
> >> How do I achieve this scenario of 50% random receives and 50% random
> sends? It's not like the classic scenario of a receive, followed by some
> synchronous code path, and then a send (within the same thread). It's not
> an option to wait for requests to finish as I dropped Thrift to use ZMQ
> because of its async ability.
> >>
> >> Thanks!
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140102/4b9bbb38/attachment.htm>


More information about the zeromq-dev mailing list