[zeromq-dev] Async access to a single socket - send and recv without reducing throughput

[Amir Taaki]

I was reading the docs and saw:

"How can I integrate ØMQ sockets with normal sockets? Or with a GUI event loop? 
You can use the zmq_poll() function to poll for events on both ØMQ and normal sockets. The zmq_poll() function accepts a timeout so if you need to poll and process GUI 
events in the same application thread you can set a timeout and 
periodically poll for GUI events. See also the reference documentation."


How can I wake up zmq_poll() with a custom event? This seems to solve my problems if possible. Then I can create a wakeup to process sends in the polling loop.


On Tuesday, December 31, 2013 6:14 AM, Amir Taaki <zgenjix at yahoo.com> wrote:

Hi!

I have a bit of a design issue. I want to achieve this:

- Software receives a request on a DEALER socket from a ROUTER socket (think the worker in paranoid pirate pattern).
- Request is executed asynchronously in the software. Result is ready inside another thread.

- Send it back over the same socket as the receive.

In an ideal world, I'd be polling the socket to receive in one thread, and then performing the sends from another. But we cannot use sockets from multiple threads - sockets can only be used from one thread.

Currently I have a multi-producer single-consumer lockless queue for sends, and then there's an std::condition_variable that's waiting for a signal to wake-up and batch process the send queue. Also there's a polling loop for receives. All the options I can think of have downsides:

* Use a separate receive and send socket. I'd need some mechanism so that the broker (ppqueue) is aware of receive/send socket pairs. Maybe a special message for the broker that isn't relayed, but indicates the identity of the receive socket.

* Polling then sending reduces the throughput of sends. I've benchmarked performance and it's a significant hit. You're essentially penalising sends by T microsecs. Sleeping for 0 is not a good idea since CPU usage hits 100%.

* Using the same socket but synchronising access from the send/recv threads - ZMQ crashes infrequently when I do this because of triggered asserts. It'd be great if I know how to do this safely (maybe by calling some method on the socket).

How do I achieve this scenario of 50% random receives and 50% random sends? It's not like the classic scenario of a receive, followed by some synchronous code path, and then a send (within the same thread). It's not an option to wait for requests to finish as I dropped Thrift to use ZMQ because of its async ability.

Thanks!
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev