[zeromq-dev] Concurrent send/receive with a ROUTER socket - efficient and idiomatic solution

Michel Pelletier pelletier.michel at gmail.com
Sun Aug 31 17:12:07 CEST 2014


On Sun, Aug 31, 2014 at 1:56 AM, Eyal Arubas <eyalarubas at gmail.com> wrote:
> Thanks Sean.
> Having the send and receive operations in the same thread indeed avoids the
> concurrency issue, but still we are left with polling the socket every few
> milliseconds.
> I think it's pretty much equivalent to using threads + mutex.
> I would prefer to use something along the lines of a blocking "select"
> statement on incoming and outgoing socket events; which will not abuse the
> CPU.

poll does that.  It blocks until the timeout or a requested event
occurs on one or more registered items:

http://api.zeromq.org/4-0:zmq-poll

If you pass a timeout of -1 it blocks indefinitely until there is
something to do.

Some bindings expose a poll socket method (czmq's zsocket_poll,
pyzmq's Socket.poll), that polls just the one socket as a convenience
and your binding maybe doing that.  You want to use zmq_poll for one
or more pollitems.

-Michel


> I was hoping for a solution which, for example, allows to "select" the
> socket for inbound or outbound messages, and do one or the other.
>
> Thanks for your solution anyway.
>
>
> On Sun, Aug 31, 2014 at 4:03 AM, Sean Robertson <sprobertson at gmail.com>
> wrote:
>>
>> I ran into this as well and came up with a potentially interesting
>> solution (though probably not idiomatic).
>>
>> What I have is a single "dispatch" goroutine to keep all socket reads
>> and writes in one place. The dispatch interacts with two channels,
>> Requests (in) and Responses (out). On every loop it tries to read the
>> socket (with DONTWAIT), adds any incoming messages to the Requests
>> channel, and then looks in the Responses channel for outgoing messages
>> to send. No mutex necessary as the operations always happen one after
>> another.
>>
>> The rest of the code then only has to worry about reading and writing
>> these channels, which are of course thread-safe.
>>
>> https://github.com/prontotype-us/somata-go/blob/master/service.go#L30
>>
>> On Sat, Aug 30, 2014 at 8:35 AM, Eyal Arubas <eyalarubas at gmail.com> wrote:
>> > I am implementing a server in Go with a ROUTER socket.
>> >
>> > In one goroutine, messages are received from the socket, and distributed
>> > to
>> > workers through work queues (buffered channels).
>> > Workers produce some result and queue it to another buffered channel.
>> > In another goroutine, results are dequeued and sent back to clients.
>> >
>> > The same ROUTER socket needs to be accessed from those two goroutines
>> > (one
>> > to receive and one to send), which creates concurrency problems; as
>> > sockets
>> > are not threads-safe.
>> >
>> > As a demonstration, the following Go code panics randomly with:
>> >
>> >> fatal error: unexpected signal during runtime execution
>> >
>> >
>> > Link to demo:
>> > https://gist.github.com/EyalAr/117125b0e72a69584dee#file-error_demo-go
>> >
>> > My current solution is to use a poller with a short timeout, and a mutex
>> > to
>> > lock access to the socket. But this seems inefficient and not idiomatic.
>> > Why
>> > should I poll and lock the socket, with some arbitrary timeout, if
>> > nothing
>> > is there?
>> >
>> > Link to my current solution:
>> > https://gist.github.com/EyalAr/117125b0e72a69584dee#file-solution-go
>> >
>> > Is there a better way?
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



More information about the zeromq-dev mailing list