[zeromq-dev] ZeroMQ with C++ and Fibers

[Eric Hill]

The common model for fiber development is to use only overlapped io
operations and have each fiber GetQueuedCompletionStatus on an io
completion port after every call to io resources.  This usually means
you'll come up with your own set of classes to perform io that wrap up a
call to an overlapped function (read file, open file, write file, etc) so
the logic of yielding occurs automagically once an io operation is queued.
Once your operation completes, any thread running any fiber that has
yielded to the completion port will restore the fiber context and continue
on like the io operation was a blocking op.  This works great.

And is terribly complicated.  The ZMQ model doesn't really lend itself to
fitting into this model.  You would have to change several of the core
waiting routines to use your shared iocp and call your custom operations to
make this work out.  It would be easier to either call ZMQ as a blocking
call (trusting ZMQ to manage its' own threads under the hood), or create
your own code using the ZMQ protocol spec as a guide.

At the end of the day, you have a huge amount of code to maintain, and only
in exceptional cases a performance increase to show for the added work.

Just my 2c.


On Fri, May 10, 2013 at 9:39 AM, Ian Upright <ian-public at upright.net> wrote:

> On Thu, 9 May 2013 23:08:27 -0400, you wrote:
>
> >On 9 May 2013 18:39, Ian Upright <ian-public at upright.net> wrote:
> >
> >> It's not completely clear to me how I would combine ZeroMQ with a task
> >> library such as above.  Is it possible to combine the two?  If so, does
> >> anyone have any idea or examples of how that could be achieved?
> >
> >
> >Fibers are a bit old hat, Windows supports them for compatibility in
> >porting.  The only caveat with 0mq is sockets passing through threads,
> >fiber implementations generally do not permit fibers to migrate.
> >
> >The challenge with fibers is how to yield.  This would generally mean each
> >fiber polling their 0mq sockets and yielding if empty.  This would
> >generally lead to inefficient power usage compared with vanilla threading
> >models as all the fibers will constantly spin unless under heavy traffic.
>
> Are you sure about that?  Wouldn't it work something like this:
> Suspend the Fiber, perhaps push the request to another IPC thread group
> that
> is farming out the requests.  Then when a reply comes back in, to send it
> back over IPC, and then that part would resume that Fiber which was
> associated with the request/reply.
>
> Polling in general seems like a bad idea.
>
> Perhaps I'm answering my own question as to how to do it?
>
> Ian
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130510/8b704dfa/attachment.htm>