[zeromq-dev] ZMQ vs FF

[A. Mark]

So far from reading the ZMQ examples I have the situation of a shared
resource (ring-buffer) by several different workers which work on the
buffers. The worker threads are dynamic in the sense that they may be
paused and started at runtime, so I need a controller also. My first basic
approach would be a single broker controlling the ring buffer state to
which clients connect to, request buffers and submit finished buffers. I
think a REQ-DEALER-ROUTER-REP chain should be able to handle this. I will
have to add a controller somehow too.

On Thu, Jan 17, 2013 at 9:39 AM, A. Mark <gougolith at gmail.com> wrote:

> Thanks Steve,
>
>
> The bottom line in my situation is this, I have a ringbuffer and several
> threads with serial dependencies on each other.  It's an MPMC situation. My
> main goal is minimizing wait and minimizing CPU cost of scheduling threads
> at the same time. In my current implementation -  which as I mentioned is a
> bit of a nightmare with many complicated rules  - I use a single
> lock/unlock interface. The only relatively "easy" way I found to make it
> deadlock-free is to use timeout's on locks and broadcasts once in a while.
> It's a messy business even though it does work. The worst part is that it
> doesn't scale well since the timeouts and wake-ups consume more and more
> CPU. These are wasted wake ups due to timeouts and broadcasts....So here I
> am looking into ZMQ and FF. I feel like if ZMQ can scale better I would
> rather pay a bit of penalty in performance (I mean slower thread
> switching), if it also less "wasteful" for unnecessary wake-ups. I'm only
> learning ZMQ and no very little about FF.
>
> Interesting article, I wish I could use it...
>
> On Thu, Jan 17, 2013 at 6:04 AM, Steven McCoy <steven.mccoy at miru.hk>wrote:
>
>> On 17 January 2013 01:35, A. Mark <gougolith at gmail.com> wrote:
>>
>>> I'm needing to implement some low level MPMC queues. I've been rolling
>>> my own using pthreads, and so far I've been pulling my hair out in the
>>> process also... My code is strictly C at this point and I'd like to keep it
>>> that way. I've looked into FF (FastFlow) and find it's rather technical for
>>> low level so far. Is there any advantages over using ZMQ for MPMC queues?
>>> What would be the pros and cons of using ZMQ over FF for MPMC? It seems to
>>> me so far that ZMQ is very well suited for MPMC solutions.
>>>
>>
>> Interesting, the technology paradigm is effectively the same but ZeroMQ
>> wraps in an easy to learn and use interface.
>>
>> Of note from this page,
>>
>>  ... queues, which require neither locks nor atomic operations. SPMC,
>>> MPSC, and MPMC queues are realized by using only SPSC queues and an arbiter
>>> thread,
>>
>>
>> http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace:architecture
>>
>>
>> On one hand you have full memory bus speed queues for single-provider
>> single-consumer, but on the other you have a broker thread with implied
>> synchronisation overheads.
>>
>> There's a paper to read, that discusses queue implementations without
>> atomic primitives:
>>
>>
>> http://calvados.di.unipi.it/dokuwiki/lib/tpl/torquati/paper_files/TR-10-20.pdf
>>
>> --
>> Steve-o
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130119/3512f33c/attachment.htm>