[zeromq-dev] NetMQ: is PUB/SUB filtering done at publisher side?

[Giacomo Tesio]

Hi Doron!

Thanks for your help.
I have two (windows7) processes that communicate via netmq pub/sub (the
publisher runs as x86, the subscriber as x64). While debugging (in visual
studio), I've set a breakpoint just after the send() (in the publisher) and
just after the Recieve (in the subscriber) and I noted that the publisher
send the message but hang while the subscriber handle it.

The publisher forwards data from another async source that is very fast and
I must ensure that:
 1) no message from the source is lost (thus I can't block the publisher
process, because the async source requires very low latency and do not
provide any cache: if no one is ready to read the message they send, the
message is lost)
 2) no message from the publisher is lost by the subscribers, even if they
have different processing speeds.

To ensure 2 I'm using an unbounded BlockingCollection<T> as a queue (that
btw, enable me to TryTake messages with a timeout, that is useful to
control the consuming thread when the publisher don't send nothing)

In case of slow subscribers, does NetMQ alone (without the unbounded
BlockingCollection) ensure that I won't neither loose any message nor block
the publisher or other subscribers?


Giacomo






On Thu, Jun 27, 2013 at 9:53 AM, Doron Somech <somdoron at gmail.com> wrote:

> Hi Giacomo,
>
> I'm happy to hear you choose netmq, answers to your questions:
>
> 1. There are no functionality differences between NetMQ and CLRZMQ/ZeroMQ,
> there are some small API differences between NetMQ and CLRZMQ.
> 2. Same as ZeroMQ, so publisher side (it's based on ZeroMQ 3.2.2)
> 3. What to do you mean blocked? how did you noted that? I don't think you
> should queue because NetMQ / ZeroMQ is already doing exactly that. Just
> consume the messages from the socket.
>
> Doron
>
>
> On Thu, Jun 27, 2013 at 1:04 AM, Giacomo Tesio <giacomo at tesio.it> wrote:
>
>> Hello everybody, I'm working to my first ZeroMQ based real-world
>> application and I have a few questions:
>>
>> 1) I'm working on .NET and I choosed NetMQ instead of CLRZMQ just on the
>> base of github activity. I've noted there are some differences between the
>> two API (for example I had to Subscribe("") instead of SubscribeAll()) but
>> they seem minor differences at a first look. Is there any difference in
>> functionality that I should be aware of?
>> 2) I have to use a topic based PUB tcp socket: is PUB/SUB filtering done
>> at publisher side?
>> 3) I noted (in debug) that the publisher blocks while the subscribers
>> handle each message: I've tried to minimize the impact of this with a queue
>> of messages of subscriber side, with the subscriber that fills the queue
>> and another thread consuming it. Is there a better approach?
>>
>> Thanks a lot!
>>
>>
>> Giacomo
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130627/2bf12e62/attachment.htm>