[zeromq-dev] NetMQ: is PUB/SUB filtering done at publisher side?

[Doron Somech]

Hi Giacomo,

I'm pretty sure  subscribers cannot block the publisher, I'm not sure the
breakpoint is a good way to test that.

Anyway regarding your other questions, NetMQ can handle slow subscribers,
you just need to set the SendWatermark to zero and NetMQ will queue all
messages, you should however know that this can cause to a memory problem.

I don't think you need any queues at any side, ZeroMQ and NetMQ are working
with queues, when you send a message or receive a message you actually
queue and dequeue, there is another thread that actually do the network
stuff. So I think you should publish the message to NetMQ socket directly
from the thread that listen to the source and on the subscriber side just
consume the messages directly from the queue.

Regarding the timeout, you can also do it with NetMQ using a poller or Poll
method on a Socket.




On Thu, Jun 27, 2013 at 11:45 AM, Giacomo Tesio <giacomo at tesio.it> wrote:

> Hi Doron!
>
> Thanks for your help.
> I have two (windows7) processes that communicate via netmq pub/sub (the
> publisher runs as x86, the subscriber as x64). While debugging (in visual
> studio), I've set a breakpoint just after the send() (in the publisher) and
> just after the Recieve (in the subscriber) and I noted that the publisher
> send the message but hang while the subscriber handle it.
>
> The publisher forwards data from another async source that is very fast
> and I must ensure that:
>  1) no message from the source is lost (thus I can't block the publisher
> process, because the async source requires very low latency and do not
> provide any cache: if no one is ready to read the message they send, the
> message is lost)
>  2) no message from the publisher is lost by the subscribers, even if they
> have different processing speeds.
>
> To ensure 2 I'm using an unbounded BlockingCollection<T> as a queue (that
> btw, enable me to TryTake messages with a timeout, that is useful to
> control the consuming thread when the publisher don't send nothing)
>
> In case of slow subscribers, does NetMQ alone (without the unbounded
> BlockingCollection) ensure that I won't neither loose any message nor block
> the publisher or other subscribers?
>
>
> Giacomo
>
>
>
>
>
>
> On Thu, Jun 27, 2013 at 9:53 AM, Doron Somech <somdoron at gmail.com> wrote:
>
>> Hi Giacomo,
>>
>> I'm happy to hear you choose netmq, answers to your questions:
>>
>> 1. There are no functionality differences between NetMQ and
>> CLRZMQ/ZeroMQ, there are some small API differences between NetMQ and
>> CLRZMQ.
>> 2. Same as ZeroMQ, so publisher side (it's based on ZeroMQ 3.2.2)
>> 3. What to do you mean blocked? how did you noted that? I don't think you
>> should queue because NetMQ / ZeroMQ is already doing exactly that. Just
>> consume the messages from the socket.
>>
>> Doron
>>
>>
>> On Thu, Jun 27, 2013 at 1:04 AM, Giacomo Tesio <giacomo at tesio.it> wrote:
>>
>>> Hello everybody, I'm working to my first ZeroMQ based real-world
>>> application and I have a few questions:
>>>
>>> 1) I'm working on .NET and I choosed NetMQ instead of CLRZMQ just on the
>>> base of github activity. I've noted there are some differences between the
>>> two API (for example I had to Subscribe("") instead of SubscribeAll()) but
>>> they seem minor differences at a first look. Is there any difference in
>>> functionality that I should be aware of?
>>> 2) I have to use a topic based PUB tcp socket: is PUB/SUB filtering done
>>> at publisher side?
>>> 3) I noted (in debug) that the publisher blocks while the subscribers
>>> handle each message: I've tried to minimize the impact of this with a queue
>>> of messages of subscriber side, with the subscriber that fills the queue
>>> and another thread consuming it. Is there a better approach?
>>>
>>> Thanks a lot!
>>>
>>>
>>> Giacomo
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130627/1aa5bc35/attachment.htm>