[zeromq-dev] Pub/sub pattern: Binding two sources to same address not working

Bruno D. Rodrigues bruno.rodrigues at litux.org
Wed Jan 15 01:33:57 CET 2014 

On my local tests I’ve been getting a rate of 1:4 of performance difference between all C code (libzmq’s perf local_thr/remote_thr/proxy) and all java code (jeromq) (and mixed results using the JZMQ bindings, mostly because of JNI itself, not the code). With some tweaking (rather bug fixing for mac) I was able to double the performance of the C proxy (and 10x’plicate on the mac), which is code still pending some cleanup on my github fork (davipt). I guess I should also look at the jeromq code and see if the same trick applies. Either way all tests I’ve done have put JeroMQ performance above 1Gbps, which was my aim, so I’ve been concentrating on other stuff. 

In the past when I did the exact same question and tried to design a way to have multiple proxies to divide the load (or cross the Gbps per machine), Pieter convinced me to ditch the proxy and move into a peer-to-peer design. It’s more complicated to get right, specially with dynamic source nodes and agents nodes, and in your case with the agents talking back to themselves, but it’s feasible and removes the bottleneck - or shall I say SPOC of the proxy. Hence why I didn’t look at JeroMQ’s proxy code yet.

But early optimization sometimes is overkill. And with ZeroMQ the advantage is that it’s so damn simple to change from one type of connection to another one - tcp vs ipc vs inproc, bind vs connect, one connection vs a dozen of them. What I mean is that if tomorrow you want to get rid of the proxy, your code won’t change, only the socket’s connect()/bind() stuff (ok you’ll need some code for node discovery, but that’s different).

About your numbers, always count not only msg/sec, but bytes/bits per second. 100K msgs/sec, if at 1K each, is already reaching the 1Gb. 

One thing you should do is design the message types to take advantage of the subscription prefixes of zeromq, so each agent really gets only the messages then need to receive, instead of receiving them and discarding like I did on my sample code. That helps a lot!

On Jan 14, 2014, at 23:29, Andreas Bauer <brm1 at gmx.net> wrote:

> Many thanks for the code! It works now. 
>  
> I'll take care of the message growth. In fact it shouldn't be a problem, because each agent only accepts a certain type of message and foremost only emits a single type of message, i.e. the message are not "blindly" forwarded by all agents. As mentioned the goal was to provide a fixed filtering point to new agents  where they can listen for the messages they need.
>  
> But now the performance questions arise and I ask for your experience with 0mq as I'm still learning :)
>  
> Can such a proxy keep the processing performance constant or will it be a bottle neck if I keep adding agents? Can I expect to keep the processing rate constant if I design my agents to take care of the suicidal snail pattern? A short test (without having the agents yet optimized): Without any agents the source can emit 100.000 message/second. With only 3 agents it is already down to 22.000. So is this an expected behaviour no matter what I do with the agents, or can I expect to keep the rate constant. 
>  
> I'm asking because this gives me a hint if my optimizations are working and it saves time if it is impossible anyhow.
>  
> Gesendet: Dienstag, 14. Januar 2014 um 12:59 Uhr
> Von: "Bruno D. Rodrigues" <bruno.rodrigues at litux.org>
> An: "zeromq-dev at lists.zeromq.org" <zeromq-dev at lists.zeromq.org>
> Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
> https://gist.github.com/davipt/8417197
>  
> Launch *one* proxy.
>  
> Launch one or more sources.
>  
> Launch one or more agents.
>  
> Rethink about agents sending messages to other agents, as that way the messages will exponentially multiply if you’re not careful :P
>  
>  
>  
>  
>  
>  
> On Jan 13, 2014, at 22:47, brm1 at gmx.net wrote:
>  
> Thanks for your advise. But I'm still struggeling.
>  
> To be honest, I still don't succeed in receiving and publishing messages at the same time using a proxy. I must be doing something wrong. Maybe you could have short look. Sometimes I'm too blind to see... http://pastebin.com/ar6bH8vq
>  
>  
>  
> Gesendet: Montag, 13. Januar 2014 um 09:57 Uhr
> Von: "Bruno D. Rodrigues" <bruno.rodrigues at litux.org>
> An: "zeromq-dev at lists.zeromq.org" <zeromq-dev at lists.zeromq.org>
> Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
> Thats the part I couldn’t understand.
>  
> But then again, it shouldn’t be that hard:
>  
> 1. with zeromq it’s irrelevant who binds and who connects. There is no “client connects to servers”.
>  
> 2. one of the services/node must be well known to the others. This is the one that should do the bind. If there are multiple, then each will have a different port, if running on the same machine, or different ip
>  
> 3. a zeromq socket can connect to multiple points; a socket can also bind to multiple points; heck a socket can even bind and connect at the same time.
>  
> 4. from what I understood you’re trying to do some kind of broadcast bus where every node connects to listen and rebroadcast stuff (hope they take care of loop messages). For this case it may be worth to have a separate note doing a XSUB(bin)-XPROXY-XPUB(bind] so you get a well known port (two) and each agent or other services can connect to it.
>  
> ctx = zmq.context(1)
> s1 = ctx.socket(ZMQ.XSUB); s2 = ctx.socket.ZMQ.XPUB);
> s1.bind(“tcp://*.5556”); s2.bind(“tcp://5557”);
> ZMQ.proxy(s1,s2, null);
>  
> Then each Agent does a SUB socket connecting to 5557 and, to republish messages, a PUB socket connecting to 5556.
>  
>  
>  
>  
> On Jan 13, 2014, at 8:13, brm1 at gmx.net wrote:
>  
> Ok, you're right regarding tcp and maybe I phrased it a bit complicated.
>  
> In fact the sources should be fan-out nodes and the agents fan-in/fan-out nodes. I'm not sure how to correctly implement it so I don't have to use a different port for every new node or message type. Or is the only correct way to specify a different port for every fan-out node?
>  
> Gesendet: Sonntag, 12. Januar 2014 um 23:33 Uhr
> Von: "Bruno D. Rodrigues" <bruno.rodrigues at litux.org>
> An: "ZeroMQ development list" <zeromq-dev at lists.zeromq.org>
> Betreff: Re: [zeromq-dev] Pub/sub pattern: Binding two sources to same address not working
> 
> It was hard for me to understand what are you trying to do but I guess the simple answer is no, you can't have the same port (ip+port to be correct) bound multiple times. That's how TCP works. 
>  
>  
> --
> Bruno Rodrigues
> Sent from my iPhone
> 
> No dia 12/01/2014, às 21:51, Andreas Bauer <brm1 at gmx.net> escreveu:
>  
>  
> Hello,
>  
> I'm fairly new to ZeroMq and that's why I'm asking for a hint regarding the correct usage of 0mq. I'm using version 3.2.2. with Java and jeromq version 0.3.2. My scenario is running on a local machine for test purposes. Here is what I try to accomplish (snippets: http://pastebin.com/YG9adHEy).
>  
> I have several publishers/sources publishing messages via tcp port 5556. Furthermore I'm trying to implement various "agents" that receive those messages by listening at port 5556 and in addtion they publish their messages also via tcp and port 5556. This is because I want to add new agents dynamically that can receive messages from existing agents as well as already running agents can accept messages from the new agents without reconfiguration. I thought they all need a common port, so I can configure all agents identically.
>  
> Hence I have implementend a source that sends message via tcp and port 5556. I've also got (at the moment only) one agent that subscribes to the messages sent from the source. This works great unless I try to publish messages from the agent via port 5556. What I try to say is, when I start the agent without the "Agent - Source" part (see snippet) the agent ("Agent -Receiver) receives the messages. If I start the agent also with publisher part (Agent - Source.init()), the agent does not receive any messages. http://pastebin.com/YG9adHEy 
>  
> At the moment I'm a bit stuck, because I'm not sure anymore if it is possible to have several publishers bind to the same port. I thought it is, but as mentioned, if try to startup a the publisher within the agent, the agent can't receive any messages. I thought I can use the Espresso pattern, but it doesn't work (maybe I've implemented it incorrectly if this was the right approach).  
>  
> I need high throughpt therefore I chose the pub/sub pattern. But 0mq offers so many different message patterns, maybe you can give me a hint if I'm using the wrong messaging pattern.
>  
> Many thanks,
>  
> Andreas
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________ zeromq-dev mailing list zeromq-dev at lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________ zeromq-dev mailing list zeromq-dev at lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________ zeromq-dev mailing list zeromq-dev at lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140115/03b73b91/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140115/03b73b91/attachment.sig>

More information about the zeromq-dev mailing list