[zeromq-dev] Socket usage question

Gary Davidson Gary.Davidson at interclick.com
Wed Aug 24 16:40:54 CEST 2011


Hi Matt

Nope it is not the best practice in this case.

Your bottleneck will be network IO. Use only the number of threads needed to get the target throughput if the thread count == core count and you still are not at your goal you need to scale up or use a different pattern or maybe multipart messages. In this case don't fall into the MS trap of Big Big Big. They license by CPU.

I have not had to use more than one task (thread) and I can kill a 100 meg card. As for me I spin a new VM

Small is the new Big

Larger messages will have lower msg/per second but can have higher bytes per second.

Remember a ZeroMQ socket is not a .net "socket",  I doubt throughput will be an issue using 1 or 2 threads. I have close to a billion messages a day I do not expect to see much more than 1 or 2 threads when I implement ZeroMq.

As for thread and CPU count a CPU core works one thread at a time, it is more efficient for the # of threads to equal the number of cores in .Net 40 TPL - IF YOU NEED THEM. There may be a use case for more threads but sending and receiving messages is not one of them.


G



[Description: C:\Users\gary.davidson\AppData\Roaming\Microsoft\AD_interclick\logo.png]

Gary Davidson, Chief Software Architect
Gary.Davidson at interclick.com<mailto:Gary.Davidson at interclick.com>
P 561 300 2764
C 954 790 0715
F 561 300 2765

4800 T-Rex Avenue Suite 120
Boca Raton, FL 33431

www.interclick.com
NASDAQ: ICLK



From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Matt Haberman
Sent: Wednesday, August 24, 2011 10:10 AM
To: zeromq-dev at lists.zeromq.org
Subject: [zeromq-dev] Socket usage question


So I am assuming that in general, the best practice for communicating with 0MQ via a multi-threaded application is to maintain a fixed number of sockets (perhaps equal to the CPU core count) instead of creating a socket for each thread? I am trying to maximize throughput.



Thanks,

Matt





1000 iterations creating and destroying 1000 sockets? Use Environment.ProcessorCount in your loop and send msgcount\ Environment.ProcessorCount messages in each task.



Alternatively you can use TPL Parallel.For.



To simulate load use the cloud. I had 5 clients sending to 1 server at 269K messages per second with the CPU  never exceeding 30% the network card was saturated though.



Even on my I7 laptop to my mac book I get excellent through put.



Currently you have thread context switching issues. You can verify this using performance counters if they will even load which I doubt.



Gary Davidson | Chief Software Architect | interclick



Gary.Davidson at interclick.com<http://lists.zeromq.org/mailman/listinfo/zeromq-dev>

P 561 300 2764  |  C 954 790 0715  |  F 561 300 2765

4800 T-Rex Avenue Suite 120 Boca Raton, FL 33431

www.interclick.com<http://www.interclick.com>    NASDAQ: ICLK



-----Original Message-----

From: zeromq-dev-bounces at lists.zeromq.org<http://lists.zeromq.org/mailman/listinfo/zeromq-dev> [mailto:zeromq-dev-bounces at lists.zeromq.org<http://lists.zeromq.org/mailman/listinfo/zeromq-dev>] On Behalf Of Matt Haberman

Sent: Tuesday, August 23, 2011 10:10 PM

To: zeromq-dev at lists.zeromq.org<http://lists.zeromq.org/mailman/listinfo/zeromq-dev>

Subject: [zeromq-dev] Socket usage question



I am trying simulate load on a process created with 0MQ (CLZMQ2 binding) and have a question about when to create sockets. My application is listening for requests using an XREP socket. I wrote a little console app to simulate concurrent clients that looks like:



public class Client

{

    Context context;



    public Client()

    {

        context = new Context(1);

    }

    public void SendRequest()

    {

        for (int i = 1; i <= 1000; i++)

        {

            Task t = new Task(

                () =>

                {

                    Socket r = context.Socket(SocketType.REQ);



                    r.Connect("tcp://localhost:5559"<tcp://localhost:5559%22>);

                    r.Send("Hello", Encoding.Unicode);

                    string reply = r.Recv(Encoding.Unicode);

                    Console.WriteLine("Received reply {0}", reply);

                    r.Dispose();

                }

            );

            t.Start();

            Thread.Sleep(100);

        }

    }

}

Frequently, when i run the console app my entire machine freezes and is unresponsive. Is there something wrong with this approach?



Thanks,

Matt

_______________________________________________

zeromq-dev mailing list

zeromq-dev at lists.zeromq.org<http://lists.zeromq.org/mailman/listinfo/zeromq-dev>

http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110824/2c279ed8/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 8293 bytes
Desc: image001.png
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110824/2c279ed8/attachment.png>


More information about the zeromq-dev mailing list