[zeromq-dev] A design question

Thomas Rodgers rodgert at twrodgers.com
Thu Dec 18 17:36:27 CET 2014


That's why I said "reserve" not "commit" :)~

I vaguely recall this to also be the case with Windows.

The problem with 32bit arch hosts is not the amount of memory commit, but
finding enough 1-2MB contiguous holes in the address space below 2GB
(Windows) or 3GB (Linux) to reserve. You could also potentially have
additional issues if you made heavy use of TLS.

On Thu, Dec 18, 2014 at 10:28 AM, Michel Pelletier <
pelletier.michel at gmail.com> wrote:
>
> Modern Linux doesn't actually map stack space into real memory unless the
> thread accesses the virtual memory addresses on demand.  I presume it's the
> same for Windows.  Garrett Smith pointed this out to me when I made the
> same claim at the Chicago meetup a couple years ago.
>
> -Michel
>
> On Thu, Dec 18, 2014 at 8:16 AM, Thomas Rodgers <rodgert at twrodgers.com>
> wrote:
>>
>> The biggest problem with 280, mostly idle, threads would be the amount of
>> stack space the OS is required to reserve (1MB for Windows/2MB for Linux).
>> On a 64bit host this is unlikely to be much of an issue.
>>
>> On Mon, Dec 15, 2014 at 6:11 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>>
>>> 280 threads sounds fine unless you have data showing that it's a
>>> problem. That's easy to test on your target platform.
>>>
>>> On Sun, Dec 14, 2014 at 7:54 AM, Bob Clarke <optionguy71 at gmail.com>
>>> wrote:
>>> > Platform: 0MQ 4.0.4 on Windows 7/Windows Server 2008
>>> >
>>> > I am writing a server monitoring program to replace an ancient (1999)
>>> > program that is almost impossible to maintain.
>>> >
>>> > All of our application servers host proprietary Windows services
>>> written in
>>> > C++ (except for a couple .Net apps) that use a proprietary synchronous
>>> > request/response system (it works very well, has been for many years,
>>> and
>>> > isn't being replaced any time soon).
>>> >
>>> > The monitoring program sends test scripts to the server apps and waits
>>> for a
>>> > reply. In most cases, the reply is almost immediate, but in some cases,
>>> > there could be a delay of up to 30 seconds. So, to avoid one script
>>> causing
>>> > delays with others, each test script runs in its own thread. This
>>> thread
>>> > must send its results back to a listener, so it creates and uses a 0MQ
>>> > socket to do so. Each test script has a repeat interval; some are run
>>> every
>>> > five seconds, but most are run every 45 seconds, 90 seconds, or less
>>> > frequently.
>>> >
>>> > There are about 280 test scripts (we have thousands of servers across
>>> three
>>> > data centers), so I seem to have two unfortunate choices:
>>> > 1) Keep the script threads around so I'm not creating and destroying a
>>> > socket each time a test runs;
>>> > 2) Let the script threads terminate so I am not keeping 280 threads
>>> open at
>>> > once. Since the script repeat intervals are quite long in computer
>>> terms,
>>> > these threads will spend most of their time do nothing.
>>> >
>>> > I lean towards the first option, but opening and closing sockets that
>>> often
>>> > doesn't sound good. Then again, keeping 280+ threads around doesn't
>>> sound so
>>> > great, either.
>>> >
>>> > So before I go too far down this development road, does anyone have
>>> some
>>> > experience they could share?
>>> >
>>> > Thanks.
>>> >
>>> > Bob
>>> >
>>> >
>>> > _______________________________________________
>>> > zeromq-dev mailing list
>>> > zeromq-dev at lists.zeromq.org
>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> >
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20141218/19c3299e/attachment.htm>


More information about the zeromq-dev mailing list