[zeromq-dev] assertion failure with master

[Chuck Remes]

On Nov 8, 2010, at 1:12 PM, Chuck Remes wrote:

> A lot of testing and discussion regarding this issue occurred on irc. Here's a summary.
> 
> 1. OSX reports very strange values when the 0mq library runs getsockopt for SO_SNDBUF. It returns 32. The thought is that it is returning kbytes instead of bytes like every other platform. :(
> 
> 2. The assertion occurs because the system is running out of space in the send buffer. On OSX there are two separate send buffers; one is for TCP for remotes and the second is for local loopback communications.
> 
> For symmetry, I have set both the send and recv buffers to larger values.
> 
> # the sendspace + recvspace buffers must be less or equal to maxsockbuf
> kern.ipc.maxsockbuf=8388608
> 
> net.inet.tcp.sendspace=1048576
> net.inet.tcp.recvspace=1048576
> net.local.stream.sendspace=3000000
> net.local.stream.recvspace=3000000
> 
> By default, OSX *ignores* these values unless a certain minimum number of sockets are open. To override that behavior and make sure that it *always* uses my values, do this:
> 
> net.inet.tcp.sockthreshold=0  # default was 64
> 
> 3. On my Mac, the maxsockbuf was stuck at 4MB. The only way to make it larger was to force the machine to boot with a 64-bit kernel (OSX defaults to a 32-bit kernel for most machines). The 64-bit kernel let me bump up the values a bit higher as can be seen above (maxsockbuf is set to 8MB).
> 
> 4. OSX uses the launchctl daemon to override the 'max open files' (file descriptor) setting. Google for it. Note that you *cannot* pass "unlimited" as the hard limit; OSX *requires* that it be a number.
> 
> 
> After I made the buffer changes, everything is working okay. I am certain there is still a problem here particularly when the mailbox code needs to modify the send buffer size (SO_SNDBUF). That logic isn't working right. I'll make a ticket to track it.
> 
> My test program can successfully create over 5k sockets now without bitching, so I'm happy.
> 
> Thanks to mato, et al for help on irc.

Oh, one more detail...

5. 0mq defaults to a maximum of 512 open sockets. This is hardcoded at build time via src/config.hpp. Bump that value up if you need to make more than 512 sockets and rebuild the library.

cr