[zeromq-dev] Assert in signaller.cpp on XP if Cisco VPN Client isinstalled - possible fix

Fri Mar 22 12:52:40 CET 2013

I think I have fix for this issue. Basically, the listener socket
created by make_fdpair should be retained from one call of make_fdpair
to the next, by making it static and not closing it at the end.

The following replacement for make_fdpair seems to work much better.

If anyone of the ZeroMQ team wants to incorporate this into the current
build, that would be great. The code below is based on the 3.2.1 build,
which I know is not current.

int zmq::signaler_t::make_fdpair (fd_t *r_, fd_t *w_)

{

#if defined ZMQ_HAVE_EVENTFD

    // Create eventfd object.

    fd_t fd = eventfd (0, 0);

    errno_assert (fd != -1);

    *w_ = fd;

    *r_ = fd;

    return 0;

#elif defined ZMQ_HAVE_WINDOWS

    SECURITY_DESCRIPTOR sd = {0};

    SECURITY_ATTRIBUTES sa = {0};

    InitializeSecurityDescriptor(&sd, SECURITY_DESCRIPTOR_REVISION);

    SetSecurityDescriptorDacl(&sd, TRUE, 0, FALSE);

    sa.nLength = sizeof(SECURITY_ATTRIBUTES);

    sa.lpSecurityDescriptor = &sd;

    //  This function has to be in a system-wide critical section so
that

    //  two instances of the library don't accidentally create signaler

    //  crossing the process boundary.

    //  We'll use named event object to implement the critical section.

    //  Note that if the event object already exists, the CreateEvent
requests

    //  EVENT_ALL_ACCESS access right. If this fails, we try to open

    //  the event object asking for SYNCHRONIZE access only.

    HANDLE sync = CreateEvent (&sa, FALSE, TRUE, TEXT
("Global\\zmq-signaler-port-sync"));

    if (sync == NULL && GetLastError () == ERROR_ACCESS_DENIED)

      sync = OpenEvent (SYNCHRONIZE | EVENT_MODIFY_STATE, FALSE, TEXT
("Global\\zmq-signaler-port-sync"));

    win_assert (sync != NULL);

    //  Enter the critical section.

    DWORD dwrc = WaitForSingleObject (sync, INFINITE);

    zmq_assert (dwrc == WAIT_OBJECT_0);

    //  Windows has no 'socketpair' function. CreatePipe is no good as
pipe

    //  handles cannot be polled on. Here we create the socketpair by
hand.

    *w_ = INVALID_SOCKET;

    *r_ = INVALID_SOCKET;

    //  Create listening socket - first time only

    static SOCKET listener = NULL;

       //  Bind listening socket to any free local port.

       struct sockaddr_in addr;

       memset (&addr, 0, sizeof (addr));

       addr.sin_family = AF_INET;

       addr.sin_addr.s_addr = htonl (INADDR_LOOPBACK);

       addr.sin_port = htons (signaler_port);

       int rc;

       BOOL tcp_nodelay = 1;

       if (listener == NULL)

       {

              listener = open_socket (AF_INET, SOCK_STREAM, 0);

        wsa_assert (listener != INVALID_SOCKET);

              //  Set SO_REUSEADDR and TCP_NODELAY on listening socket.

              BOOL so_reuseaddr = 1;

              rc = setsockopt (listener, SOL_SOCKET, SO_REUSEADDR,

                     (char *)&so_reuseaddr, sizeof (so_reuseaddr));

              wsa_assert (rc != SOCKET_ERROR);

              rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELAY,

                     (char *)&tcp_nodelay, sizeof (tcp_nodelay));

              wsa_assert (rc != SOCKET_ERROR);

              rc = bind (listener, (const struct sockaddr*) &addr,
sizeof (addr));

              wsa_assert (rc != SOCKET_ERROR);

              //  Listen for incomming connections.

              rc = listen (listener, 1);

              wsa_assert (rc != SOCKET_ERROR);

       }

       //  Create the writer socket.

    *w_ = WSASocket (AF_INET, SOCK_STREAM, 0, NULL, 0,  0);

    wsa_assert (*w_ != INVALID_SOCKET);

    //  On Windows, preventing sockets to be inherited by child
processes.

    BOOL brc = SetHandleInformation ((HANDLE) *w_, HANDLE_FLAG_INHERIT,
0);

    win_assert (brc);

    //  Set TCP_NODELAY on writer socket.

    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELAY,

        (char *)&tcp_nodelay, sizeof (tcp_nodelay));

    wsa_assert (rc != SOCKET_ERROR);

    //  Connect writer to the listener.

    rc = connect (*w_, (struct sockaddr*) &addr, sizeof (addr));

    wsa_assert (rc != SOCKET_ERROR);

    //  Accept connection from writer.

    *r_ = accept (listener, NULL, NULL);

    wsa_assert (*r_ != INVALID_SOCKET);

    //  On Windows, preventing sockets to be inherited by child
processes.

    brc = SetHandleInformation ((HANDLE) *r_, HANDLE_FLAG_INHERIT, 0);

    win_assert (brc);

    //  We don't need the listening socket anymore. Close it.

    //rc = closesocket (listener);

    //wsa_assert (rc != SOCKET_ERROR);

    //  Exit the critical section.

    brc = SetEvent (sync);

    win_assert (brc != 0);

    return 0;

#elif defined ZMQ_HAVE_OPENVMS

    //  Whilst OpenVMS supports socketpair - it maps to AF_INET only.
Further,

    //  it does not set the socket options TCP_NODELAY and TCP_NODELACK
which

    //  can lead to performance problems.

    //

    //  The bug will be fixed in V5.6 ECO4 and beyond.  In the meantime,
we'll

    //  create the socket pair manually.

    struct sockaddr_in lcladdr;

    memset (&lcladdr, 0, sizeof (lcladdr));

    lcladdr.sin_family = AF_INET;

    lcladdr.sin_addr.s_addr = htonl (INADDR_LOOPBACK);

    lcladdr.sin_port = 0;

    int listener = open_socket (AF_INET, SOCK_STREAM, 0);

    errno_assert (listener != -1);

    int on = 1;

    int rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELAY, &on, sizeof
(on));

    errno_assert (rc != -1);

    rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELACK, &on, sizeof
(on));

    errno_assert (rc != -1);

    rc = bind(listener, (struct sockaddr*) &lcladdr, sizeof (lcladdr));

    errno_assert (rc != -1);

    socklen_t lcladdr_len = sizeof (lcladdr);

    rc = getsockname (listener, (struct sockaddr*) &lcladdr,
&lcladdr_len);

    errno_assert (rc != -1);

    rc = listen (listener, 1);

    errno_assert (rc != -1);

    *w_ = open_socket (AF_INET, SOCK_STREAM, 0);

    errno_assert (*w_ != -1);

    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELAY, &on, sizeof (on));

    errno_assert (rc != -1);

    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELACK, &on, sizeof (on));

    errno_assert (rc != -1);

    rc = connect (*w_, (struct sockaddr*) &lcladdr, sizeof (lcladdr));

    errno_assert (rc != -1);

    *r_ = accept (listener, NULL, NULL);

    errno_assert (*r_ != -1);

    close (listener);

    return 0;

#else // All other implementations support socketpair()

    int sv [2];

    int rc = socketpair (AF_UNIX, SOCK_STREAM, 0, sv);

    errno_assert (rc == 0);

    *w_ = sv [0];

    *r_ = sv [1];

    return 0;

#endif

}

From: zeromq-dev-bounces at lists.zeromq.org
[mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Martin
Robinson
Sent: 22 March 2013 10:26
To: zeromq-dev at lists.zeromq.org
Subject: [zeromq-dev] Assert in signaller.cpp on XP if Cisco VPN Client
isinstalled

I've found an interesting issue which is causing me a bit of a
problem...

We have an application which we have recently re-written to use ZeroMQ
as a messaging layer (version 3.2.1).

It works very nicely, except when running on a Windows machine which
also has the Cisco VPN Client installed. The VPN client does not need to
be connected - it just needs to be installed for the problem to show up.

What I have found is that we get a crash (wsa_assert fail)  at line 310
in signaller.cpp. This is the connect call in
zmq::signaler_t::make_fdpair

The connect seems to fail with a 'Connection refused' error. Why on
earth would this connect fail? We have just opened the listener socket
successfully!

Curiously, it only fails the 2nd time this code is executed - the first
time works. I suspect this is something about the way the listener
socket is created, a connection accepted, then the socket is closed an
reopened again quickly.  There appears to be some strange interaction
between the 'Deterministic Network Enhancer' and winsock which causes
this code to fail.

Any ideas welcomed!

Thanks,

-- 
Martin Robinson
Senior Systems Engineer

DAI,

York House,  4 Wolsey Business Park

Tolpits Lane,  Watford,  WD18 9BL
Tel: +44 1923 815408   Fax: +44 1923 235786

Web: http://www.dai.co.uk/transport

The integrity of this message cannot be guaranteed on the

internet. DAI cannot therefore be considered responsible for the

contents, unless confirmed in written correspondence signed by

an authorised signatory. If you are not the intended recipient of

this message, please delete it and notify the sender.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130322/93eff2f0/attachment.htm>