[zeromq-dev] Assert in signaller.cpp on XP if Cisco VPN Clientisinstalled - possible fix

[Martin Robinson]

I've had a stab at rewriting make_fdpair using ephemeral ports, and it
does indeed appear to solve the problem.

Only changes required from the 3.2.1 code are:

1.	set sin_port to 0 in the addr structure
2.	call getsockname after the bind, to read back the actual port to
connect to.

The code now looks as follows:

int zmq::signaler_t::make_fdpair (fd_t *r_, fd_t *w_)
{
#if defined ZMQ_HAVE_EVENTFD

    // Create eventfd object.
    fd_t fd = eventfd (0, 0);
    errno_assert (fd != -1);
    *w_ = fd;
    *r_ = fd;
    return 0;

#elif defined ZMQ_HAVE_WINDOWS
    SECURITY_DESCRIPTOR sd = {0};
    SECURITY_ATTRIBUTES sa = {0};

    InitializeSecurityDescriptor(&sd, SECURITY_DESCRIPTOR_REVISION);
    SetSecurityDescriptorDacl(&sd, TRUE, 0, FALSE);

    sa.nLength = sizeof(SECURITY_ATTRIBUTES);
    sa.lpSecurityDescriptor = &sd;

    //  This function has to be in a system-wide critical section so
that
    //  two instances of the library don't accidentally create signaler
    //  crossing the process boundary.
    //  We'll use named event object to implement the critical section.
    //  Note that if the event object already exists, the CreateEvent
requests
    //  EVENT_ALL_ACCESS access right. If this fails, we try to open
    //  the event object asking for SYNCHRONIZE access only.
    HANDLE sync = CreateEvent (&sa, FALSE, TRUE, TEXT
("Global\\zmq-signaler-port-sync"));
    if (sync == NULL && GetLastError () == ERROR_ACCESS_DENIED)
      sync = OpenEvent (SYNCHRONIZE | EVENT_MODIFY_STATE, FALSE, TEXT
("Global\\zmq-signaler-port-sync"));

    win_assert (sync != NULL);

    //  Enter the critical section.
    DWORD dwrc = WaitForSingleObject (sync, INFINITE);
    zmq_assert (dwrc == WAIT_OBJECT_0);

    //  Windows has no 'socketpair' function. CreatePipe is no good as
pipe
    //  handles cannot be polled on. Here we create the socketpair by
hand.
    *w_ = INVALID_SOCKET;
    *r_ = INVALID_SOCKET;

    //  Create listening socket - first time only
    static SOCKET listener = NULL;

	//  Bind listening socket to any free local port.
	struct sockaddr_in addr;
	memset (&addr, 0, sizeof (addr));
	addr.sin_family = AF_INET;
	addr.sin_addr.s_addr = htonl (INADDR_LOOPBACK);
	addr.sin_port = htons (0);
	int rc;
	BOOL tcp_nodelay = 1;

	listener = open_socket (AF_INET, SOCK_STREAM, 0);
    wsa_assert (listener != INVALID_SOCKET);

	//  Set SO_REUSEADDR and TCP_NODELAY on listening socket.
	BOOL so_reuseaddr = 1;
	rc = setsockopt (listener, SOL_SOCKET, SO_REUSEADDR,
		(char *)&so_reuseaddr, sizeof (so_reuseaddr));
	wsa_assert (rc != SOCKET_ERROR);
	rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELAY,
		(char *)&tcp_nodelay, sizeof (tcp_nodelay));
	wsa_assert (rc != SOCKET_ERROR);

	rc = bind (listener, (const struct sockaddr*) &addr, sizeof
(addr));
	wsa_assert (rc != SOCKET_ERROR);

	// Read back which port we bound too, so we can connect to it
	int namelen = sizeof(addr);
	rc = getsockname (listener, (struct sockaddr*) &addr, &namelen);
	wsa_assert (rc != SOCKET_ERROR);

	//  Listen for incomming connections.
	rc = listen (listener, 1);
	wsa_assert (rc != SOCKET_ERROR);

	//  Create the writer socket.
    *w_ = WSASocket (AF_INET, SOCK_STREAM, 0, NULL, 0,  0);
    wsa_assert (*w_ != INVALID_SOCKET);

    //  On Windows, preventing sockets to be inherited by child
processes.
    BOOL brc = SetHandleInformation ((HANDLE) *w_, HANDLE_FLAG_INHERIT,
0);
    win_assert (brc);

    //  Set TCP_NODELAY on writer socket.
    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELAY,
        (char *)&tcp_nodelay, sizeof (tcp_nodelay));
    wsa_assert (rc != SOCKET_ERROR);

    //  Connect writer to the listener.
    rc = connect (*w_, (struct sockaddr*) &addr, sizeof (addr));
    wsa_assert (rc != SOCKET_ERROR);

    //  Accept connection from writer.
    *r_ = accept (listener, NULL, NULL);
    wsa_assert (*r_ != INVALID_SOCKET);

    //  On Windows, preventing sockets to be inherited by child
processes.
    brc = SetHandleInformation ((HANDLE) *r_, HANDLE_FLAG_INHERIT, 0);
    win_assert (brc);

    //  We don't need the listening socket anymore. Close it.
    rc = closesocket (listener);
    wsa_assert (rc != SOCKET_ERROR);

    //  Exit the critical section.
    brc = SetEvent (sync);
    win_assert (brc != 0);

    return 0;

#elif defined ZMQ_HAVE_OPENVMS

    //  Whilst OpenVMS supports socketpair - it maps to AF_INET only.
Further,
    //  it does not set the socket options TCP_NODELAY and TCP_NODELACK
which
    //  can lead to performance problems.
    //
    //  The bug will be fixed in V5.6 ECO4 and beyond.  In the meantime,
we'll
    //  create the socket pair manually.
    struct sockaddr_in lcladdr;
    memset (&lcladdr, 0, sizeof (lcladdr));
    lcladdr.sin_family = AF_INET;
    lcladdr.sin_addr.s_addr = htonl (INADDR_LOOPBACK);
    lcladdr.sin_port = 0;

    int listener = open_socket (AF_INET, SOCK_STREAM, 0);
    errno_assert (listener != -1);

    int on = 1;
    int rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELAY, &on, sizeof
(on));
    errno_assert (rc != -1);

    rc = setsockopt (listener, IPPROTO_TCP, TCP_NODELACK, &on, sizeof
(on));
    errno_assert (rc != -1);

    rc = bind(listener, (struct sockaddr*) &lcladdr, sizeof (lcladdr));
    errno_assert (rc != -1);

    socklen_t lcladdr_len = sizeof (lcladdr);

    rc = getsockname (listener, (struct sockaddr*) &lcladdr,
&lcladdr_len);
    errno_assert (rc != -1);

    rc = listen (listener, 1);
    errno_assert (rc != -1);

    *w_ = open_socket (AF_INET, SOCK_STREAM, 0);
    errno_assert (*w_ != -1);

    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELAY, &on, sizeof (on));
    errno_assert (rc != -1);

    rc = setsockopt (*w_, IPPROTO_TCP, TCP_NODELACK, &on, sizeof (on));
    errno_assert (rc != -1);

    rc = connect (*w_, (struct sockaddr*) &lcladdr, sizeof (lcladdr));
    errno_assert (rc != -1);

    *r_ = accept (listener, NULL, NULL);
    errno_assert (*r_ != -1);

    close (listener);

    return 0;

#else // All other implementations support socketpair()

    int sv [2];
    int rc = socketpair (AF_UNIX, SOCK_STREAM, 0, sv);
    errno_assert (rc == 0);
    *w_ = sv [0];
    *r_ = sv [1];
    return 0;

#endif
}

-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org
[mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Martin
Robinson
Sent: 22 March 2013 15:51
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Assert in signaller.cpp on XP if Cisco VPN
Clientisinstalled - possible fix

I'm pretty sure it failed with ECONREFUSED and returned an error message
'Connection refused'. 

I suspect using ephemeral ports would probably solve the issue. I might
have a play with that in due course, because as KIU Shueng Chuan pointed
out, my solution will blow up if more than one ZeroMQ process was
running at once.

I think this is all caused by some strange timing issue, as if the old
listening socket is not really deleted immediately, such that the
connect() call sees the dying socket instead of the newly created one. I
did try putting in a Sleep(1000) before the connect, and it seemed to
make no difference.

Martin

-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org
[mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
Hintjens
Sent: 22 March 2013 15:27
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Assert in signaller.cpp on XP if Cisco VPN
Client isinstalled - possible fix

Could you get the detailed error message from the failed connect() call?

The code does look accurate so it might be a Windows weirdness -
SO_REUSEADDR not working.

It should be easy to isolate and test with a stand-alone piece of code.

If there's no other solution then we might use ephemeral ports instead
of reusing the same listening port over and over.

-Pieter
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev