[zeromq-dev] Mac OS X: test_shutdown_stress sometimes fails

[Dhammika Pathirana]

Hi Martin,

On Sat, Nov 13, 2010 at 2:18 AM, Martin Sustrik <sustrik at 250bpm.com> wrote:
> On 11/13/2010 11:13 AM, Dhammika Pathirana wrote:
>
>> In finalize_initialization(), we detach the engine and pass it over to
>> app thread.
>
> Actually, it's passed to another I/O thread rather than to the app thread,
> but that's irrelevant to the problem.

correct, my bad.


>
>>             engine->unplug ();
>>             send_attach (session, engine, peer_identity, false);
>>             engine = NULL;
>>             terminate ();
>>
>> But there's a stack reference to this engine object,
>> engine->out_event() callback.
>
> How does the stack look like when the out_event() is called on destroyed
> engine?
>
>> I'll check if this shows up in valgrind.
>

I added following dbg prints,

@ -51,6 +51,7 @@ zmq::zmq_engine_t::zmq_engine_t (fd_t fd_, const
options_t &options_) :
 zmq::zmq_engine_t::~zmq_engine_t ()
 {
     zmq_assert (!plugged);
+    printf("~engine() %p\n", (void *)this);
 }

 void zmq::zmq_engine_t::plug (io_thread_t *io_thread_, i_inout *inout_)
@@ -160,6 +161,9 @@ void zmq::zmq_engine_t::out_event ()
         }
     }

+    if (!plugged)
+        printf("engine %p unplugged\n", (void *)this);
+

stdout just before crash,

engine 0x63e5130 unplugged
~engine() 0x63d3320
~engine() 0x63dbfa0
~engine() 0x63e5130
~engine() 0x63f7810
~engine() 0x63f3410
~engine() 0x63d7bb0
~engine() 0x63e0cd0
engine 0x63f7810 unplugged
Bad file descriptor
~engine() 0x63eefb0
nbytes != -1 (tcp_socket.cpp:197)
Killed

Note "~engine() 0x63f7810" before "engine 0x63f7810 unplugged" msg.
Here's stack trace,

(gdb) bt
#0  0x0000000005ec8a75 in *__GI_raise (sig=<value optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x0000000005ecc5c0 in *__GI_abort () at abort.c:92
#2  0x0000000004e97962 in zmq::tcp_socket_t::write (this=0x63f7828,
data=0x63f9a30, size=2) at tcp_socket.cpp:197
#3  0x0000000004e9fd3b in zmq::zmq_engine_t::out_event
(this=0x63f7810) at zmq_engine.cpp:169
#4  0x0000000004e76db3 in zmq::epoll_t::loop (this=0x639b1a0) at epoll.cpp:156
#5  0x0000000004e76ee2 in zmq::epoll_t::worker_routine
(arg_=0x639b1a0) at epoll.cpp:173
#6  0x0000000004e97c9a in zmq::thread_t::thread_routine
(arg_=0x639b210) at thread.cpp:79
#7  0x00000000054d09ca in start_thread (arg=<value optimized out>) at
pthread_create.c:300
#8  0x0000000005f7b70d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#9  0x0000000000000000 in ?? ()
(gdb) fr 3
#3  0x0000000004e9fd3b in zmq::zmq_engine_t::out_event
(this=0x63f7810) at zmq_engine.cpp:169
169         int nbytes = tcp_socket.write (outpos, outsize);
(gdb) p this
$1 = (zmq::zmq_engine_t * const) 0x63f7810


I could be reading this wrong, and I had to run it few dozen times to
get it to crash.


Dhammika