[zeromq-dev] Mac OS X: test_shutdown_stress sometimes fails

[Martin Sustrik]

Hi Dhammika,

I am trying to make sense from this, but I still don't see how it can 
happen. Can you possibly run the same test with printing "unplugged" in 
zmq_engine_t::unplug and "plugged" in zmq_engine_t::plug?

That way we'll be sure that engine is deallocated while it's plugged in.

Thanks.
Martin

On 11/14/2010 02:47 AM, Dhammika Pathirana wrote:
> Hi Martin,
>
> On Sat, Nov 13, 2010 at 2:18 AM, Martin Sustrik<sustrik at 250bpm.com>  wrote:
>> On 11/13/2010 11:13 AM, Dhammika Pathirana wrote:
>>
>>> In finalize_initialization(), we detach the engine and pass it over to
>>> app thread.
>>
>> Actually, it's passed to another I/O thread rather than to the app thread,
>> but that's irrelevant to the problem.
>
> correct, my bad.
>
>
>>
>>>              engine->unplug ();
>>>              send_attach (session, engine, peer_identity, false);
>>>              engine = NULL;
>>>              terminate ();
>>>
>>> But there's a stack reference to this engine object,
>>> engine->out_event() callback.
>>
>> How does the stack look like when the out_event() is called on destroyed
>> engine?
>>
>>> I'll check if this shows up in valgrind.
>>
>
> I added following dbg prints,
>
> @ -51,6 +51,7 @@ zmq::zmq_engine_t::zmq_engine_t (fd_t fd_, const
> options_t&options_) :
>   zmq::zmq_engine_t::~zmq_engine_t ()
>   {
>       zmq_assert (!plugged);
> +    printf("~engine() %p\n", (void *)this);
>   }
>
>   void zmq::zmq_engine_t::plug (io_thread_t *io_thread_, i_inout *inout_)
> @@ -160,6 +161,9 @@ void zmq::zmq_engine_t::out_event ()
>           }
>       }
>
> +    if (!plugged)
> +        printf("engine %p unplugged\n", (void *)this);
> +
>
> stdout just before crash,
>
> engine 0x63e5130 unplugged
> ~engine() 0x63d3320
> ~engine() 0x63dbfa0
> ~engine() 0x63e5130
> ~engine() 0x63f7810
> ~engine() 0x63f3410
> ~engine() 0x63d7bb0
> ~engine() 0x63e0cd0
> engine 0x63f7810 unplugged
> Bad file descriptor
> ~engine() 0x63eefb0
> nbytes != -1 (tcp_socket.cpp:197)
> Killed
>
> Note "~engine() 0x63f7810" before "engine 0x63f7810 unplugged" msg.
> Here's stack trace,
>
> (gdb) bt
> #0  0x0000000005ec8a75 in *__GI_raise (sig=<value optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1  0x0000000005ecc5c0 in *__GI_abort () at abort.c:92
> #2  0x0000000004e97962 in zmq::tcp_socket_t::write (this=0x63f7828,
> data=0x63f9a30, size=2) at tcp_socket.cpp:197
> #3  0x0000000004e9fd3b in zmq::zmq_engine_t::out_event
> (this=0x63f7810) at zmq_engine.cpp:169
> #4  0x0000000004e76db3 in zmq::epoll_t::loop (this=0x639b1a0) at epoll.cpp:156
> #5  0x0000000004e76ee2 in zmq::epoll_t::worker_routine
> (arg_=0x639b1a0) at epoll.cpp:173
> #6  0x0000000004e97c9a in zmq::thread_t::thread_routine
> (arg_=0x639b210) at thread.cpp:79
> #7  0x00000000054d09ca in start_thread (arg=<value optimized out>) at
> pthread_create.c:300
> #8  0x0000000005f7b70d in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
> #9  0x0000000000000000 in ?? ()
> (gdb) fr 3
> #3  0x0000000004e9fd3b in zmq::zmq_engine_t::out_event
> (this=0x63f7810) at zmq_engine.cpp:169
> 169         int nbytes = tcp_socket.write (outpos, outsize);
> (gdb) p this
> $1 = (zmq::zmq_engine_t * const) 0x63f7810
>
>
> I could be reading this wrong, and I had to run it few dozen times to
> get it to crash.
>
>
> Dhammika