[zeromq-dev] 0MQ-based proxy worker crashes with "Assertion failed: pipe (bundled/zeromq/src/session_base.cpp:441)"

[Tomas Krajca]

Thanks, that's a good idea, I'll give that a go on Monday.

Tomas

> On 13 Sep 2014, at 1:59 am, Martin Hurton <hurtonm at gmail.com> wrote:
> 
> Hi Tomas, can you please check with the master and report back? Thanks.
> 
>> On Sep 9, 2014 3:44 AM, "Tomas Krajca" <t.l.krajca at gmail.com> wrote:
>> Hi,
>> 
>> I've got a 0MQ-based proxy, clients talk 0MQ to the proxy, the proxy then talks HTTP to do either a GET on a specific url endpoint or a POST on a specific endpoint (it always goes to one of these two url endpoints). I've got a master process that has a zmq.ROUTER towards its clients (zmq.REQ) and a zmq.DEALER towards its workers (zmq.REP). The master is a single process, no threading, normal 0MQ, it spawns a worker processes via multiprocessing, this worker process uses gevent and zmq.green to spawn the actual (green) workers (those use grequests to talk HTTP). The master uses 0MQ auth to authenticate its clients. It should all be pretty standard but note that this is my first gevent/zmq.green based project.
>> 
>> So this proxy runs pretty well untill the worker process (I run only 1 worker process) crashes with 'Assertion failed: pipe (bundled/zeromq/src/session_base.cpp:441)"' on its stderr. There is nothing else in the logs or on stdout that would give me any more idea of what is going on. I can see the master running and netcat to its zmq.ROUTER so its definitely the worker that dies. Sorry, I have no idea how to reproduce this, once it crashed after 5 hours of working nicely, second time it crashed after about a day.
>> 
>> Here is a snippet of the worker code (the relevant bits): http://pastebin.com/usi0FXDL
>> 
>> The STSDBResponder uses grequests to do the HTTP, there should be nothing special about that.
>> 
>> This happens on CentOS 6.5, the proxy is running in virtualenv (pip install pyzmq):
>> 
>> Python 2.7.6 (default, Jul 10 2014, 04:59:13) 
>> [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import zmq
>> >>> zmq.zmq_version()
>> '4.0.4'
>> >>> zmq.__version__
>> '14.3.1'
>> >>> 
>> 
>> I have no idea whether this is a libzmq bug or pyzmq bug or a bug in my code or a system misconfiguration (do I need to increase ulimit or something?), I run 64 gevent threads. I tried to see session_base.cpp but it didn't help me understand why this could happen either.
>> 
>> If anybody could please point me to a direction as to why the worker crashes, it would be much appreciated.
>> 
>> Thanks,
>> Tomas
>> 
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140913/5bd2a68d/attachment.htm>