[zeromq-dev] 0MQ-based proxy worker crashes with "Assertion failed: pipe (bundled/zeromq/src/session_base.cpp:441)"
Pieter Hintjens
ph at imatix.com
Fri Sep 12 09:17:07 CEST 2014
Is there any way you can cut down your code to a minimal case that you
can get to crash? That is usually the best way to get the problem
resolved.
On Fri, Sep 12, 2014 at 8:26 AM, Tomas Krajca <t.l.krajca at gmail.com> wrote:
> Does anybody have any idea about the original proxy crash? The proxy did about 80000 requests just fine today and then it crashed with the pipe assertion again, really weird.
>
> Thanks,
> Tomas
>
>> On 10 Sep 2014, at 10:51 am, Tomas Krajca <t.l.krajca at gmail.com> wrote:
>>
>> Thanks Justin, zurl is definitely worth looking at.
>>
>> Regards,
>> Tomas
>>
>>> On 9 Sep 2014, at 12:35 pm, Justin Karneges <justin at affinix.com> wrote:
>>>
>>> Hi Tomas,
>>>
>>> This does not answer your question at all, but you might be interested
>>> in the Zurl project. It is a 0MQ daemon that does HTTP requests. You can
>>> speak to it with REQ/REP.
>>>
>>> https://github.com/fanout/zurl
>>>
>>>> On 09/08/2014 06:44 PM, Tomas Krajca wrote:
>>>> Hi,
>>>>
>>>> I've got a 0MQ-based proxy, clients talk 0MQ to the proxy, the proxy
>>>> then talks HTTP to do either a GET on a specific url endpoint or a POST
>>>> on a specific endpoint (it always goes to one of these two url
>>>> endpoints). I've got a master process that has a zmq.ROUTER towards its
>>>> clients (zmq.REQ) and a zmq.DEALER towards its workers (zmq.REP). The
>>>> master is a single process, no threading, normal 0MQ, it spawns a worker
>>>> processes via multiprocessing, this worker process uses gevent and
>>>> zmq.green to spawn the actual (green) workers (those use grequests to
>>>> talk HTTP). The master uses 0MQ auth to authenticate its clients. It
>>>> should all be pretty standard but note that this is my first
>>>> gevent/zmq.green based project.
>>>>
>>>> So this proxy runs pretty well untill the worker process (I run only 1
>>>> worker process) crashes with 'Assertion failed: pipe
>>>> (bundled/zeromq/src/session_base.cpp:441)"' on its stderr. There is
>>>> nothing else in the logs or on stdout that would give me any more idea
>>>> of what is going on. I can see the master running and netcat to its
>>>> zmq.ROUTER so its definitely the worker that dies. Sorry, I have no idea
>>>> how to reproduce this, once it crashed after 5 hours of working nicely,
>>>> second time it crashed after about a day.
>>>>
>>>> Here is a snippet of the worker code (the relevant bits):
>>>> http://pastebin.com/usi0FXDL
>>>>
>>>> The STSDBResponder uses grequests to do the HTTP, there should be
>>>> nothing special about that.
>>>>
>>>> This happens on CentOS 6.5, the proxy is running in virtualenv (pip
>>>> install pyzmq):
>>>>
>>>> Python 2.7.6 (default, Jul 10 2014, 04:59:13)
>>>> [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>>> import zmq
>>>>>>> zmq.zmq_version()
>>>> '4.0.4'
>>>>>>> zmq.__version__
>>>> '14.3.1'
>>>>
>>>> I have no idea whether this is a libzmq bug or pyzmq bug or a bug in my
>>>> code or a system misconfiguration (do I need to increase ulimit or
>>>> something?), I run 64 gevent threads. I tried to see session_base.cpp
>>>> but it didn't help me understand why this could happen either.
>>>>
>>>> If anybody could please point me to a direction as to why the worker
>>>> crashes, it would be much appreciated.
>>>>
>>>> Thanks,
>>>> Tomas
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
More information about the zeromq-dev
mailing list