[zeromq-dev] czmq vs forked process

Jay Rolette rolette at infinite.io
Thu Dec 22 13:55:21 CET 2016


It's an inline network processing application built on top of DPDK. Init is
an involved process. There is some relatively early processing that happens
in the parent process that ends up making some calls through czmq to get
some parameters used to initialize DPDK's environment abstraction layer
(EAL). Until EAL initialization is complete, we can't fork the child
process off.

The child process is used as a bridge to communicate with the deep packet
inspection engine from non-DPDK applications (control plane apps).

The app could be refactored to get around this, but there are a lot of
moving parts involved, so it would be a risky change to something that
already works. It's a shipping product and ZeroMQ wasn't part of the
architecture originally. I'm retrofitting it under the covers as part of
our configuration subsystem.

If czmq doesn't work out for this, I'd just fall back to using libzmq
directly, but czmq is attractive.

Jay

On Thu, Dec 22, 2016 at 3:29 AM, Kevin Sapper <kevinsapper88 at gmail.com>
wrote:

> Hi Jay,
>
> czmq will create the context for you whenever you touch the sockets. So as
> you already figured out fork before using czmq sockets. I cannot recommend
> using zsys_shutdown() to force destruction of the context as there might be
> side effects. What's your use case anyhow?
>
> //Kevin
>
>
> On Mi, Dez 21, 2016 at 10:42 , Jay Rolette <rolette at infinite.io> wrote:
>
> *> If I call zsys_shutdown() in the child process immediately after the
> fork (or, at least, before I try to do anything else with czmq), then czmq
> reinitializes itself, so the child gets a new context and the IO threads
> are created.*
> *>*
> *> Still more testing needed to make sure it works reliably, but initial
> tests are positive. If anyone knows of issues with calling zsys_shutdown()
> like this, I'd love to hear about it.*
>
> Minor patch required to make this work. zsys_init() fails a safety check:
> assert(!s_process_ctx). If I modify zsys_shutdown() to set s_process_ctx =
> NULL after the zmq_term() call, then it works.
>
> I'll see about putting together a pull request unless someone can point me
> to a better answer using czmq.
>
> Jay
>
> On Wed, Dec 21, 2016 at 12:26 PM, Jay Rolette <rolette at infinite.io> wrote:
>
>> Hi Kevin,
>>
>> Thanks. I've read the guide and that all makes sense, but in this case,
>> I'm trying to use czmq as a higher-level API. In it's latest incarnation,
>> it doesn't expose the zmq context at all.
>>
>> I'll go back and rewrite my code to just use the base zmq APIs if
>> necessary, but I'm hoping to stick with czmq for now.
>>
>> It looks like I may have found a solution that fits within czmq...
>>
>> If I call zsys_shutdown() in the child process immediately after the fork
>> (or, at least, before I try to do anything else with czmq), then czmq
>> reinitializes itself, so the child gets a new context and the IO threads
>> are created.
>>
>> Still more testing needed to make sure it works reliably, but initial
>> tests are positive. If anyone knows of issues with calling zsys_shutdown()
>> like this, I'd love to hear about it.
>>
>> Thanks,
>> Jay
>>
>> On Wed, Dec 21, 2016 at 11:49 AM, Kevin Sapper <kevinsapper88 at gmail.com>
>> wrote:
>>
>>> Hi Jay,
>>>
>>> this is for you http://zguide.zeromq.org/page:all#Getting-the-Context-Ri
>>> ght
>>>
>>> //Kevin
>>>
>>> On Mi, Dez 21, 2016 at 4:49 , Jay Rolette <rolette at infinite.io> wrote:
>>>
>>> I recently started using czmq and have run into a problem with the
>>> global zmq context it uses when the process forks...
>>>
>>> If the parent process does anything to cause the global context to be
>>> created prior to forking, the child process is unable to send/recv
>>> messages. That's not too surprising since zmq contexts aren't shareable
>>> across processes, but I've been looking through the czmq code for a way to
>>> destroy/recreate the global context in the child process and not finding an
>>> obvious way to do that.
>>>
>>> In older versions of czmq, it looks like I probably could have used
>>> zctx_* calls to create a new context, but those were removed. Not seeing
>>> anything similar in zsock_* or zsys_*.
>>>
>>> Details about the behavior I'm seeing in the child process:
>>>
>>> zmsg_send() returns success, but nothing is received on the server side.
>>> On the client, the subsequent call to zmsg_recv() never returns.
>>>
>>> This is on Ubuntu 14.04 LTS, czmq 4.0.1, libzmq 4.2 with pretty basic
>>> REQ/REP sockets.
>>>
>>> Is there anyway to stick with czmq for this or do I have to back off to
>>> vanilla zmq?
>>>
>>> Thanks,
>>> Jay
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20161222/15175547/attachment.htm>


More information about the zeromq-dev mailing list