[zeromq-dev] CZMQ zsys layer

Stephen Gray riskybizlive at live.com
Mon Jan 22 16:41:22 CET 2018

Hi everyone,

Lately I've been using CZMQ on Windows developing  .dlls as custom-user-extensions to a proprietary-app.   The proprietary-app is able to launch as many processes as it deems necessary;  each process will have a copy of the relevant dll and (I think) issue a copy to each thread as appropriate. The end result is multiple instances of classes using the CZMQ API, executing on different threads in the same process.

To my understanding there is only one instance of the CZMQ zsys_layer per process, no matter how many times zsys_init() gets called.

Here comes the tricky part......those multiple class instances need to come and go on their own terms.  If any one of them calls zsys_shutdown() then the zsys_layer instance serving all the other instances on threads in the process  will  be terminated and the system will crash.  If zsys_shutdown is not called at all then when the process terminates it will hang and be unsatisfactory.

With this setup I encountered termination/destruction issues which were somewhat tricky to isolate and solve; happily they are solved :)

I solved the issue by using boost::interprocess::managed_shared_memory with named-shared-memory-objects and the "open_or_create" functionality of boost::interprocess. With a boost::interprocess::named_mutex for good measure.  The shared memory segment is named using the value from GetCurrentProcess() from <windows.h>.  Therefore any thread of the same process can find the relevant shared memory.  In the shared memory is a std::atomic<int> holding the instance count.  At instantiation each thread increments the instance count, at termination each thread decrements the instance count.  If the instance count hits zero then it is safe to call zsys_shutdown().  Last one out; turns off the lights.

In the challenging environment of working all this out it struck me that the functionality and interface of zsys_shutdown could be improved to cater for other situations of similar multi-threaded complexity.

I would like to open the topic for discussion, but wonder if it best done here on the mailing list or better done on GitHub.  Not sure how really.

My feature suggestions are;

1.       Permit multiple instances of zsys_layer per-process.  Currently zsys_init() returns a pointer.  Use this pointer to permit targeted shutdown of a particular zsys_layer instance by changing the interface to; zsys_shutdown(ACCEPT_POINTER).

2.       Keep a record of user threads calling into the zsys_layer. On zys_shutdown if another user thread remains active then keep the zsys_layer operational. i.e. zsys_shutdown(SOFT_OPTION) would not have any effect in this circumstance.  Keep zsys_shutdown() as the HARD_WAY to brutally terminate the zsys_layer.

Either of these options would negate the need for my shared memory work-around in this complex case.

If someone deeply familiar with the zsys_layer could comment on this it would be awesome.  My C skills aren't up to making a pull request though, I work in C++.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20180122/ab7eb97b/attachment.htm>

More information about the zeromq-dev mailing list