[zeromq-dev] Custom refactoring of ZeroMQ to work with pintools...

Fri Jan 24 19:37:52 CET 2014

It must use pintools, because I had to refactor libzmq in order to play
nice with pintool; that's what the question is about. The 200 line program
perfectly replicates the issue of not being able to create a
router-proxy-dealer setup exactly as laid out in the manual; I get the same
error using that sample as in my currently closed source pintool.

The reason why I have to use a pintool to demonstrate the problem is
because I must give an example that shows the augmented libzmq operating,
and the augmented libzmq spawns threads using the PIN thread API (just as
the pintool manual specifies that thread control must in a pintool
executing environment). I just realized I didn't try to explain that thread
spawning and thread control primitives must be done through the PIN api;
basically because PIN is a dynamic instrumentation framework, if you spawn
a thread, even in the instrumentation or analysis routines that you define,
using the normal facilities, then PIN will have a recursive condition
against the thread that you innocently meant to spawn to help you do
analysis or instrumentation faster. In order to prevent this, they provide
the PIN thread APIs.

I like zeromq, and I would like to use it to coordinate my analysis and
instrumentation routines. The problem is, the zeromq library uses some
thread control and primitives that it shouldn't.

I'm sorry, but I can't make the example any less complicated by moving the
example out of a pintool context. I can try and eliminate some code
lines... If you have any questions about pintool itself, please feel free
to ask. I'm the one getting help from you guys in the long run, so I'm more
than happy to help you help me :-)

To be honest, I acknowledge that I'm bringing in a complex library that has
nothing to do with this mailing list itself, but this porting needs to
occur in order for the pintool community to benefit from the capabilities
that zmq offers. I'm highly motivated by zeromq's success and flexibility,
enough that I'm willing to dive an into unfamiliar code dig through and
find what I think needs to be changed and change it.

On Fri, Jan 24, 2014 at 12:02 PM, Pieter Hintjens <ph at imatix.com> wrote:

> I saw the repository and saw some code but a 200+ line program isn't
> really a minimal test case. As far as I understand, using pintools
> should be transparent to users, since it doesn't modify the ZMQ API,
> right? So you can make a minimal test case that doesn't require any
> knowledge of pintools to understand.
>
> On Fri, Jan 24, 2014 at 8:39 AM, Kenneth Adam Miller
> <kennethadammiller at gmail.com> wrote:
> > The example is on the github repo where it has PinTool-ErrorReplication.
> >
> >
> > On Fri, Jan 24, 2014 at 8:29 AM, Pieter Hintjens <ph at imatix.com> wrote:
> >>
> >> Hi Kenneth,
> >>
> >> Could you post a minimal example of the code that causes an error? If
> >> possible, in C, using the low level API. First step is to let us
> >> simply inspect this and check for obvious misuse of sockets.
> >>
> >> The "socket operation on non-socket" can happen when you pass an
> >> invalid argument to a zmq socket call, or when you are trying to use a
> >> closed socket.
> >>
> >> -Pieter
> >>
> >> On Fri, Jan 24, 2014 at 7:28 AM, Kenneth Adam Miller
> >> <kennethadammiller at gmail.com> wrote:
> >> > I would like to use zeromq to work with intel pintool (pintool.org).
> >> > Pintool
> >> > is a dynamic binary instrumentation framework; it allows the developer
> >> > to
> >> > specify some callbacks at many levels of execution of any binary
> >> > program.
> >> > But to allow zeromq to work with it, you have to restrict thread
> >> > creation
> >> > down to only the context or whatever other threads you have, and then
> >> > specify that the thread creation and any other thread apis go through
> >> > the
> >> > pin interface. If they don't then you potentially have a recursive
> >> > instrument and execute condition, where a thread you want to work for
> >> > the
> >> > instrumentation/analysis side is also being targeted for
> >> > instrumentation.
> >> >
> >> > My changes to zeromq consist of some minor alterations to thread.hpp
> and
> >> > mutex.hpp. You can see what I've changed on my forked repository over
> >> > here-all I do is replace the threading primitives that each class is
> >> > abstracting from with an instance for pin's.
> >> >
> >> > I know for certain that my changes are at taking effect, because what
> I
> >> > can
> >> > see is that inter thread communication for just ZMQ_PULL and ZMQ_PUSH
> >> > works
> >> > just fine. But when I go to create dealer's and router's and link them
> >> > with
> >> > a proxy I start getting errors with code that came straight from the
> >> > manual,
> >> > which is really not good. I would prefer that my hack continue to
> work,
> >> > and
> >> > I have confidence that with some help I can make it work.
> >> >
> >> > custom libzmq implementation:
> >> > https://github.com/KennethAdamMiller/libzmq
> >> >
> >> > ./autogen.sh
> >> >
> >> > (for 64 bit)
> >> > ./configure CXXFLAGS="-DBIGARRAY_MULTIPLIER=1 -Wall -Werror
> >> > -Wno-unknown-pragmas -fno-stack-protector -DTARGET_IA32E -DHOST_IA32E
> >> > -fPIC
> >> > -DTARGET_LINUX  -I$HOME/pin-2.13/source/include/pin
> >> > -I$HOME/pin-2.13/source/include/pin/gen
> >> > -I$HOME/pin-2.13/extras/components/include
> >> > -I$HOME/pin-2.13/extras/xed2-intel64/include
> >> > -I$HOME/pin-2.13/source/tools/InstLib -O3 -fomit-frame-pointer
> >> > -fno-strict-aliasing  -g -std=c++11 -L$HOMEpin-2.13/intel64/lib
> >> > -L$HOME/pin-2.13/intel64/lib-ext
> -L$HOME/pin-2.13/intel64/runtime/glibc
> >> > -L$HOME/pin-2.13/extras/xed2-intel64/lib -lpin -lxed -ldwarf -lelf
> -ldl
> >> > "
> >> >
> >> > sudo make install
> >> >
> >> > repeatable pintool error:
> >> > https://github.com/KennethAdamMiller/PinTool-ErrorReplication
> >> > Building this pintool is easy:
> >> > download a pin kit and extract to some location, like ~/pin-2.13
> >> >
> >> > export PIN_ROOT=location
> >> > export TOOLS_ROOT=$PIN_ROOT/source/tools
> >> > export CONFIG_ROOT=$TOOLS_ROOT/Config
> >> >
> >> > then just cd to where you checked out my tool and type make. You'll
> see
> >> > that
> >> > you get socket operation on non-socket. I was hoping that someone else
> >> > could
> >> > help me find out why it is getting that error, and then may suggest
> what
> >> > I
> >> > change in libzmq.
> >> >
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> >
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140124/836e405b/attachment.htm>