[zeromq-dev] Custom refactoring of ZeroMQ to work with pintools...

Kenneth Adam Miller kennethadammiller at gmail.com
Fri Jan 24 19:57:17 CET 2014


In that case, it would be a separately named library; the developers of
their application wouldn't execute their application in a pintool context,
so they would use the vanilla library, because they would never be able to
get their application off the ground due to link errors from undefined
symbols and runtime unresolved references. They would never think to
themselves "I'm going to run my normal everyday application which uses
regular threads in a regular environment with an obscure variant of
zeromq." As a result, the vanilla zeromq would spawn threads the normal
way, and hence they would get instrumented. Sure you would have some
unnecessary data/space consumption from replicated

This particular library could just be named to something like libpinzmq or
libzmq-pintool. Then it should work fine.


On Fri, Jan 24, 2014 at 12:51 PM, Lindley French <lindleyf at gmail.com> wrote:

> Ideally, it would be possible to use ZMQ simultaneously inside of and
> outside of a pintool context. I'm not familiar with pintools, but if it's
> an instrumentation framework, what if they code you're trying to instrument
> *also* uses ZMQ?
>
>
> On Fri, Jan 24, 2014 at 1:44 PM, Kenneth Adam Miller <
> kennethadammiller at gmail.com> wrote:
>
>> No; It calls PIN_SpawnInternalThread, which only makes sense in a
>> pintool's executing environment. Basically, if you use this particular
>> modified version of zmq, you specifically want to use zmq in a pintool
>> context.
>>
>> I wouldn't say my changes are "harm", they are what I see as necessary in
>> order to realize pintool utilization of zeromq. I'm just not  as familiar
>> with the zeromq code base as perhaps you guys are. :-)
>>
>>
>> On Fri, Jan 24, 2014 at 12:42 PM, Lindley French <lindleyf at gmail.com>wrote:
>>
>>> Question: Does your modified code work properly outside of a pintool
>>> context? EG, does it at least pass the "do no harm" test, before you worry
>>> about whether it does what you want it to do?
>>>
>>>
>>> On Fri, Jan 24, 2014 at 1:37 PM, Kenneth Adam Miller <
>>> kennethadammiller at gmail.com> wrote:
>>>
>>>> It must use pintools, because I had to refactor libzmq in order to play
>>>> nice with pintool; that's what the question is about. The 200 line program
>>>> perfectly replicates the issue of not being able to create a
>>>> router-proxy-dealer setup exactly as laid out in the manual; I get the same
>>>> error using that sample as in my currently closed source pintool.
>>>>
>>>> The reason why I have to use a pintool to demonstrate the problem is
>>>> because I must give an example that shows the augmented libzmq operating,
>>>> and the augmented libzmq spawns threads using the PIN thread API (just as
>>>> the pintool manual specifies that thread control must in a pintool
>>>> executing environment). I just realized I didn't try to explain that thread
>>>> spawning and thread control primitives must be done through the PIN api;
>>>> basically because PIN is a dynamic instrumentation framework, if you spawn
>>>> a thread, even in the instrumentation or analysis routines that you define,
>>>> using the normal facilities, then PIN will have a recursive condition
>>>> against the thread that you innocently meant to spawn to help you do
>>>> analysis or instrumentation faster. In order to prevent this, they provide
>>>> the PIN thread APIs.
>>>>
>>>> I like zeromq, and I would like to use it to coordinate my analysis and
>>>> instrumentation routines. The problem is, the zeromq library uses some
>>>> thread control and primitives that it shouldn't.
>>>>
>>>> I'm sorry, but I can't make the example any less complicated by moving
>>>> the example out of a pintool context. I can try and eliminate some code
>>>> lines... If you have any questions about pintool itself, please feel free
>>>> to ask. I'm the one getting help from you guys in the long run, so I'm more
>>>> than happy to help you help me :-)
>>>>
>>>> To be honest, I acknowledge that I'm bringing in a complex library that
>>>> has nothing to do with this mailing list itself, but this porting needs to
>>>> occur in order for the pintool community to benefit from the capabilities
>>>> that zmq offers. I'm highly motivated by zeromq's success and flexibility,
>>>> enough that I'm willing to dive an into unfamiliar code dig through and
>>>> find what I think needs to be changed and change it.
>>>>
>>>>
>>>> On Fri, Jan 24, 2014 at 12:02 PM, Pieter Hintjens <ph at imatix.com>wrote:
>>>>
>>>>> I saw the repository and saw some code but a 200+ line program isn't
>>>>> really a minimal test case. As far as I understand, using pintools
>>>>> should be transparent to users, since it doesn't modify the ZMQ API,
>>>>> right? So you can make a minimal test case that doesn't require any
>>>>> knowledge of pintools to understand.
>>>>>
>>>>> On Fri, Jan 24, 2014 at 8:39 AM, Kenneth Adam Miller
>>>>> <kennethadammiller at gmail.com> wrote:
>>>>> > The example is on the github repo where it has
>>>>> PinTool-ErrorReplication.
>>>>> >
>>>>> >
>>>>> > On Fri, Jan 24, 2014 at 8:29 AM, Pieter Hintjens <ph at imatix.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Hi Kenneth,
>>>>> >>
>>>>> >> Could you post a minimal example of the code that causes an error?
>>>>> If
>>>>> >> possible, in C, using the low level API. First step is to let us
>>>>> >> simply inspect this and check for obvious misuse of sockets.
>>>>> >>
>>>>> >> The "socket operation on non-socket" can happen when you pass an
>>>>> >> invalid argument to a zmq socket call, or when you are trying to
>>>>> use a
>>>>> >> closed socket.
>>>>> >>
>>>>> >> -Pieter
>>>>> >>
>>>>> >> On Fri, Jan 24, 2014 at 7:28 AM, Kenneth Adam Miller
>>>>> >> <kennethadammiller at gmail.com> wrote:
>>>>> >> > I would like to use zeromq to work with intel pintool (
>>>>> pintool.org).
>>>>> >> > Pintool
>>>>> >> > is a dynamic binary instrumentation framework; it allows the
>>>>> developer
>>>>> >> > to
>>>>> >> > specify some callbacks at many levels of execution of any binary
>>>>> >> > program.
>>>>> >> > But to allow zeromq to work with it, you have to restrict thread
>>>>> >> > creation
>>>>> >> > down to only the context or whatever other threads you have, and
>>>>> then
>>>>> >> > specify that the thread creation and any other thread apis go
>>>>> through
>>>>> >> > the
>>>>> >> > pin interface. If they don't then you potentially have a recursive
>>>>> >> > instrument and execute condition, where a thread you want to work
>>>>> for
>>>>> >> > the
>>>>> >> > instrumentation/analysis side is also being targeted for
>>>>> >> > instrumentation.
>>>>> >> >
>>>>> >> > My changes to zeromq consist of some minor alterations to
>>>>> thread.hpp and
>>>>> >> > mutex.hpp. You can see what I've changed on my forked repository
>>>>> over
>>>>> >> > here-all I do is replace the threading primitives that each class
>>>>> is
>>>>> >> > abstracting from with an instance for pin's.
>>>>> >> >
>>>>> >> > I know for certain that my changes are at taking effect, because
>>>>> what I
>>>>> >> > can
>>>>> >> > see is that inter thread communication for just ZMQ_PULL and
>>>>> ZMQ_PUSH
>>>>> >> > works
>>>>> >> > just fine. But when I go to create dealer's and router's and link
>>>>> them
>>>>> >> > with
>>>>> >> > a proxy I start getting errors with code that came straight from
>>>>> the
>>>>> >> > manual,
>>>>> >> > which is really not good. I would prefer that my hack continue to
>>>>> work,
>>>>> >> > and
>>>>> >> > I have confidence that with some help I can make it work.
>>>>> >> >
>>>>> >> > custom libzmq implementation:
>>>>> >> > https://github.com/KennethAdamMiller/libzmq
>>>>> >> >
>>>>> >> > ./autogen.sh
>>>>> >> >
>>>>> >> > (for 64 bit)
>>>>> >> > ./configure CXXFLAGS="-DBIGARRAY_MULTIPLIER=1 -Wall -Werror
>>>>> >> > -Wno-unknown-pragmas -fno-stack-protector -DTARGET_IA32E
>>>>> -DHOST_IA32E
>>>>> >> > -fPIC
>>>>> >> > -DTARGET_LINUX  -I$HOME/pin-2.13/source/include/pin
>>>>> >> > -I$HOME/pin-2.13/source/include/pin/gen
>>>>> >> > -I$HOME/pin-2.13/extras/components/include
>>>>> >> > -I$HOME/pin-2.13/extras/xed2-intel64/include
>>>>> >> > -I$HOME/pin-2.13/source/tools/InstLib -O3 -fomit-frame-pointer
>>>>> >> > -fno-strict-aliasing  -g -std=c++11 -L$HOMEpin-2.13/intel64/lib
>>>>> >> > -L$HOME/pin-2.13/intel64/lib-ext
>>>>> -L$HOME/pin-2.13/intel64/runtime/glibc
>>>>> >> > -L$HOME/pin-2.13/extras/xed2-intel64/lib -lpin -lxed -ldwarf
>>>>> -lelf -ldl
>>>>> >> > "
>>>>> >> >
>>>>> >> > sudo make install
>>>>> >> >
>>>>> >> > repeatable pintool error:
>>>>> >> > https://github.com/KennethAdamMiller/PinTool-ErrorReplication
>>>>> >> > Building this pintool is easy:
>>>>> >> > download a pin kit and extract to some location, like ~/pin-2.13
>>>>> >> >
>>>>> >> > export PIN_ROOT=location
>>>>> >> > export TOOLS_ROOT=$PIN_ROOT/source/tools
>>>>> >> > export CONFIG_ROOT=$TOOLS_ROOT/Config
>>>>> >> >
>>>>> >> > then just cd to where you checked out my tool and type make.
>>>>> You'll see
>>>>> >> > that
>>>>> >> > you get socket operation on non-socket. I was hoping that someone
>>>>> else
>>>>> >> > could
>>>>> >> > help me find out why it is getting that error, and then may
>>>>> suggest what
>>>>> >> > I
>>>>> >> > change in libzmq.
>>>>> >> >
>>>>> >> > _______________________________________________
>>>>> >> > zeromq-dev mailing list
>>>>> >> > zeromq-dev at lists.zeromq.org
>>>>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> >> >
>>>>> >> _______________________________________________
>>>>> >> zeromq-dev mailing list
>>>>> >> zeromq-dev at lists.zeromq.org
>>>>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > zeromq-dev mailing list
>>>>> > zeromq-dev at lists.zeromq.org
>>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>> >
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140124/c111f57b/attachment.htm>


More information about the zeromq-dev mailing list