[zeromq-dev] Custom refactoring of ZeroMQ to work with pintools...

Fri Jan 24 23:51:30 CET 2014

Is anybody still willing to work on this with me?

On Fri, Jan 24, 2014 at 12:57 PM, Kenneth Adam Miller <
kennethadammiller at gmail.com> wrote:

> In that case, it would be a separately named library; the developers of
> their application wouldn't execute their application in a pintool context,
> so they would use the vanilla library, because they would never be able to
> get their application off the ground due to link errors from undefined
> symbols and runtime unresolved references. They would never think to
> themselves "I'm going to run my normal everyday application which uses
> regular threads in a regular environment with an obscure variant of
> zeromq." As a result, the vanilla zeromq would spawn threads the normal
> way, and hence they would get instrumented. Sure you would have some
> unnecessary data/space consumption from replicated
>
> This particular library could just be named to something like libpinzmq or
> libzmq-pintool. Then it should work fine.
>
>
> On Fri, Jan 24, 2014 at 12:51 PM, Lindley French <lindleyf at gmail.com>wrote:
>
>> Ideally, it would be possible to use ZMQ simultaneously inside of and
>> outside of a pintool context. I'm not familiar with pintools, but if it's
>> an instrumentation framework, what if they code you're trying to instrument
>> *also* uses ZMQ?
>>
>>
>> On Fri, Jan 24, 2014 at 1:44 PM, Kenneth Adam Miller <
>> kennethadammiller at gmail.com> wrote:
>>
>>> No; It calls PIN_SpawnInternalThread, which only makes sense in a
>>> pintool's executing environment. Basically, if you use this particular
>>> modified version of zmq, you specifically want to use zmq in a pintool
>>> context.
>>>
>>> I wouldn't say my changes are "harm", they are what I see as necessary
>>> in order to realize pintool utilization of zeromq. I'm just not  as
>>> familiar with the zeromq code base as perhaps you guys are. :-)
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:42 PM, Lindley French <lindleyf at gmail.com>wrote:
>>>
>>>> Question: Does your modified code work properly outside of a pintool
>>>> context? EG, does it at least pass the "do no harm" test, before you worry
>>>> about whether it does what you want it to do?
>>>>
>>>>
>>>> On Fri, Jan 24, 2014 at 1:37 PM, Kenneth Adam Miller <
>>>> kennethadammiller at gmail.com> wrote:
>>>>
>>>>> It must use pintools, because I had to refactor libzmq in order to
>>>>> play nice with pintool; that's what the question is about. The 200 line
>>>>> program perfectly replicates the issue of not being able to create a
>>>>> router-proxy-dealer setup exactly as laid out in the manual; I get the same
>>>>> error using that sample as in my currently closed source pintool.
>>>>>
>>>>> The reason why I have to use a pintool to demonstrate the problem is
>>>>> because I must give an example that shows the augmented libzmq operating,
>>>>> and the augmented libzmq spawns threads using the PIN thread API (just as
>>>>> the pintool manual specifies that thread control must in a pintool
>>>>> executing environment). I just realized I didn't try to explain that thread
>>>>> spawning and thread control primitives must be done through the PIN api;
>>>>> basically because PIN is a dynamic instrumentation framework, if you spawn
>>>>> a thread, even in the instrumentation or analysis routines that you define,
>>>>> using the normal facilities, then PIN will have a recursive condition
>>>>> against the thread that you innocently meant to spawn to help you do
>>>>> analysis or instrumentation faster. In order to prevent this, they provide
>>>>> the PIN thread APIs.
>>>>>
>>>>> I like zeromq, and I would like to use it to coordinate my analysis
>>>>> and instrumentation routines. The problem is, the zeromq library uses some
>>>>> thread control and primitives that it shouldn't.
>>>>>
>>>>> I'm sorry, but I can't make the example any less complicated by moving
>>>>> the example out of a pintool context. I can try and eliminate some code
>>>>> lines... If you have any questions about pintool itself, please feel free
>>>>> to ask. I'm the one getting help from you guys in the long run, so I'm more
>>>>> than happy to help you help me :-)
>>>>>
>>>>> To be honest, I acknowledge that I'm bringing in a complex library
>>>>> that has nothing to do with this mailing list itself, but this porting
>>>>> needs to occur in order for the pintool community to benefit from the
>>>>> capabilities that zmq offers. I'm highly motivated by zeromq's success and
>>>>> flexibility, enough that I'm willing to dive an into unfamiliar code dig
>>>>> through and find what I think needs to be changed and change it.
>>>>>
>>>>>
>>>>> On Fri, Jan 24, 2014 at 12:02 PM, Pieter Hintjens <ph at imatix.com>wrote:
>>>>>
>>>>>> I saw the repository and saw some code but a 200+ line program isn't
>>>>>> really a minimal test case. As far as I understand, using pintools
>>>>>> should be transparent to users, since it doesn't modify the ZMQ API,
>>>>>> right? So you can make a minimal test case that doesn't require any
>>>>>> knowledge of pintools to understand.
>>>>>>
>>>>>> On Fri, Jan 24, 2014 at 8:39 AM, Kenneth Adam Miller
>>>>>> <kennethadammiller at gmail.com> wrote:
>>>>>> > The example is on the github repo where it has
>>>>>> PinTool-ErrorReplication.
>>>>>> >
>>>>>> >
>>>>>> > On Fri, Jan 24, 2014 at 8:29 AM, Pieter Hintjens <ph at imatix.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Hi Kenneth,
>>>>>> >>
>>>>>> >> Could you post a minimal example of the code that causes an error?
>>>>>> If
>>>>>> >> possible, in C, using the low level API. First step is to let us
>>>>>> >> simply inspect this and check for obvious misuse of sockets.
>>>>>> >>
>>>>>> >> The "socket operation on non-socket" can happen when you pass an
>>>>>> >> invalid argument to a zmq socket call, or when you are trying to
>>>>>> use a
>>>>>> >> closed socket.
>>>>>> >>
>>>>>> >> -Pieter
>>>>>> >>
>>>>>> >> On Fri, Jan 24, 2014 at 7:28 AM, Kenneth Adam Miller
>>>>>> >> <kennethadammiller at gmail.com> wrote:
>>>>>> >> > I would like to use zeromq to work with intel pintool (
>>>>>> pintool.org).
>>>>>> >> > Pintool
>>>>>> >> > is a dynamic binary instrumentation framework; it allows the
>>>>>> developer
>>>>>> >> > to
>>>>>> >> > specify some callbacks at many levels of execution of any binary
>>>>>> >> > program.
>>>>>> >> > But to allow zeromq to work with it, you have to restrict thread
>>>>>> >> > creation
>>>>>> >> > down to only the context or whatever other threads you have, and
>>>>>> then
>>>>>> >> > specify that the thread creation and any other thread apis go
>>>>>> through
>>>>>> >> > the
>>>>>> >> > pin interface. If they don't then you potentially have a
>>>>>> recursive
>>>>>> >> > instrument and execute condition, where a thread you want to
>>>>>> work for
>>>>>> >> > the
>>>>>> >> > instrumentation/analysis side is also being targeted for
>>>>>> >> > instrumentation.
>>>>>> >> >
>>>>>> >> > My changes to zeromq consist of some minor alterations to
>>>>>> thread.hpp and
>>>>>> >> > mutex.hpp. You can see what I've changed on my forked repository
>>>>>> over
>>>>>> >> > here-all I do is replace the threading primitives that each
>>>>>> class is
>>>>>> >> > abstracting from with an instance for pin's.
>>>>>> >> >
>>>>>> >> > I know for certain that my changes are at taking effect, because
>>>>>> what I
>>>>>> >> > can
>>>>>> >> > see is that inter thread communication for just ZMQ_PULL and
>>>>>> ZMQ_PUSH
>>>>>> >> > works
>>>>>> >> > just fine. But when I go to create dealer's and router's and
>>>>>> link them
>>>>>> >> > with
>>>>>> >> > a proxy I start getting errors with code that came straight from
>>>>>> the
>>>>>> >> > manual,
>>>>>> >> > which is really not good. I would prefer that my hack continue
>>>>>> to work,
>>>>>> >> > and
>>>>>> >> > I have confidence that with some help I can make it work.
>>>>>> >> >
>>>>>> >> > custom libzmq implementation:
>>>>>> >> > https://github.com/KennethAdamMiller/libzmq
>>>>>> >> >
>>>>>> >> > ./autogen.sh
>>>>>> >> >
>>>>>> >> > (for 64 bit)
>>>>>> >> > ./configure CXXFLAGS="-DBIGARRAY_MULTIPLIER=1 -Wall -Werror
>>>>>> >> > -Wno-unknown-pragmas -fno-stack-protector -DTARGET_IA32E
>>>>>> -DHOST_IA32E
>>>>>> >> > -fPIC
>>>>>> >> > -DTARGET_LINUX  -I$HOME/pin-2.13/source/include/pin
>>>>>> >> > -I$HOME/pin-2.13/source/include/pin/gen
>>>>>> >> > -I$HOME/pin-2.13/extras/components/include
>>>>>> >> > -I$HOME/pin-2.13/extras/xed2-intel64/include
>>>>>> >> > -I$HOME/pin-2.13/source/tools/InstLib -O3 -fomit-frame-pointer
>>>>>> >> > -fno-strict-aliasing  -g -std=c++11 -L$HOMEpin-2.13/intel64/lib
>>>>>> >> > -L$HOME/pin-2.13/intel64/lib-ext
>>>>>> -L$HOME/pin-2.13/intel64/runtime/glibc
>>>>>> >> > -L$HOME/pin-2.13/extras/xed2-intel64/lib -lpin -lxed -ldwarf
>>>>>> -lelf -ldl
>>>>>> >> > "
>>>>>> >> >
>>>>>> >> > sudo make install
>>>>>> >> >
>>>>>> >> > repeatable pintool error:
>>>>>> >> > https://github.com/KennethAdamMiller/PinTool-ErrorReplication
>>>>>> >> > Building this pintool is easy:
>>>>>> >> > download a pin kit and extract to some location, like ~/pin-2.13
>>>>>> >> >
>>>>>> >> > export PIN_ROOT=location
>>>>>> >> > export TOOLS_ROOT=$PIN_ROOT/source/tools
>>>>>> >> > export CONFIG_ROOT=$TOOLS_ROOT/Config
>>>>>> >> >
>>>>>> >> > then just cd to where you checked out my tool and type make.
>>>>>> You'll see
>>>>>> >> > that
>>>>>> >> > you get socket operation on non-socket. I was hoping that
>>>>>> someone else
>>>>>> >> > could
>>>>>> >> > help me find out why it is getting that error, and then may
>>>>>> suggest what
>>>>>> >> > I
>>>>>> >> > change in libzmq.
>>>>>> >> >
>>>>>> >> > _______________________________________________
>>>>>> >> > zeromq-dev mailing list
>>>>>> >> > zeromq-dev at lists.zeromq.org
>>>>>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>> >> >
>>>>>> >> _______________________________________________
>>>>>> >> zeromq-dev mailing list
>>>>>> >> zeromq-dev at lists.zeromq.org
>>>>>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > zeromq-dev mailing list
>>>>>> > zeromq-dev at lists.zeromq.org
>>>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>> >
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140124/df9a6d24/attachment.htm>