[zeromq-dev] Custom refactoring of ZeroMQ to work with pintools...

Kenneth Adam Miller kennethadammiller at gmail.com
Sat Jan 25 20:12:11 CET 2014 

I understand, and I appreciate you all for that!

And I would guess that there is possibly more that can be done, but it
would be debugging work that would be pertinent to the pintool environment.
It is something I will have to tackle. I appreciate your willingness to
help me out and provide suggestions :)


On Sat, Jan 25, 2014 at 1:19 AM, Michel Pelletier <
pelletier.michel at gmail.com> wrote:

> We as a community have probably provided the most that can be reasonable
> expected.  You're working with a new, apparently closed source library that
> few of us, or perhaps none at all, have heard of.  If it's not possible to
> provide us with an isolated test case then there's likely little we can do
> besides provide some broad debugging support.
>
> I looked at you diff against zeromq master on github, it looked reasonable
> enough.   Your best bet now would be to probably use some debugging tools
> to dig deeper.  As Pieter pointed out it seems your socket is invalid or
> closed.  Have you investigated those possibilities?
>
> -Michel
>
>
> On Fri, Jan 24, 2014 at 2:51 PM, Kenneth Adam Miller <
> kennethadammiller at gmail.com> wrote:
>
>> Is anybody still willing to work on this with me?
>>
>>
>> On Fri, Jan 24, 2014 at 12:57 PM, Kenneth Adam Miller <
>> kennethadammiller at gmail.com> wrote:
>>
>>> In that case, it would be a separately named library; the developers of
>>> their application wouldn't execute their application in a pintool context,
>>> so they would use the vanilla library, because they would never be able to
>>> get their application off the ground due to link errors from undefined
>>> symbols and runtime unresolved references. They would never think to
>>> themselves "I'm going to run my normal everyday application which uses
>>> regular threads in a regular environment with an obscure variant of
>>> zeromq." As a result, the vanilla zeromq would spawn threads the normal
>>> way, and hence they would get instrumented. Sure you would have some
>>> unnecessary data/space consumption from replicated
>>>
>>> This particular library could just be named to something like libpinzmq
>>> or libzmq-pintool. Then it should work fine.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 12:51 PM, Lindley French <lindleyf at gmail.com>wrote:
>>>
>>>> Ideally, it would be possible to use ZMQ simultaneously inside of and
>>>> outside of a pintool context. I'm not familiar with pintools, but if it's
>>>> an instrumentation framework, what if they code you're trying to instrument
>>>> *also* uses ZMQ?
>>>>
>>>>
>>>> On Fri, Jan 24, 2014 at 1:44 PM, Kenneth Adam Miller <
>>>> kennethadammiller at gmail.com> wrote:
>>>>
>>>>> No; It calls PIN_SpawnInternalThread, which only makes sense in a
>>>>> pintool's executing environment. Basically, if you use this particular
>>>>> modified version of zmq, you specifically want to use zmq in a pintool
>>>>> context.
>>>>>
>>>>> I wouldn't say my changes are "harm", they are what I see as necessary
>>>>> in order to realize pintool utilization of zeromq. I'm just not  as
>>>>> familiar with the zeromq code base as perhaps you guys are. :-)
>>>>>
>>>>>
>>>>> On Fri, Jan 24, 2014 at 12:42 PM, Lindley French <lindleyf at gmail.com>wrote:
>>>>>
>>>>>> Question: Does your modified code work properly outside of a pintool
>>>>>> context? EG, does it at least pass the "do no harm" test, before you worry
>>>>>> about whether it does what you want it to do?
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 24, 2014 at 1:37 PM, Kenneth Adam Miller <
>>>>>> kennethadammiller at gmail.com> wrote:
>>>>>>
>>>>>>> It must use pintools, because I had to refactor libzmq in order to
>>>>>>> play nice with pintool; that's what the question is about. The 200 line
>>>>>>> program perfectly replicates the issue of not being able to create a
>>>>>>> router-proxy-dealer setup exactly as laid out in the manual; I get the same
>>>>>>> error using that sample as in my currently closed source pintool.
>>>>>>>
>>>>>>> The reason why I have to use a pintool to demonstrate the problem is
>>>>>>> because I must give an example that shows the augmented libzmq operating,
>>>>>>> and the augmented libzmq spawns threads using the PIN thread API (just as
>>>>>>> the pintool manual specifies that thread control must in a pintool
>>>>>>> executing environment). I just realized I didn't try to explain that thread
>>>>>>> spawning and thread control primitives must be done through the PIN api;
>>>>>>> basically because PIN is a dynamic instrumentation framework, if you spawn
>>>>>>> a thread, even in the instrumentation or analysis routines that you define,
>>>>>>> using the normal facilities, then PIN will have a recursive condition
>>>>>>> against the thread that you innocently meant to spawn to help you do
>>>>>>> analysis or instrumentation faster. In order to prevent this, they provide
>>>>>>> the PIN thread APIs.
>>>>>>>
>>>>>>> I like zeromq, and I would like to use it to coordinate my analysis
>>>>>>> and instrumentation routines. The problem is, the zeromq library uses some
>>>>>>> thread control and primitives that it shouldn't.
>>>>>>>
>>>>>>> I'm sorry, but I can't make the example any less complicated by
>>>>>>> moving the example out of a pintool context. I can try and eliminate some
>>>>>>> code lines... If you have any questions about pintool itself, please feel
>>>>>>> free to ask. I'm the one getting help from you guys in the long run, so I'm
>>>>>>> more than happy to help you help me :-)
>>>>>>>
>>>>>>> To be honest, I acknowledge that I'm bringing in a complex library
>>>>>>> that has nothing to do with this mailing list itself, but this porting
>>>>>>> needs to occur in order for the pintool community to benefit from the
>>>>>>> capabilities that zmq offers. I'm highly motivated by zeromq's success and
>>>>>>> flexibility, enough that I'm willing to dive an into unfamiliar code dig
>>>>>>> through and find what I think needs to be changed and change it.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 24, 2014 at 12:02 PM, Pieter Hintjens <ph at imatix.com>wrote:
>>>>>>>
>>>>>>>> I saw the repository and saw some code but a 200+ line program isn't
>>>>>>>> really a minimal test case. As far as I understand, using pintools
>>>>>>>> should be transparent to users, since it doesn't modify the ZMQ API,
>>>>>>>> right? So you can make a minimal test case that doesn't require any
>>>>>>>> knowledge of pintools to understand.
>>>>>>>>
>>>>>>>> On Fri, Jan 24, 2014 at 8:39 AM, Kenneth Adam Miller
>>>>>>>> <kennethadammiller at gmail.com> wrote:
>>>>>>>> > The example is on the github repo where it has
>>>>>>>> PinTool-ErrorReplication.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Fri, Jan 24, 2014 at 8:29 AM, Pieter Hintjens <ph at imatix.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> Hi Kenneth,
>>>>>>>> >>
>>>>>>>> >> Could you post a minimal example of the code that causes an
>>>>>>>> error? If
>>>>>>>> >> possible, in C, using the low level API. First step is to let us
>>>>>>>> >> simply inspect this and check for obvious misuse of sockets.
>>>>>>>> >>
>>>>>>>> >> The "socket operation on non-socket" can happen when you pass an
>>>>>>>> >> invalid argument to a zmq socket call, or when you are trying to
>>>>>>>> use a
>>>>>>>> >> closed socket.
>>>>>>>> >>
>>>>>>>> >> -Pieter
>>>>>>>> >>
>>>>>>>> >> On Fri, Jan 24, 2014 at 7:28 AM, Kenneth Adam Miller
>>>>>>>> >> <kennethadammiller at gmail.com> wrote:
>>>>>>>> >> > I would like to use zeromq to work with intel pintool (
>>>>>>>> pintool.org).
>>>>>>>> >> > Pintool
>>>>>>>> >> > is a dynamic binary instrumentation framework; it allows the
>>>>>>>> developer
>>>>>>>> >> > to
>>>>>>>> >> > specify some callbacks at many levels of execution of any
>>>>>>>> binary
>>>>>>>> >> > program.
>>>>>>>> >> > But to allow zeromq to work with it, you have to restrict
>>>>>>>> thread
>>>>>>>> >> > creation
>>>>>>>> >> > down to only the context or whatever other threads you have,
>>>>>>>> and then
>>>>>>>> >> > specify that the thread creation and any other thread apis go
>>>>>>>> through
>>>>>>>> >> > the
>>>>>>>> >> > pin interface. If they don't then you potentially have a
>>>>>>>> recursive
>>>>>>>> >> > instrument and execute condition, where a thread you want to
>>>>>>>> work for
>>>>>>>> >> > the
>>>>>>>> >> > instrumentation/analysis side is also being targeted for
>>>>>>>> >> > instrumentation.
>>>>>>>> >> >
>>>>>>>> >> > My changes to zeromq consist of some minor alterations to
>>>>>>>> thread.hpp and
>>>>>>>> >> > mutex.hpp. You can see what I've changed on my forked
>>>>>>>> repository over
>>>>>>>> >> > here-all I do is replace the threading primitives that each
>>>>>>>> class is
>>>>>>>> >> > abstracting from with an instance for pin's.
>>>>>>>> >> >
>>>>>>>> >> > I know for certain that my changes are at taking effect,
>>>>>>>> because what I
>>>>>>>> >> > can
>>>>>>>> >> > see is that inter thread communication for just ZMQ_PULL and
>>>>>>>> ZMQ_PUSH
>>>>>>>> >> > works
>>>>>>>> >> > just fine. But when I go to create dealer's and router's and
>>>>>>>> link them
>>>>>>>> >> > with
>>>>>>>> >> > a proxy I start getting errors with code that came straight
>>>>>>>> from the
>>>>>>>> >> > manual,
>>>>>>>> >> > which is really not good. I would prefer that my hack continue
>>>>>>>> to work,
>>>>>>>> >> > and
>>>>>>>> >> > I have confidence that with some help I can make it work.
>>>>>>>> >> >
>>>>>>>> >> > custom libzmq implementation:
>>>>>>>> >> > https://github.com/KennethAdamMiller/libzmq
>>>>>>>> >> >
>>>>>>>> >> > ./autogen.sh
>>>>>>>> >> >
>>>>>>>> >> > (for 64 bit)
>>>>>>>> >> > ./configure CXXFLAGS="-DBIGARRAY_MULTIPLIER=1 -Wall -Werror
>>>>>>>> >> > -Wno-unknown-pragmas -fno-stack-protector -DTARGET_IA32E
>>>>>>>> -DHOST_IA32E
>>>>>>>> >> > -fPIC
>>>>>>>> >> > -DTARGET_LINUX  -I$HOME/pin-2.13/source/include/pin
>>>>>>>> >> > -I$HOME/pin-2.13/source/include/pin/gen
>>>>>>>> >> > -I$HOME/pin-2.13/extras/components/include
>>>>>>>> >> > -I$HOME/pin-2.13/extras/xed2-intel64/include
>>>>>>>> >> > -I$HOME/pin-2.13/source/tools/InstLib -O3 -fomit-frame-pointer
>>>>>>>> >> > -fno-strict-aliasing  -g -std=c++11 -L$HOMEpin-2.13/intel64/lib
>>>>>>>> >> > -L$HOME/pin-2.13/intel64/lib-ext
>>>>>>>> -L$HOME/pin-2.13/intel64/runtime/glibc
>>>>>>>> >> > -L$HOME/pin-2.13/extras/xed2-intel64/lib -lpin -lxed -ldwarf
>>>>>>>> -lelf -ldl
>>>>>>>> >> > "
>>>>>>>> >> >
>>>>>>>> >> > sudo make install
>>>>>>>> >> >
>>>>>>>> >> > repeatable pintool error:
>>>>>>>> >> > https://github.com/KennethAdamMiller/PinTool-ErrorReplication
>>>>>>>> >> > Building this pintool is easy:
>>>>>>>> >> > download a pin kit and extract to some location, like
>>>>>>>> ~/pin-2.13
>>>>>>>> >> >
>>>>>>>> >> > export PIN_ROOT=location
>>>>>>>> >> > export TOOLS_ROOT=$PIN_ROOT/source/tools
>>>>>>>> >> > export CONFIG_ROOT=$TOOLS_ROOT/Config
>>>>>>>> >> >
>>>>>>>> >> > then just cd to where you checked out my tool and type make.
>>>>>>>> You'll see
>>>>>>>> >> > that
>>>>>>>> >> > you get socket operation on non-socket. I was hoping that
>>>>>>>> someone else
>>>>>>>> >> > could
>>>>>>>> >> > help me find out why it is getting that error, and then may
>>>>>>>> suggest what
>>>>>>>> >> > I
>>>>>>>> >> > change in libzmq.
>>>>>>>> >> >
>>>>>>>> >> > _______________________________________________
>>>>>>>> >> > zeromq-dev mailing list
>>>>>>>> >> > zeromq-dev at lists.zeromq.org
>>>>>>>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>> >> >
>>>>>>>> >> _______________________________________________
>>>>>>>> >> zeromq-dev mailing list
>>>>>>>> >> zeromq-dev at lists.zeromq.org
>>>>>>>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > _______________________________________________
>>>>>>>> > zeromq-dev mailing list
>>>>>>>> > zeromq-dev at lists.zeromq.org
>>>>>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>> >
>>>>>>>> _______________________________________________
>>>>>>>> zeromq-dev mailing list
>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> zeromq-dev mailing list
>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140125/b51684ed/attachment.htm>

More information about the zeromq-dev mailing list