[zeromq-dev] Urgent! Where is the link between ZeroMq and IpoIB

Krishna Narasimhan krishna.nm86 at gmail.com
Wed May 7 11:49:25 CEST 2014


Hi Jim,
    Thanks for the response. So then I would like to ask you one follow up
question : (hopefully the last since you have been surprisingly patient)

Lets assume I write a wrapper that directs to libraries like OpenMPI (with
direct RDMA support) or the old calls in Zeromq depending on the hardware
available, where do you think I should start looking. I mean,

1) Is it possible to do such a thing where I pick out calls to the IP
stack(indirectly IPoIB) and replace them with the wrapper?

2) If so, which are the calls I should start looking at and how do I
identify them in the Zeromq code?



On Tue, May 6, 2014 at 6:51 PM, Jim Garlick <garlick at llnl.gov> wrote:

> I'm speaking of Linux (RHEL 6.x) where IPoIB presents a regular 'ib0'
> network interface and is fully integrated with the kernel IP stack.
> Zeromq thus doesn't need (and doesn't have) special linkage to the
> user space InfinBand libraries.
>
> Jim
>
> On Tue, May 06, 2014 at 04:16:15PM +0200, Krishna Narasimhan wrote:
> > Hi Jim,
> >     Thanks a lot for the response. Really glad you took time out to read
> my
> > issue completely.
> >
> > It is really interesting the links you have provided. Good to know that
> > there was prior attempts at this.
> >
> > Btw, I dont understand still how ZeroMq is linked to IpOIb. Do you mean
> to
> > say that ZeroMq code never changed, but the underlying Socket mechanism
> > takes care of the decision of normal or (IpoIB sockets) based on the
> > request and hardware present?
> >
> > I saw some codes like
> >
> >  fd_t s = socket (domain_, type_, protocol_);
> >
> >
> > Is this where the dynamic choice of using IpoIb happens?
> >
> > I am not sure I explained my question properly.
> >
> > I understand that IpoIb codes are available in a linux driver code , so I
> > was looking for references to IpoIb.h
> >
> > If all my above explanation turned out to be confusing , here goes my
> last
> > attempt :
> >
> > How are Zeromw code and IpoIb.h linked?
> >
> >
> > On Mon, May 5, 2014 at 10:53 PM, Jim Garlick <garlick at llnl.gov> wrote:
> >
> > > Hi Krishna,
> > >
> > > As far as I know ZeroMQ can only use IPoIB via normal IP sockets.
> > > Looks like the idea of an RDMA zeromq transport came up in 2011,
> > > but went nowhere?
> > >
> > > http://lists.zeromq.org/pipermail/zeromq-dev/2011-December/014784.html
> > >
> > > Another interesting example of adding RDMA support to a project with
> > > pluggable transports is net/9p (transport for fs/v9fs) in the Linux
> kernel.
> > > It might be useful to look at that code and follow its history to
> > > understand
> > > the problem better.
> > >
> > > http://lwn.net/Articles/301896/
> > >
> > > As I recall the orig code in 2008, though functional, didn't deliver a
> big
> > > performance win over TCP (for that stack) and introduced new failure
> modes,
> > > making it unattractive and frustrating for casual users.  It languished
> > > for years in a semi-broken state until people at CEA/Bull began working
> > > on it again in 2012-13.  I'm guessing the second order issues CEA/Bull
> > > dealt with would be instructive.  Also their Mooshika RDMA abstraction,
> > > which I think allows for a wireshark tap and testing w/o Infiniband
> card
> > > could be interesting.
> > >
> > > https://github.com/martinetd/mooshika
> > >
> > > Jim
> > >
> > > On Mon, May 05, 2014 at 07:38:05PM +0200, Krishna Narasimhan wrote:
> > > > Hello,
> > > >         I am a researcher at Frankfurt University and currently
> working
> > > on
> > > > contributing to folks who use ZeroMq at Gsi , Darmstadt. Can anyone
> guide
> > > > me on which part of the code links ZeroMq to IPoIB. Because it doesnt
> > > seem
> > > > to be linked at code level. Does ZeroMq call some wrapper which
> directs
> > > to
> > > > IPoIb or normal IP?
> > > >
> > > > Any help here would be appreciated. this is rather urgent.
> > > >
> > > > On the other hand, I am also looking to contribute to including MPI
> > > (direct
> > > > RDMA) access for ZeroMq based applications, so anyone with this
> knowledge
> > > > will also be helpful. But am not sure this belongs to this thread.
> > > >
> > > > Thanks a lot.
> > > >
> > > > Regards
> > > >        Krishna
> > >
> > > > _______________________________________________
> > > > zeromq-dev mailing list
> > > > zeromq-dev at lists.zeromq.org
> > > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > >
> > > _______________________________________________
> > > zeromq-dev mailing list
> > > zeromq-dev at lists.zeromq.org
> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> > >
> >
> >
> >
> > --
> > -----------------------------------------------------
> > I dare do all that may become a man; Who dares do more, is none -
> Macbeth,
> > twelfh night!
> > Regards
> >        Krishna
>



-- 
-----------------------------------------------------
I dare do all that may become a man; Who dares do more, is none - Macbeth,
twelfh night!
Regards
       Krishna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140507/3c1e6700/attachment.htm>


More information about the zeromq-dev mailing list