[zeromq-dev] Proposal to unify ZMQ Java projects

Stephen Riesenberg stephen.riesenberg at gmail.com
Mon Mar 28 02:42:47 CEST 2016


Mario,

Have you had any additional thoughts on this recently? I hate to drag up an
old-ish thread, but this one seems like a good recurring conversation to
have. I wish I had been paying attention when you introduced it, but I was
not. :(

You mentioned the jzmq-api project, and I wanted to address that point. I
think the idea of using a project (jzmq-api, or one like it) is a fantastic
idea! I too have been working on this but with a slightly different
approach, one that would be supplemented greatly by a more formal approach
like that outlined previously in this thread.

My current approach has been to abstract the jzmq-api dependency against
jzmq into a Maven profile that allows one to switch between jzmq and
jeromq. It was not without problems, some of which I addressed as best I
could (e.g. small P/Rs to work out little kinks in the low-level API
differences between them). However, it's still currently in a state that's
not really production ready (jzmq-api was in that state anyway, mostly due
to some missing features and configurability, among others). But it
currently works quite well for basic things, and demonstrates a working
solution of one way to solve the problem you've outlined.

Since jzmq-api is still an experimental library, I don't see why we
couldn't use it as you outlined, and add the SPI now. As others have
mentioned, we could potentially incorporate zproject for that, if it's a
good fit, though I haven't looked at it myself. As you mentioned, zmq-jni
and zmtp-java seem incomplete, and probably aren't part of the "minimal"
solution for this problem, though they'd be good to see incorporated later.

Thoughts? Action items?


On Tue, Feb 2, 2016 at 10:12 AM, Pieter Hintjens <ph at imatix.com> wrote:

> zproject does already generate JNI bindings for CZMQ et al. We could
> definitely use these as the basis for new classes in JZMQ.
>
> On Tue, Feb 2, 2016 at 3:18 PM, Arnaud Loonstra <arnaud at sphaero.org>
> wrote:
> > I have zero hands-on experience with java and zeromq but I was
> > wondering what role zproject can claim in this.
> >
> > Zproject already generates the API so it should do this for Java as
> > well no matter how it is implemented (native Java or JNI)
> >
> > Rg,
> >
> > Arnaud
> >
> > On 2016-02-02 14:39, Trevor Bernard wrote:
> >> I agree, the ZeroMQ Java ecosystem is fragmented and leaves much to
> >> be
> >> desired. I agree with Pieter in that you should only tackle the most
> >> crucial pieces first since this can easily become overwhelming. I'd
> >> happily answer any questions you might have.
> >>
> >> Best,
> >> Trevor
> >>
> >> On Tue, Feb 2, 2016 at 4:08 AM, Pieter Hintjens <ph at imatix.com>
> >> wrote:
> >>> This all sounds great. My advice is to start on the most crucial of
> >>> these, get it working minimally, then repeat. You don't need to get
> >>> it
> >>> right every time. Make small steps. See our contribution process.
> >>>
> >>> Regarding the higher level classes I'd reshape them to be more
> >>> idiomatic Java, and put them into a single library, yes. But find a
> >>> name that is appropriate, and don't be afraid of re-using the old
> >>> 'jzmq' name if that's what you need to do. ('javaczmq' is not a good
> >>> name.)
> >>>
> >>> Perhaps the outcome is a new jzmq that includes JeroMQ as a
> >>> dependency.
> >>>
> >>> -Pieter
> >>>
> >>> On Tue, Feb 2, 2016 at 2:24 AM, Mario Steinhoff
> >>> <steinhoff.mario at gmail.com> wrote:
> >>>> Hi list,
> >>>>
> >>>> after a few weeks spending on building two little Java applications
> >>>> that
> >>>> communicate with each other over ZMQ, I want to share my thoughts
> >>>> about the
> >>>> current state of the Java-related ZMQ projects.
> >>>>
> >>>> I first read the zguide and then started with JeroMQ because I
> >>>> didn't want to
> >>>> deal with jzmq and JNI. After looking at the jeromq and jzmq
> >>>> projects, I was a
> >>>> bit confused about the correct usage of the APIs and when to use
> >>>> which classes
> >>>> how.
> >>>>
> >>>> Today I managed to create a little analysis of what I think there
> >>>> currently is
> >>>> available and what problems I see:
> >>>>
> >>>> Problem: Currently, both Java libraries provide some of the
> >>>> implementations that
> >>>> originated from CZMQ (ZMsg, ZFrame, ZThread, ZLoop, ZActor, ZAuth),
> >>>> but with
> >>>> more or less minor differences, probably caused by independent bugs
> >>>> and changes
> >>>> on both sides:
> >>>>
> >>>>
> >>>> https://github.com/zeromq/jzmq
> >>>>
> >>>> JZMQ Core ("Yay, I can use libzmq from Java"):
> >>>> These three classes define the actual, public-usable, low-level
> >>>> ZeroMQ Java API:
> >>>> - org.zeromq.ZMQ
> >>>> - org.zeromq.ZMQException
> >>>> - org.zeromq.ZContext
> >>>> Plus some utility classes:
> >>>> - org.zeromq.EmbeddedLibraryTools (JNI support)
> >>>> - org.zeromq.App (not required to use libzmq, prints version info)
> >>>>
> >>>> CZMQ in Java ("Sockets and byte arrays are fine but sometimes I'd
> >>>> like to have
> >>>> a litte more"):
> >>>> - org.zeromq.ZMsg
> >>>> - org.zeromq.ZFrame (+Utils.java is only used here, could as well
> >>>> be eleminated)
> >>>> - org.zeromq.ZThread
> >>>> - org.zeromq.ZLoop
> >>>> - org.zeromq.ZAuth (ZThread-based Actor that implements ZAP)
> >>>> - org.zeromq.ZDispatcher (Like ZLoop, with less features and more
> >>>> performance?)
> >>>>
> >>>> ZeroMQ Devices (Looking at libzmq #422, devices seem to be an
> >>>> outdated concept)
> >>>> - org.zeromq.ZMQQueue (native Java implementation, doesn't use
> >>>> zmq_proxy)
> >>>> - org.zeromq.ZMQForwarder
> >>>> - org.zeromq.ZMQStreamer
> >>>>
> >>>> https://github.com/zeromq/jeromq
> >>>>
> >>>> JeroMQ Core ("I want to use ZMQ from Java, but without the JNI
> >>>> mess."):
> >>>> Again, the actual, public-usable, ZeroMQ low-level Java API:
> >>>> - org.zeromq.ZMQ
> >>>> - org.zeromq.ZMQException
> >>>> - org.zeromq.ZContext
> >>>> Plus JeroMQ-related code:
> >>>> - org.zeromq.ManagedContext (Only usable in JeroMQ, uses classes
> >>>> from zmq.*)
> >>>> - zmq.* (The actual Java-native ZMQ implementation)
> >>>>
> >>>> CZMQ in Java:
> >>>> - org.zeromq.ZMsg
> >>>> - org.zeromq.ZFrame
> >>>> - org.zeromq.ZThread
> >>>> - org.zeromq.ZLoop
> >>>> - org.zeromq.ZActor (+ZStar+ZAgent)
> >>>> - org.zeromq.ZBeacon
> >>>>
> >>>> ZeroMQ Devices:
> >>>> - org.zeromq.ZMQQueue (uses JeroMQ native proxy + minor differences
> >>>> to jzmq)
> >>>> Forwarder and Streamer are missing here, but as the device concept
> >>>> is outdated
> >>>> this doesn't seem like a big problem.
> >>>>
> >>>> From what I see, jzmq wants to provide the low-level bridge to
> >>>> libzmq while
> >>>> jeromq wants to provide a drop-in replacement for jzmq if one does
> >>>> not want to
> >>>> deal with JNI compilation or does not need libzmq performance. Both
> >>>> projects
> >>>> want to provide those CZMQ classes because they make life easier.
> >>>>
> >>>> Solution: Move and merge the CZMQ classes from the original
> >>>> projects into a new
> >>>> 'java-czmq' project that builds an independent jar file and change
> >>>> the original
> >>>> projects to include that jar file as a dependency. This way we stay
> >>>> backward
> >>>> compatible but can provide bugfixes and new features to both
> >>>> projects without
> >>>> the overhead of copying changes between projects. Instead of a new
> >>>> project, I
> >>>> could also imagine to move those to the jzmq-api project, as it
> >>>> seems to be most
> >>>> similar to the original czmq project.
> >>>>
> >>>> Problem: Java implementations provide outdated devices.
> >>>>
> >>>> When I search what zmq devices are about, I get API docs for ZMQ
> >>>> 2.1.11 and a
> >>>> few blog posts which all seem to be 3+ years old. This is
> >>>> confusing.
> >>>>
> >>>> Solution: Phase out devices, move and merge implementations into
> >>>> the jzmq-api
> >>>> project, make a separate module and add that as a dependency to the
> >>>> original
> >>>> projects, then make the dependency optional somewhere in the
> >>>> future.
> >>>>
> >>>> Problem: ZThread API is not idomatic Java.
> >>>>
> >>>> It looks like this is a 1:1 port of the CZMQ API:
> >>>>
> >>>> https://github.com/zeromq/czmq/blob/master/include/zthread.h#L22-L26
> >>>>
> >>>> But a DetachedRunnable in Java makes no sense at all, an
> >>>> abstraction to run
> >>>> "normal" background threads already exists with java.lang.Thread.
> >>>>
> >>>> Using AttachedRunnable to setup threads and PIPEs is very cool, but
> >>>> then the
> >>>> Object[] args parameter is suboptional.
> >>>>
> >>>> 1) It's not needed, I can use the constructor to pass arguments
> >>>> into my actor
> >>>> class.
> >>>>
> >>>> 2) It's a nice source for subtle bugs because I have to do casting
> >>>> and also know
> >>>> the order of the arguments if there is more than one.
> >>>>
> >>>> Solution: Build idiomatic interfaces, phase out old interfaces.
> >>>>
> >>>> Problem: Java code uses CLASS comments, which breaks javadoc.
> >>>>
> >>>> Example:
> >>>>
> >>>>
> https://github.com/zeromq/jzmq/blob/master/src/main/java/org/zeromq/ZThread.java#L65-L70
> >>>> http://zeromq.github.io/jzmq/javadocs/org/zeromq/ZThread.html
> >>>>
> >>>> Solution: Avoid CLASS comments in ZMQ Java code.
> >>>>
> >>>> In the long term, I imagine the following project/library
> >>>> structure:
> >>>>
> >>>> - jzmq-api provides the main user-faceable API
> >>>> - jzmq-api provides a SPI that both jzmq and jeromq have to
> >>>> implement
> >>>> - In a project, I add jzmq-api + either jzmq or jeromq as a
> >>>> dependency
> >>>>
> >>>> There is also zmq-jni and zmtp-java, but both projects look like
> >>>> incomplete
> >>>> experiments so I'll ignore them for now.
> >>>>
> >>>> Comments, Ideas, Suggestions?
> >>>>
> >>>> Cheers
> >>>> Mario
> >>>> _______________________________________________
> >>>> zeromq-dev mailing list
> >>>> zeromq-dev at lists.zeromq.org
> >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>> _______________________________________________
> >>> zeromq-dev mailing list
> >>> zeromq-dev at lists.zeromq.org
> >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20160327/7bee4712/attachment.htm>


More information about the zeromq-dev mailing list