[zeromq-dev] Proposal to unify ZMQ Java projects

Tue Feb 2 15:45:59 CET 2016

Thanks everyone for the support, I am still figuring out how the
existing projects actually work to avoid any unnecessary changes :)

Does libzmq actually use zproject? I haven't found a project.xml in
the repo, only in czmq.

Assuming that libzmq provides a model for its API, it sounds like a
good idea to keep things in sync.

I have not yet looked at zproject in depth but it looks like we could
use the model to generate a Java interface that resembles the libzmq C
API. This would be the 'leading' API that JZMQ-JNI and JeroMQ both
would have to implement. For methods that are not supported by JeroMQ
we could throw UnsupportedOperationException's. Finally, there would
be the hand-written idiomatic layer on top of the generated one. A
user could then choose to either use the low-level API directly or the
idiomatic layer.

There is also https://github.com/jnr/jnr-ffi, which looks like it
could simplify the JNI bridge. It was created by JRuby as a
replacement for JNI (see
http://www.oracle.com/technetwork/java/jvmls2013nutter-2013526.pdf),
but I have no experience with JNI to evaulate if it is a good fit.

Cheers
Mario

2016-02-02 15:18 GMT+01:00 Arnaud Loonstra <arnaud at sphaero.org>:
> I have zero hands-on experience with java and zeromq but I was
> wondering what role zproject can claim in this.
>
> Zproject already generates the API so it should do this for Java as
> well no matter how it is implemented (native Java or JNI)
>
> Rg,
>
> Arnaud
>
> On 2016-02-02 14:39, Trevor Bernard wrote:
>> I agree, the ZeroMQ Java ecosystem is fragmented and leaves much to
>> be
>> desired. I agree with Pieter in that you should only tackle the most
>> crucial pieces first since this can easily become overwhelming. I'd
>> happily answer any questions you might have.
>>
>> Best,
>> Trevor
>>
>> On Tue, Feb 2, 2016 at 4:08 AM, Pieter Hintjens <ph at imatix.com>
>> wrote:
>>> This all sounds great. My advice is to start on the most crucial of
>>> these, get it working minimally, then repeat. You don't need to get
>>> it
>>> right every time. Make small steps. See our contribution process.
>>>
>>> Regarding the higher level classes I'd reshape them to be more
>>> idiomatic Java, and put them into a single library, yes. But find a
>>> name that is appropriate, and don't be afraid of re-using the old
>>> 'jzmq' name if that's what you need to do. ('javaczmq' is not a good
>>> name.)
>>>
>>> Perhaps the outcome is a new jzmq that includes JeroMQ as a
>>> dependency.
>>>
>>> -Pieter
>>>
>>> On Tue, Feb 2, 2016 at 2:24 AM, Mario Steinhoff
>>> <steinhoff.mario at gmail.com> wrote:
>>>> Hi list,
>>>>
>>>> after a few weeks spending on building two little Java applications
>>>> that
>>>> communicate with each other over ZMQ, I want to share my thoughts
>>>> about the
>>>> current state of the Java-related ZMQ projects.
>>>>
>>>> I first read the zguide and then started with JeroMQ because I
>>>> didn't want to
>>>> deal with jzmq and JNI. After looking at the jeromq and jzmq
>>>> projects, I was a
>>>> bit confused about the correct usage of the APIs and when to use
>>>> which classes
>>>> how.
>>>>
>>>> Today I managed to create a little analysis of what I think there
>>>> currently is
>>>> available and what problems I see:
>>>>
>>>> Problem: Currently, both Java libraries provide some of the
>>>> implementations that
>>>> originated from CZMQ (ZMsg, ZFrame, ZThread, ZLoop, ZActor, ZAuth),
>>>> but with
>>>> more or less minor differences, probably caused by independent bugs
>>>> and changes
>>>> on both sides:
>>>>
>>>>
>>>> https://github.com/zeromq/jzmq
>>>>
>>>> JZMQ Core ("Yay, I can use libzmq from Java"):
>>>> These three classes define the actual, public-usable, low-level
>>>> ZeroMQ Java API:
>>>> - org.zeromq.ZMQ
>>>> - org.zeromq.ZMQException
>>>> - org.zeromq.ZContext
>>>> Plus some utility classes:
>>>> - org.zeromq.EmbeddedLibraryTools (JNI support)
>>>> - org.zeromq.App (not required to use libzmq, prints version info)
>>>>
>>>> CZMQ in Java ("Sockets and byte arrays are fine but sometimes I'd
>>>> like to have
>>>> a litte more"):
>>>> - org.zeromq.ZMsg
>>>> - org.zeromq.ZFrame (+Utils.java is only used here, could as well
>>>> be eleminated)
>>>> - org.zeromq.ZThread
>>>> - org.zeromq.ZLoop
>>>> - org.zeromq.ZAuth (ZThread-based Actor that implements ZAP)
>>>> - org.zeromq.ZDispatcher (Like ZLoop, with less features and more
>>>> performance?)
>>>>
>>>> ZeroMQ Devices (Looking at libzmq #422, devices seem to be an
>>>> outdated concept)
>>>> - org.zeromq.ZMQQueue (native Java implementation, doesn't use
>>>> zmq_proxy)
>>>> - org.zeromq.ZMQForwarder
>>>> - org.zeromq.ZMQStreamer
>>>>
>>>> https://github.com/zeromq/jeromq
>>>>
>>>> JeroMQ Core ("I want to use ZMQ from Java, but without the JNI
>>>> mess."):
>>>> Again, the actual, public-usable, ZeroMQ low-level Java API:
>>>> - org.zeromq.ZMQ
>>>> - org.zeromq.ZMQException
>>>> - org.zeromq.ZContext
>>>> Plus JeroMQ-related code:
>>>> - org.zeromq.ManagedContext (Only usable in JeroMQ, uses classes
>>>> from zmq.*)
>>>> - zmq.* (The actual Java-native ZMQ implementation)
>>>>
>>>> CZMQ in Java:
>>>> - org.zeromq.ZMsg
>>>> - org.zeromq.ZFrame
>>>> - org.zeromq.ZThread
>>>> - org.zeromq.ZLoop
>>>> - org.zeromq.ZActor (+ZStar+ZAgent)
>>>> - org.zeromq.ZBeacon
>>>>
>>>> ZeroMQ Devices:
>>>> - org.zeromq.ZMQQueue (uses JeroMQ native proxy + minor differences
>>>> to jzmq)
>>>> Forwarder and Streamer are missing here, but as the device concept
>>>> is outdated
>>>> this doesn't seem like a big problem.
>>>>
>>>> From what I see, jzmq wants to provide the low-level bridge to
>>>> libzmq while
>>>> jeromq wants to provide a drop-in replacement for jzmq if one does
>>>> not want to
>>>> deal with JNI compilation or does not need libzmq performance. Both
>>>> projects
>>>> want to provide those CZMQ classes because they make life easier.
>>>>
>>>> Solution: Move and merge the CZMQ classes from the original
>>>> projects into a new
>>>> 'java-czmq' project that builds an independent jar file and change
>>>> the original
>>>> projects to include that jar file as a dependency. This way we stay
>>>> backward
>>>> compatible but can provide bugfixes and new features to both
>>>> projects without
>>>> the overhead of copying changes between projects. Instead of a new
>>>> project, I
>>>> could also imagine to move those to the jzmq-api project, as it
>>>> seems to be most
>>>> similar to the original czmq project.
>>>>
>>>> Problem: Java implementations provide outdated devices.
>>>>
>>>> When I search what zmq devices are about, I get API docs for ZMQ
>>>> 2.1.11 and a
>>>> few blog posts which all seem to be 3+ years old. This is
>>>> confusing.
>>>>
>>>> Solution: Phase out devices, move and merge implementations into
>>>> the jzmq-api
>>>> project, make a separate module and add that as a dependency to the
>>>> original
>>>> projects, then make the dependency optional somewhere in the
>>>> future.
>>>>
>>>> Problem: ZThread API is not idomatic Java.
>>>>
>>>> It looks like this is a 1:1 port of the CZMQ API:
>>>>
>>>> https://github.com/zeromq/czmq/blob/master/include/zthread.h#L22-L26
>>>>
>>>> But a DetachedRunnable in Java makes no sense at all, an
>>>> abstraction to run
>>>> "normal" background threads already exists with java.lang.Thread.
>>>>
>>>> Using AttachedRunnable to setup threads and PIPEs is very cool, but
>>>> then the
>>>> Object[] args parameter is suboptional.
>>>>
>>>> 1) It's not needed, I can use the constructor to pass arguments
>>>> into my actor
>>>> class.
>>>>
>>>> 2) It's a nice source for subtle bugs because I have to do casting
>>>> and also know
>>>> the order of the arguments if there is more than one.
>>>>
>>>> Solution: Build idiomatic interfaces, phase out old interfaces.
>>>>
>>>> Problem: Java code uses CLASS comments, which breaks javadoc.
>>>>
>>>> Example:
>>>>
>>>> https://github.com/zeromq/jzmq/blob/master/src/main/java/org/zeromq/ZThread.java#L65-L70
>>>> http://zeromq.github.io/jzmq/javadocs/org/zeromq/ZThread.html
>>>>
>>>> Solution: Avoid CLASS comments in ZMQ Java code.
>>>>
>>>> In the long term, I imagine the following project/library
>>>> structure:
>>>>
>>>> - jzmq-api provides the main user-faceable API
>>>> - jzmq-api provides a SPI that both jzmq and jeromq have to
>>>> implement
>>>> - In a project, I add jzmq-api + either jzmq or jeromq as a
>>>> dependency
>>>>
>>>> There is also zmq-jni and zmtp-java, but both projects look like
>>>> incomplete
>>>> experiments so I'll ignore them for now.
>>>>
>>>> Comments, Ideas, Suggestions?
>>>>
>>>> Cheers
>>>> Mario
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-- 
Mario Steinhoff
https://github.com/msteinhoff
https://twitter.com/msteinhofff
T: +49 173 7265158
In der Gelpe 79
42349 Wuppertal