[zeromq-dev] Proposal to unify ZMQ Java projects

[Mario Steinhoff]

Hi list,

after a few weeks spending on building two little Java applications that
communicate with each other over ZMQ, I want to share my thoughts about the
current state of the Java-related ZMQ projects.

I first read the zguide and then started with JeroMQ because I didn't want to
deal with jzmq and JNI. After looking at the jeromq and jzmq projects, I was a
bit confused about the correct usage of the APIs and when to use which classes
how.

Today I managed to create a little analysis of what I think there currently is
available and what problems I see:

Problem: Currently, both Java libraries provide some of the implementations that
originated from CZMQ (ZMsg, ZFrame, ZThread, ZLoop, ZActor, ZAuth), but with
more or less minor differences, probably caused by independent bugs and changes
on both sides:


https://github.com/zeromq/jzmq

JZMQ Core ("Yay, I can use libzmq from Java"):
These three classes define the actual, public-usable, low-level ZeroMQ Java API:
- org.zeromq.ZMQ
- org.zeromq.ZMQException
- org.zeromq.ZContext
Plus some utility classes:
- org.zeromq.EmbeddedLibraryTools (JNI support)
- org.zeromq.App (not required to use libzmq, prints version info)

CZMQ in Java ("Sockets and byte arrays are fine but sometimes I'd like to have
a litte more"):
- org.zeromq.ZMsg
- org.zeromq.ZFrame (+Utils.java is only used here, could as well be eleminated)
- org.zeromq.ZThread
- org.zeromq.ZLoop
- org.zeromq.ZAuth (ZThread-based Actor that implements ZAP)
- org.zeromq.ZDispatcher (Like ZLoop, with less features and more performance?)

ZeroMQ Devices (Looking at libzmq #422, devices seem to be an outdated concept)
- org.zeromq.ZMQQueue (native Java implementation, doesn't use zmq_proxy)
- org.zeromq.ZMQForwarder
- org.zeromq.ZMQStreamer

https://github.com/zeromq/jeromq

JeroMQ Core ("I want to use ZMQ from Java, but without the JNI mess."):
Again, the actual, public-usable, ZeroMQ low-level Java API:
- org.zeromq.ZMQ
- org.zeromq.ZMQException
- org.zeromq.ZContext
Plus JeroMQ-related code:
- org.zeromq.ManagedContext (Only usable in JeroMQ, uses classes from zmq.*)
- zmq.* (The actual Java-native ZMQ implementation)

CZMQ in Java:
- org.zeromq.ZMsg
- org.zeromq.ZFrame
- org.zeromq.ZThread
- org.zeromq.ZLoop
- org.zeromq.ZActor (+ZStar+ZAgent)
- org.zeromq.ZBeacon

ZeroMQ Devices:
- org.zeromq.ZMQQueue (uses JeroMQ native proxy + minor differences to jzmq)
Forwarder and Streamer are missing here, but as the device concept is outdated
this doesn't seem like a big problem.

>From what I see, jzmq wants to provide the low-level bridge to libzmq while
jeromq wants to provide a drop-in replacement for jzmq if one does not want to
deal with JNI compilation or does not need libzmq performance. Both projects
want to provide those CZMQ classes because they make life easier.

Solution: Move and merge the CZMQ classes from the original projects into a new
'java-czmq' project that builds an independent jar file and change the original
projects to include that jar file as a dependency. This way we stay backward
compatible but can provide bugfixes and new features to both projects without
the overhead of copying changes between projects. Instead of a new project, I
could also imagine to move those to the jzmq-api project, as it seems to be most
similar to the original czmq project.

Problem: Java implementations provide outdated devices.

When I search what zmq devices are about, I get API docs for ZMQ 2.1.11 and a
few blog posts which all seem to be 3+ years old. This is confusing.

Solution: Phase out devices, move and merge implementations into the jzmq-api
project, make a separate module and add that as a dependency to the original
projects, then make the dependency optional somewhere in the future.

Problem: ZThread API is not idomatic Java.

It looks like this is a 1:1 port of the CZMQ API:
https://github.com/zeromq/czmq/blob/master/include/zthread.h#L22-L26

But a DetachedRunnable in Java makes no sense at all, an abstraction to run
"normal" background threads already exists with java.lang.Thread.

Using AttachedRunnable to setup threads and PIPEs is very cool, but then the
Object[] args parameter is suboptional.

1) It's not needed, I can use the constructor to pass arguments into my actor
class.

2) It's a nice source for subtle bugs because I have to do casting and also know
the order of the arguments if there is more than one.

Solution: Build idiomatic interfaces, phase out old interfaces.

Problem: Java code uses CLASS comments, which breaks javadoc.

Example:
https://github.com/zeromq/jzmq/blob/master/src/main/java/org/zeromq/ZThread.java#L65-L70
http://zeromq.github.io/jzmq/javadocs/org/zeromq/ZThread.html

Solution: Avoid CLASS comments in ZMQ Java code.

In the long term, I imagine the following project/library structure:

- jzmq-api provides the main user-faceable API
- jzmq-api provides a SPI that both jzmq and jeromq have to implement
- In a project, I add jzmq-api + either jzmq or jeromq as a dependency

There is also zmq-jni and zmtp-java, but both projects look like incomplete
experiments so I'll ignore them for now.

Comments, Ideas, Suggestions?

Cheers
Mario