[zeromq-dev] jzmq and sending java objects inproc

[Paul Cameron]

I wish to use jzmq for sending Java objects inproc, while using other
formalised serialising mechanism (thrift etc) over ipc/tcp.

I went ahead and implemented the ability to send java objects using a
suboptimal method here:
    https://github.com/cameronpm/jzmq/tree/send_object

Here are various problems presented and discussed in no particular
order, in order to get this implemented (I wrote the below section
before implementing the change, skip to the second bullet point for
the gotchas)

* HOW WOULD AN OBJECT BE SENT

A new ZFrame object would have to be allocated, perhaps as a static
constructor accepting an object so it does overlap with existing
constructors. This object would be stored as a field, just as the
char[] data array is. In the ZFrame.send() call, a different JNI
method would have to be invoked.

This new underlying method would call zmq_msg_init_data(...) instead
of the currently called zmq_msg_init_size(...), and our own custom
zmq_free_fn will be provided. To determine what we pass to
zmq_msg_init_data(...):

  1) Create a global reference (ch5.1.2 in JNI manual) to the Java
Object. This will pin it into memory and lets us safely use it across
method calls

  2) Alloc memory to contain this global reference and memcpy it (this
memory is smaller than ZMQ_MAX_VSM_SIZE=30 bytes default), the alloc'd
size will be 4-8 bytes, with perhaps an extra 4 bytes of magic
prepended for sanity checking

  3) Pass in our own zmq_free_fn. This must call
(*env)->DeleteGlobalRef(...) and then free() the globref wrapper

Step #2 is optional saving a malloc if one decides to pass in the
address of the Java Global Reference, but then you have no means of
sanity checking. Under certain conditions (method #2 below) that is
fine.

* HOW WOULD AN OBJECT BE RECEIVED

This is where the problems begin. There is no "good" way to determine
whether a zmq_msg_t object is ours or not. Here are some methods:

  1) if zmq_msg_size(&msg) == <our alloced size> && msg.content ==
(void*)ZMQ_VSM then this was allocated via zmq_msg_init_data. 0MQ does
not call zmq_msg_init_data internally, nor does the existing jzmq
implementation.

  2) If we had a way of getting msg->content->ffn we could check
whether this was equal to our custom zmq_free_fn. However,
msg->content is an opaque void pointer and the structure of
zmq::msg_content_t is internal to 0MQ, so this way is impossible

  3) Calling zmq_msg_close() and detecting whether our zmq_free_fn was
called. I think this fails with pub/sub methods where refcounts can be
greater than 1, so that's unreliable

  4) We allocate a wrapper around the java global reference from a
known pool when creating the message with zmq_msg_init_data(...). On
message recv, we get the data (zmq_msg_data(...)) and range-check the
address wrapper from our known pools. This adds complexity and we have
to deal with threading problems in C++ (although I'd just end up using
Java monitors). I do not want to go there.

Method #2 is the best but there's no way of introspecting
msg->content->ffn/hint, #1 will work for now but may break in the
future (that is what I used), #3 does not work in all cases and #4
hurts to even think about.


Opinions? I'd really like introspection of msg->content->ffn (and I
guess 'hint')