[zeromq-dev] Zero Copy in Java jzmq

Trevor Bernard trevor.bernard at gmail.com
Sat Feb 16 21:26:13 CET 2013


I added support for zero copy send and recv.

https://github.com/zeromq/jzmq/issues/170

I haven't had time to test this but it compiles and the unit tests pass.

-Trev

On Fri, Feb 15, 2013 at 5:26 PM, The Carrolls <thecarrolls at jiminger.com> wrote:
> Awesome! I'll be watching for it then.
>
> On 02/15/13 13:25, Trevor Bernard wrote:
>> This looks simple enough. I should be able to port this into a jzmq
>> branch fairly easily.
>>
>> I'll likely do it over the weekend.
>>
>> -Trev
>>
>> On Fri, Feb 15, 2013 at 1:55 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:
>>> I remembered that I did implement this for a tentative Java binding for Crossroads IO. The relevant methods where send() and recv(). This all worked and performed MUCH better that current binding. Some details follow; if more is required, let me know.
>>>
>>> Usage:
>>>
>>>              int size = 128;
>>>              ByteBuffer bb = ByteBuffer.allocateDirect(size);
>>> ...
>>>              int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>> ...
>>>              int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>>
>>> ---------------------
>>> In Java:
>>>
>>> public class XsLibrary {
>>> ...
>>>      public native int xs_send(long socket,
>>>                                ByteBuffer buffer,
>>>                                int offset,
>>>                                int length,
>>>                                int flags);
>>>      public native int xs_recv(long socket,
>>>                                ByteBuffer buffer,
>>>                                int offset,
>>>                                int length,
>>>                                int flags);
>>> }
>>>
>>> ---------------------
>>> In C (the JNI part):
>>>
>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1send(JNIEnv* env,
>>>                                                                   jobject obj,
>>>                                                                   jlong socket,
>>>                                                                   jobject buffer,
>>>                                                                   jint offset,
>>>                                                                   jint length,
>>>                                                                   jint flags)
>>> {
>>>      void* sock = 0;
>>>      jbyte* buf = 0;
>>>      int ret = 0;
>>>
>>>      sock = (void*) socket;
>>>      XS_ASSERT(sock);
>>>      buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>      XS_ASSERT(buf);
>>>      ret = xs_send(sock, buf, length, flags);
>>>      return ret;
>>> }
>>>
>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1recv(JNIEnv* env,
>>>                                                                   jobject obj,
>>>                                                                   jlong socket,
>>>                                                                   jobject buffer,
>>>                                                                   jint offset,
>>>                                                                   jint length,
>>>                                                                   jint flags)
>>> {
>>>      void* sock = 0;
>>>      jbyte* buf = 0;
>>>      int ret = 0;
>>>
>>>      sock = (void*) socket;
>>>      XS_ASSERT(sock);
>>>      buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>      XS_ASSERT(buf);
>>>      ret = xs_recv(sock, buf, length, flags);
>>>      return ret;
>>> }
>>>
>>> ---------------------
>>>
>>> --
>>> Gonzalo Diethelm
>>> DCV Chile
>>>
>>>
>>>> -----Original Message-----
>>>> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-
>>>> bounces at lists.zeromq.org] On Behalf Of Trevor Bernard
>>>> Sent: Friday, February 15, 2013 2:02 PM
>>>> To: thecarrolls at jiminger.com; ZeroMQ development list
>>>> Subject: Re: [zeromq-dev] Zero Copy in Java jzmq
>>>>
>>>>> I'm thinking of extending jzmq to optionally allow the zero-copy
>>>>> technique using the concepts outlined by Martin Thompson here:
>>>> That's awesome
>>>>
>>>>> I'm not 100% sure I'll do it, I need some more experimentation first,
>>>>> but is there any interest or words of advice if someone's tried this
>>>>> already?
>>>> I'd love for send/recv to have a ByteBuffer API.
>>>>
>>>> Some resources and general suggestions:
>>>>
>>>> Have a peak at zmq_msg_t in zmq.h.
>>>>
>>>> Defines ZMTP/2.0 Spec:
>>>> * http://rfc.zeromq.org/spec:15
>>>>
>>>> Also depending on the size of the msg, it may be allocated on the stack or
>>>> heap.
>>>> * http://api.zeromq.org/3-2:zmq-msg-init-size
>>>>
>>>> At some point you'll have to access a malloc'ed array from Java/JNI.
>>>> *
>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions.
>>>> html#nio_support
>>>>
>>>> Something like this would do the trick. This wraps a native pointer in a
>>>> ByteBufer
>>>>
>>>> void *data = ...
>>>> jobject bb = (*env)->NewDirectByteBuffer(env, (void*) data,
>>>> sizeof(zmq_msg_t));
>>>>
>>>> Here is another option: Use a library like javolution to do your mapping:
>>>>
>>>> http://javolution.org/target/site/apidocs/javolution/io/Struct.html
>>>>
>>>> class Message extends Struct {
>>>>     Unsigned8 _ = new Unsigned8(32);
>>>>      Message() {
>>>>          setByteBuffer(Message.nativeBuffer(), 0);
>>>>      }
>>>>      private static native ByteBuffer nativeBuffer(); }
>>>>
>>>> Something along those lines.
>>>>
>>>> Hope this helps.
>>>>
>>>> -Trev
>>>> _______________________________________________
>>>> zeromq-dev mailing list
>>>> zeromq-dev at lists.zeromq.org
>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>> -----------------------------------------
>>> Declaración de confidencialidad: Este Mensaje esta destinado para
>>> el uso de la o las personas o entidades a quien ha sido dirigido y
>>> puede contener información reservada y confidencial que no puede
>>> ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
>>> autorizado de la información contenida en este correo podrá ser
>>> sancionado de conformidad con la ley chilena.
>>> Si usted ha recibido este correo electrónico por error, le pedimos
>>> eliminarlo junto con los archivos adjuntos y avisar inmediatamente
>>> al remitente, respondiendo este mensaje.
>>>
>>> "Before printing this e-mail think if is really necesary".
>>> Disclosure: This Message is to be used by the individual,
>>> individuals or entities that it is addressed to and may include
>>> private and confidential information that may not be disclosed,
>>> made public nor used in any way at all. Unauthorized use of the
>>> information in this electronic mail message may be subject to the
>>> penalties set forth by Chilean law.
>>> If you have received this electronic mail message in error, we ask
>>> you to destroy the message and its attached file(s) and to
>>> immediately notify the sender by answering this message.
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list