[zeromq-dev] Zero Copy in Java jzmq
Trevor Bernard
trevor.bernard at gmail.com
Sat Feb 16 21:28:34 CET 2013
Also you'll need to use ByteBuffer.allocateDirect(...) in order for it
to work. I explicitly didn't check GetDirectBufferAddress for perf
reasons.
For the JZMQ api, I used sendZeroCopy and recvZeroCopy for a lack of a
better/fancier name. ;)
-Trev
On Sat, Feb 16, 2013 at 4:26 PM, Trevor Bernard
<trevor.bernard at gmail.com> wrote:
> I added support for zero copy send and recv.
>
> https://github.com/zeromq/jzmq/issues/170
>
> I haven't had time to test this but it compiles and the unit tests pass.
>
> -Trev
>
> On Fri, Feb 15, 2013 at 5:26 PM, The Carrolls <thecarrolls at jiminger.com> wrote:
>> Awesome! I'll be watching for it then.
>>
>> On 02/15/13 13:25, Trevor Bernard wrote:
>>> This looks simple enough. I should be able to port this into a jzmq
>>> branch fairly easily.
>>>
>>> I'll likely do it over the weekend.
>>>
>>> -Trev
>>>
>>> On Fri, Feb 15, 2013 at 1:55 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:
>>>> I remembered that I did implement this for a tentative Java binding for Crossroads IO. The relevant methods where send() and recv(). This all worked and performed MUCH better that current binding. Some details follow; if more is required, let me know.
>>>>
>>>> Usage:
>>>>
>>>> int size = 128;
>>>> ByteBuffer bb = ByteBuffer.allocateDirect(size);
>>>> ...
>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>>> ...
>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>>>
>>>> ---------------------
>>>> In Java:
>>>>
>>>> public class XsLibrary {
>>>> ...
>>>> public native int xs_send(long socket,
>>>> ByteBuffer buffer,
>>>> int offset,
>>>> int length,
>>>> int flags);
>>>> public native int xs_recv(long socket,
>>>> ByteBuffer buffer,
>>>> int offset,
>>>> int length,
>>>> int flags);
>>>> }
>>>>
>>>> ---------------------
>>>> In C (the JNI part):
>>>>
>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1send(JNIEnv* env,
>>>> jobject obj,
>>>> jlong socket,
>>>> jobject buffer,
>>>> jint offset,
>>>> jint length,
>>>> jint flags)
>>>> {
>>>> void* sock = 0;
>>>> jbyte* buf = 0;
>>>> int ret = 0;
>>>>
>>>> sock = (void*) socket;
>>>> XS_ASSERT(sock);
>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>> XS_ASSERT(buf);
>>>> ret = xs_send(sock, buf, length, flags);
>>>> return ret;
>>>> }
>>>>
>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1recv(JNIEnv* env,
>>>> jobject obj,
>>>> jlong socket,
>>>> jobject buffer,
>>>> jint offset,
>>>> jint length,
>>>> jint flags)
>>>> {
>>>> void* sock = 0;
>>>> jbyte* buf = 0;
>>>> int ret = 0;
>>>>
>>>> sock = (void*) socket;
>>>> XS_ASSERT(sock);
>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>> XS_ASSERT(buf);
>>>> ret = xs_recv(sock, buf, length, flags);
>>>> return ret;
>>>> }
>>>>
>>>> ---------------------
>>>>
>>>> --
>>>> Gonzalo Diethelm
>>>> DCV Chile
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-
>>>>> bounces at lists.zeromq.org] On Behalf Of Trevor Bernard
>>>>> Sent: Friday, February 15, 2013 2:02 PM
>>>>> To: thecarrolls at jiminger.com; ZeroMQ development list
>>>>> Subject: Re: [zeromq-dev] Zero Copy in Java jzmq
>>>>>
>>>>>> I'm thinking of extending jzmq to optionally allow the zero-copy
>>>>>> technique using the concepts outlined by Martin Thompson here:
>>>>> That's awesome
>>>>>
>>>>>> I'm not 100% sure I'll do it, I need some more experimentation first,
>>>>>> but is there any interest or words of advice if someone's tried this
>>>>>> already?
>>>>> I'd love for send/recv to have a ByteBuffer API.
>>>>>
>>>>> Some resources and general suggestions:
>>>>>
>>>>> Have a peak at zmq_msg_t in zmq.h.
>>>>>
>>>>> Defines ZMTP/2.0 Spec:
>>>>> * http://rfc.zeromq.org/spec:15
>>>>>
>>>>> Also depending on the size of the msg, it may be allocated on the stack or
>>>>> heap.
>>>>> * http://api.zeromq.org/3-2:zmq-msg-init-size
>>>>>
>>>>> At some point you'll have to access a malloc'ed array from Java/JNI.
>>>>> *
>>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions.
>>>>> html#nio_support
>>>>>
>>>>> Something like this would do the trick. This wraps a native pointer in a
>>>>> ByteBufer
>>>>>
>>>>> void *data = ...
>>>>> jobject bb = (*env)->NewDirectByteBuffer(env, (void*) data,
>>>>> sizeof(zmq_msg_t));
>>>>>
>>>>> Here is another option: Use a library like javolution to do your mapping:
>>>>>
>>>>> http://javolution.org/target/site/apidocs/javolution/io/Struct.html
>>>>>
>>>>> class Message extends Struct {
>>>>> Unsigned8 _ = new Unsigned8(32);
>>>>> Message() {
>>>>> setByteBuffer(Message.nativeBuffer(), 0);
>>>>> }
>>>>> private static native ByteBuffer nativeBuffer(); }
>>>>>
>>>>> Something along those lines.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> -Trev
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>
>>>> -----------------------------------------
>>>> Declaración de confidencialidad: Este Mensaje esta destinado para
>>>> el uso de la o las personas o entidades a quien ha sido dirigido y
>>>> puede contener información reservada y confidencial que no puede
>>>> ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
>>>> autorizado de la información contenida en este correo podrá ser
>>>> sancionado de conformidad con la ley chilena.
>>>> Si usted ha recibido este correo electrónico por error, le pedimos
>>>> eliminarlo junto con los archivos adjuntos y avisar inmediatamente
>>>> al remitente, respondiendo este mensaje.
>>>>
>>>> "Before printing this e-mail think if is really necesary".
>>>> Disclosure: This Message is to be used by the individual,
>>>> individuals or entities that it is addressed to and may include
>>>> private and confidential information that may not be disclosed,
>>>> made public nor used in any way at all. Unauthorized use of the
>>>> information in this electronic mail message may be subject to the
>>>> penalties set forth by Chilean law.
>>>> If you have received this electronic mail message in error, we ask
>>>> you to destroy the message and its attached file(s) and to
>>>> immediately notify the sender by answering this message.
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
More information about the zeromq-dev
mailing list