[zeromq-dev] Zero Copy in Java jzmq
Trevor Bernard
trevor.bernard at gmail.com
Sat Feb 16 22:47:39 CET 2013
@gonzalo Did you do anything special on the client side to read the
direct bytebuffer? My recvZeroCopy isn't working ATM.
On Sat, Feb 16, 2013 at 4:28 PM, Trevor Bernard
<trevor.bernard at gmail.com> wrote:
> Also you'll need to use ByteBuffer.allocateDirect(...) in order for it
> to work. I explicitly didn't check GetDirectBufferAddress for perf
> reasons.
>
> For the JZMQ api, I used sendZeroCopy and recvZeroCopy for a lack of a
> better/fancier name. ;)
>
> -Trev
>
> On Sat, Feb 16, 2013 at 4:26 PM, Trevor Bernard
> <trevor.bernard at gmail.com> wrote:
>> I added support for zero copy send and recv.
>>
>> https://github.com/zeromq/jzmq/issues/170
>>
>> I haven't had time to test this but it compiles and the unit tests pass.
>>
>> -Trev
>>
>> On Fri, Feb 15, 2013 at 5:26 PM, The Carrolls <thecarrolls at jiminger.com> wrote:
>>> Awesome! I'll be watching for it then.
>>>
>>> On 02/15/13 13:25, Trevor Bernard wrote:
>>>> This looks simple enough. I should be able to port this into a jzmq
>>>> branch fairly easily.
>>>>
>>>> I'll likely do it over the weekend.
>>>>
>>>> -Trev
>>>>
>>>> On Fri, Feb 15, 2013 at 1:55 PM, gonzalo diethelm <gdiethelm at dcv.cl> wrote:
>>>>> I remembered that I did implement this for a tentative Java binding for Crossroads IO. The relevant methods where send() and recv(). This all worked and performed MUCH better that current binding. Some details follow; if more is required, let me know.
>>>>>
>>>>> Usage:
>>>>>
>>>>> int size = 128;
>>>>> ByteBuffer bb = ByteBuffer.allocateDirect(size);
>>>>> ...
>>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>>>> ...
>>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0);
>>>>>
>>>>> ---------------------
>>>>> In Java:
>>>>>
>>>>> public class XsLibrary {
>>>>> ...
>>>>> public native int xs_send(long socket,
>>>>> ByteBuffer buffer,
>>>>> int offset,
>>>>> int length,
>>>>> int flags);
>>>>> public native int xs_recv(long socket,
>>>>> ByteBuffer buffer,
>>>>> int offset,
>>>>> int length,
>>>>> int flags);
>>>>> }
>>>>>
>>>>> ---------------------
>>>>> In C (the JNI part):
>>>>>
>>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1send(JNIEnv* env,
>>>>> jobject obj,
>>>>> jlong socket,
>>>>> jobject buffer,
>>>>> jint offset,
>>>>> jint length,
>>>>> jint flags)
>>>>> {
>>>>> void* sock = 0;
>>>>> jbyte* buf = 0;
>>>>> int ret = 0;
>>>>>
>>>>> sock = (void*) socket;
>>>>> XS_ASSERT(sock);
>>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>>> XS_ASSERT(buf);
>>>>> ret = xs_send(sock, buf, length, flags);
>>>>> return ret;
>>>>> }
>>>>>
>>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1recv(JNIEnv* env,
>>>>> jobject obj,
>>>>> jlong socket,
>>>>> jobject buffer,
>>>>> jint offset,
>>>>> jint length,
>>>>> jint flags)
>>>>> {
>>>>> void* sock = 0;
>>>>> jbyte* buf = 0;
>>>>> int ret = 0;
>>>>>
>>>>> sock = (void*) socket;
>>>>> XS_ASSERT(sock);
>>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer);
>>>>> XS_ASSERT(buf);
>>>>> ret = xs_recv(sock, buf, length, flags);
>>>>> return ret;
>>>>> }
>>>>>
>>>>> ---------------------
>>>>>
>>>>> --
>>>>> Gonzalo Diethelm
>>>>> DCV Chile
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-
>>>>>> bounces at lists.zeromq.org] On Behalf Of Trevor Bernard
>>>>>> Sent: Friday, February 15, 2013 2:02 PM
>>>>>> To: thecarrolls at jiminger.com; ZeroMQ development list
>>>>>> Subject: Re: [zeromq-dev] Zero Copy in Java jzmq
>>>>>>
>>>>>>> I'm thinking of extending jzmq to optionally allow the zero-copy
>>>>>>> technique using the concepts outlined by Martin Thompson here:
>>>>>> That's awesome
>>>>>>
>>>>>>> I'm not 100% sure I'll do it, I need some more experimentation first,
>>>>>>> but is there any interest or words of advice if someone's tried this
>>>>>>> already?
>>>>>> I'd love for send/recv to have a ByteBuffer API.
>>>>>>
>>>>>> Some resources and general suggestions:
>>>>>>
>>>>>> Have a peak at zmq_msg_t in zmq.h.
>>>>>>
>>>>>> Defines ZMTP/2.0 Spec:
>>>>>> * http://rfc.zeromq.org/spec:15
>>>>>>
>>>>>> Also depending on the size of the msg, it may be allocated on the stack or
>>>>>> heap.
>>>>>> * http://api.zeromq.org/3-2:zmq-msg-init-size
>>>>>>
>>>>>> At some point you'll have to access a malloc'ed array from Java/JNI.
>>>>>> *
>>>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions.
>>>>>> html#nio_support
>>>>>>
>>>>>> Something like this would do the trick. This wraps a native pointer in a
>>>>>> ByteBufer
>>>>>>
>>>>>> void *data = ...
>>>>>> jobject bb = (*env)->NewDirectByteBuffer(env, (void*) data,
>>>>>> sizeof(zmq_msg_t));
>>>>>>
>>>>>> Here is another option: Use a library like javolution to do your mapping:
>>>>>>
>>>>>> http://javolution.org/target/site/apidocs/javolution/io/Struct.html
>>>>>>
>>>>>> class Message extends Struct {
>>>>>> Unsigned8 _ = new Unsigned8(32);
>>>>>> Message() {
>>>>>> setByteBuffer(Message.nativeBuffer(), 0);
>>>>>> }
>>>>>> private static native ByteBuffer nativeBuffer(); }
>>>>>>
>>>>>> Something along those lines.
>>>>>>
>>>>>> Hope this helps.
>>>>>>
>>>>>> -Trev
>>>>>> _______________________________________________
>>>>>> zeromq-dev mailing list
>>>>>> zeromq-dev at lists.zeromq.org
>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>
>>>>> -----------------------------------------
>>>>> Declaración de confidencialidad: Este Mensaje esta destinado para
>>>>> el uso de la o las personas o entidades a quien ha sido dirigido y
>>>>> puede contener información reservada y confidencial que no puede
>>>>> ser divulgada, difundida, ni aprovechada en forma alguna. El uso no
>>>>> autorizado de la información contenida en este correo podrá ser
>>>>> sancionado de conformidad con la ley chilena.
>>>>> Si usted ha recibido este correo electrónico por error, le pedimos
>>>>> eliminarlo junto con los archivos adjuntos y avisar inmediatamente
>>>>> al remitente, respondiendo este mensaje.
>>>>>
>>>>> "Before printing this e-mail think if is really necesary".
>>>>> Disclosure: This Message is to be used by the individual,
>>>>> individuals or entities that it is addressed to and may include
>>>>> private and confidential information that may not be disclosed,
>>>>> made public nor used in any way at all. Unauthorized use of the
>>>>> information in this electronic mail message may be subject to the
>>>>> penalties set forth by Chilean law.
>>>>> If you have received this electronic mail message in error, we ask
>>>>> you to destroy the message and its attached file(s) and to
>>>>> immediately notify the sender by answering this message.
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
More information about the zeromq-dev
mailing list