[zeromq-dev] P/Invoke-based CLR access to zeroMQ

[Martin Sustrik]

Dirk,

Unfortunately, the buffer has to exist until it is physically send to 
the network. 0MQ is a zero-copy system, meaning that it uses the buffer 
supplied by user instead of creating its own. At the same time it is an 
async system meaning that the messages are enqueued as fast as possible 
and then sent to the network in the background. That's why ffn is needed 
(async deallocation of the buffer).

I would rather think of a way to pin the CLI-supplied buffer down, send 
it to 0MQ and ask it (via ffn) to unpin the buffer once the data are 
physically sent to the network. Is that kind of thing possible?

Martin

Dirk O. Siebnich wrote:
> Hi,
> 
> I think I am on to something. Please check the Send function. What are
> the requirements for data_ buffer to exist beyond the czmq_send
> function, i.e. is there any real need for the ffn_ callback?
> 
> If the czmq_receive could be modified to request a buffer of an
> appropriate size via callback, instead of requiring a ffn_ free
> function, we could have zero copies in the P/Invoke interface!
> 
> -Dirk
> 
> On Sun, 2009-02-22 at 09:01 +0100, Martin Sustrik wrote:
>> Hi Barak,
>>
>>> The PInvoke implementation can be used for both .NET and Mono, but I 
>>> still prefer the power I get from C++/CLI not going through C# -> C -> 
>>> C++ way.
>> We'll measure this (you can do so as well). If there's no significant 
>> performance advantage for using C++/CLI over P/Invoke, I would prefer 
>> having only a single implementation of .NET/Mono API. Less code = less bugs.
>>
>>> We copy the message bytes into a managed byte array on 'Receive' and 
>>> free the original copy (if required).  The byte array is passed by 
>>> reference, so no copy overhead here.
>> Great. I'm not a CLI developer. I was concerned because returning arrays 
>> in C++ is a problem - code like this causes whole array to be copied:
>>
>> vector<int> fx ()
>> {
>>       vector<int> v (1000);
>>       return v;
>> }
>>
>> We should also think about whether it's possible to avoid the copy from 
>> message_t to Array<Byte> in the receive function. If message is modeled 
>> as an object this should be possible in some way.
>>
>>> Agree that we change the C API for the C# implementation, the same 
>>> should be done for all languages.  What I mean is that the 0MZ interface 
>>> for each language Java, C#, Python should supply a common functionality. 
>>>  So if the C API will give access to the api_thread's mask I should be 
>>> able to access it also from Python.
>> Yes, we've already added parameters allowing to control queue limits to 
>> all languages. Let's move in "same API for all languages" direction in 
>> the future.
>>
>>> This will also affect how a 'Message' should be implemented, if any, we 
>>> can use 'enums' to specify 'gap' and 'delimiter' instead using heap 
>>> references.
>> Ok. let's try to get the interface right. To follow "same API for all 
>> languages" principle, CLI and Python bindings should be based on primary 
>> C++ interface. Thar would mean that "receive" function should have two 
>> return value a.) queue-id b.) message-object. Message object itself is 
>> componsed of two parts: a.) message-type (enum) b.) message body (array).
>>
>> Let's think about what would be the best way to model this in 
>> CLI/Python/Java/C...
>>
>> Martin