[zeromq-dev] Introducing Caravan: Simple, Faithful ZMQ-Bindings for Objective Caml (Pedro Borges)

[Guillaume Yziquel]

Le Monday 04 Apr 2011 à 13:24:17 (+0100), Pedro Borges a écrit :
> On Mon, Apr 4, 2011 at 12:31 PM, Guillaume Yziquel
> <guillaume.yziquel at citycable.ch> wrote:
> > Le Monday 04 Apr 2011 à 12:17:01 (+0100), Pedro Borges a écrit :
> 
> >> Unfortunately implementing zero-copy in ocaml is impossible, AFAIK,
> >> without blocking every other thread. If another thread makes an
> >> allocation the GC might move the string you are trying to send.
> >
> > Not so.
> >
> > There are multiple solutions. You can allocate your stuff out of the GC,
> > and the GC will not compact the heap that it doesn't own.
> 
> If I allocate a buffer outside the heap, the buffer from ocaml (a
> string) would have to be memcpy'ed to the buffer.
> 
> > You also the cute caml_register_generational_global_root function if you want buffers
> > that are sort of fixed.
> 
> That approach looks promising.

It's quite a nice one, but it's a bit tricky.

> > There is also the fact that when OCaml code enters a C stub, there is no OCaml code running at the same time (global master lock mechanism).
> 
> That is the "stop all threads" solution I talked about.

Then you'd have to release the global lock in the C stub to allow OCaml
to run in parallel.

Had a cursory glance at the way zero copy is implemented.

Seems that you need to feed the ZeroMQ API some kind of finaliser to
free the OCaml buffer block. That seems tricky at best. Two items:

-1- If the ZeroMQ finaliser is triggered from the ZeroMQ API in a system
thread, and if this system touches the OCaml runtime (which is very very
likely), then the system will have to be registered with OCaml so that
it knows about it.

-2- Releasing the buffer block. Perhaps simply unregistering the
generational global root could do the trick. Not sure but I think it
could do it. Otherwise, weak pointers. I think it depends how it's
implemented, but somehow I doubt that weak pointers would do the trick.

Anyway, that's just handwaving for now.

-- 
     Guillaume Yziquel