[zeromq-dev] Request: Serialization hook.

[Oliver Smith]

I'd like to see making transport mixing a little easier.

Consider the following code:

   struct Foo {
     std::string a ;
     std::vector<int> b ;
   } ;

   zmq::socket_t socket(ctx, ZMQ_REQ) ;
   socket.connect("tcp://192.168.0.1:1234") ;
   socket.connect("ipc://workers") ;

   Foo foo("Hello", { 1, 2, 3, 4 }) ;    // C++0x

   zmq::message_t msg(&foo, sizeof(foo), NULL) ;
   socket.send(msg, 0) ;

   zmq::message_t reply ;
   socket.recv(&reply, 0) ;

Depending which endpoint the message gets sent on, this may or may not 
crash.

You *could* create a serialization thread, which actually handles the 
remote socket connect, but this introduces a significant learning curve 
and it defeats the "your code is virtually ready to go!" claim.

I would like to suggest

     zmq_setsockopt(socket, ZMQ_SERIALIZER, function) ;    (I don't care 
what it's called, ZMQ_SO_SERIALIZER, whatever :)

The transport would then determine whether 'function' needs to be called 
depending on whether a message is being sent via IPC or out of memory space.

Function would take two parameters: a bool providing true/false whether 
the message is being received or sent, and a pointer to a zmq::message_t.

It would return a pointer to a zmq::message_t.

If the routine determines no work is required, it can return the source 
message. If it returns a new zmq::message_t, then the transport can free 
the old zmq::message_t.

Caveat: Because serialization may take some cpu cycles, this would be a 
good case for having more than one context thread.

Which brings me to another thought: Might it be a good idea to introduce 
a warnings system into the ZeroMQ debug-build flavor.

debugWarn(ZMQ_WARNING_33) ;    // ZeroMQ Caution #33: Application called 
recv() on a ZMQ_SUB socket without subscribing to any topics. (See 
zmq_setsockopt).

if ( settingSockOptZMQ_SERIALIZE_FUNCTION and context.numThreads <= 1 )
   debugWarn(ZMQ_WARNING_34) ;    // ZeroMQ Caution #34: Consider using 
more than one IO thread in applications which may have to serialize data 
to maintain throughput (see zmq_init)

etc.

Perhaps coupled with a sock-opt for disabling a particular warning - 
e.g. if your recv() is waiting for a term ;)

- Oliver