[zeromq-dev] Message format questions and high speed queue

[Ben Kloosterman]

Hi new to 0mq..

I have a very high speed queue ( actually more like a ring buffer) which I'd
like to port to zeromq which has a number of features 

- Optional Zero copy ( it returns the pointer to write to , which is in the
buffer) 
- Receiver pulls 
- Messages are built in registers ( if it can fit)  and written directly to
the queue via non temporal writes
- The queue supports variable sized messages
- No locking or CAS ( single producer / single consumer) 
- Cache optimal design eg the pull and register write direct to queue
results in little to no cache pollution for the sender ( and the receiver
needs to process the contents) . 
- Also the updates on the counters ensure they are in a different cache
line.  Ensuring no CPU messages for cores that don't share cache. ( when you
have a lot of threads affinity becomes an issue) 
- Uses alignment

Now the above is designed for large amounts of queues and inter process , I
want to port it so clients have  other options eg cross machine. Which means
I need to change my messages. 

Now looking at the message structure 

typedef struct
{
    void *content;
    unsigned char flags;
    unsigned char vsm_size;
    unsigned char vsm_data [ZMQ_MAX_VSM_SIZE];
} zmq_msg_t;

 Are there any times when the *content and vsm_size and vsm_data are used ?
If so why is it not a Union ? The size of the message will be 36 bytes or 40
bytes ( or more ) depending on alignment and whether 64 bit.  

I want to store it in the queue something like this 
*Msg1
Flags
Vsm_size 
Variable Data eg 16 bytes [Ensures next Variable data is on alignment via
padding]
*Msg2
Flags
Vsm_size   
Variable Data eg 128 bytes
*NextMsg ptr

When you want to write you can either copy the message to the queue or you
can return the pointer and  write the header and data. 

Still scanning code ... Any comments , ideas  or tips ?


Regards, 

Ben