[zeromq-dev] endian conversions?

[Martin Sustrik]

Aamir,

> Sorry, I was confusing the ZMQ wire format with the user wire format. 
> The issue is that my own data (e.g. doubles) won't undergo endian 
> conversion. I think the solution is to use an encoding such as Google 
> Protocol Buffers. ZMQ will take care of endian conversion of the 
> Protocol Buffers data, while Protocol Buffers will take care of endian 
> conversion of native datatypes when it decodes to data sent over ZMQ.
> 
> I know from my experience with QPid that AMQP provides the user with a 
> Buffer class that has methods such as putDouble and getDouble, as well 
> as decode and encode methods that are supposed to automatically handle 
> endian conversion. But I guess that ZeroMQ is much lightweight and it is 
> up to the user to provide the wire format.

Ok, here's some background to the topic:

When designing AMQP we've chosen to separate message into two parts. 
There's a structured part that holds named strongly-typed properties and 
there's a BLOB part that holds binary data to be interpreted by 
applications.

The generic case is the latter one. Former one is present to enable a 
single use case: content-based routing. The point with content based 
routing is that AMQP broker has to know about message semantics so that 
it can perform routing/filtering of "get me all the messages with 
name='abc' and price < 10" type.

If you don't need content-based routing, using BLOB-style approach is 
preferable. Structured, strongly typed message format is not for free 
and if you aim at high performance you should avoid any unneeded processing.

As for 0MQ, content-based routing was not implemented so far, so there's 
no need for structured and typed message content at the moment.

Adding a convenience wrapper to transform C/Java/Python types into their 
binary representations is a possibility, however, there's a whole lot of 
serialisation libraries around so it seems to be a kind of overkill to 
hardwire one into 0MQ.

Martin