[zeromq-dev] Fwd: [hybi] hum #3: Message

[Pieter Hintjens]

Guys,

FYI and as part of the slow but inevitable process of turning 0MQ's
wire level protocols into a real specification for interoperability,
here is the state of the framing discussion for WebSockets, which is
an asynchronous message carrying protocol.  Closer to TCP than
anything else but what interests me is the proposed framing format:

 1 bit - fragment
 2 bits - reserved
 5 bits - opcode
 8 bits - short length
 [if length = 255: 6 bytes length]
 length bytes of payload

IMO it would be desirable if (a) IETF WebSockets used a framing format
close to what 0MQ does, for efficiency reasons and (b) if 0MQ adopted
the final IETF WebSockets framing format, if it remains this similar.

Using an IETF-standard framing format would make the 0MQ wire-level
protocol a lot more compelling.

Cheers
Pieter


---------- Forwarded message ----------
From: Greg Wilkins <gregw at webtide.com>
Date: Mon, Aug 9, 2010 at 2:50 AM
Subject: Re: [hybi] hum #3: Message
To: John Tamplin <jat at google.com>
Cc: "hybi at ietf.org" <hybi at ietf.org>


All,

I do not disagree that message length (as opposed to frame length),
will often be a useful bit of meta-data for the receiver.

However, I think there is plenty of other meta-data that is related to
the total message that could be useful for the receiver: content-type,
content-encoding, expiry, etc are all things that a smart receiving to
potentially use to process/discard fragments on the fly without
waiting for the full message.

So I think it far better to support message length with a generic meta
data mechanism rather than give it a special spot in the first frame.
 Defining a mime-message op-code would be sufficient to allow us to
send a message with length and other meta data, that the framing layer
could then fragment into multiple chunks.

However, I do like having a very small minimal frame size for short
messages, so I would modify Johns proposal to:

 1 bit - fragment
 2 bits - reserved
 5 bits - opcode
 8 bits - short length
 [if length = 255: 6 bytes length]
 length bytes of payload

With opcodes defined for text, binary, mime and ping/pong frames

I've proposed a 48 bit length to make the total header 64bits, but I'm
ok with it being 64 bit length as well.


cheers





On 7 August 2010 00:17, John Tamplin <jat at google.com> wrote:
> Ok, does this summarize the various positions regarding chunking and
> lengths?
>
> We want to have a length of the complete message when it is available to
> make buffer management simpler for the receiver
> We want to be able to fragment messages to avoid extra buffering when the
> length of the entire message is not known up front, such as when sending
> dynamic content, compressing frames, etc.
> We want to keep small frames having small overhead (though there is a limit
> on how much this matters given TCP/IP overhead)
> We want to be able to send very large files in a single fragment so we can
> just hand it off to sendfile or equivalent
>
> What about the following:
>
> 1 bit - Initial fragment
> 1 bit - Final fragment
> 1 bit - reserved
> 5 bits - opcode
> 8 bits - short length
> [if length = 255: 8 bytes - length]
> [if Initial = 1 and Final = 0: 8 bytes - overall message length, with 0
> meaning "unknown"]
> length bytes of payload
>
> This could be extended with any of the extension or padding proposals out
> there, just trying to get the basic idea of having initial/fragment bits
> with an optional overall message length up for discussion.  The length could
> be a variable-length integer as in v76 if you prefer, likewise for the
> message length.
> --
> John A. Tamplin
> Software Engineer (GWT), Google
>
> _______________________________________________
> hybi mailing list
> hybi at ietf.org
> https://www.ietf.org/mailman/listinfo/hybi
>
>
_______________________________________________
hybi mailing list
hybi at ietf.org
https://www.ietf.org/mailman/listinfo/hybi