[zeromq-dev] New Projects: NetMQ.WebSockets and JSMQ

Pieter Hintjens ph at imatix.com
Fri Jun 27 13:40:56 CEST 2014


Looks great. Would you make a pull request to the github repo, and
then you can also post that to the RFC wiki.

On Thu, Jun 26, 2014 at 11:43 PM, Doron Somech <somdoron at gmail.com> wrote:
> Pieter, Michael, I created a raw RFC, let me know what you think:
>
> https://github.com/somdoron/rfc/blob/master/spec_39.txt
>
> Michael, send me your github user I will add you as collaborator (also feel
> free to add yourself as editor and contributor to the RFC).
>
> Regards,
>
> Doron
>
>
> On Wed, Jun 25, 2014 at 3:34 PM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>> Nice. The RFCs are also in zeromq/rfc on GitHub.
>>
>> On Wed, Jun 25, 2014 at 12:19 PM, Doron Somech <somdoron at gmail.com> wrote:
>> > I will create a page at http://rfc.zeromq.org/ and will cover the
>> > framing,
>> > but I will only get it on the weekend...
>> >
>> >
>> >
>> >
>> > On Tue, Jun 24, 2014 at 12:40 PM, Michael Haberler <mail17 at mah.priv.at>
>> > wrote:
>> >>
>> >> Hi Doron,
>> >>
>> >>
>> >> Am 24.06.2014 um 09:36 schrieb Doron Somech <somdoron at gmail.com>:
>> >>
>> >> > what do you mean "ZMTP/multiple destination URI's dance"?
>> >>
>> >> well, I see the choices being either a pretty complete ZMTP
>> >> implementation
>> >> on the JS side of things - then you can multiplex, connect several
>> >> socket
>> >> destinations in JS and so forth; or one uses URI options, and the
>> >> proxy's
>> >> functionality, and only multipart framing is done over the ws
>> >> connection; I
>> >> prefer the latter because the first option looks like significant
>> >> effort for
>> >> no clear upside
>> >>
>> >> >
>> >> > He can create a page under http://rfc.zeromq.org/, anyway some of my
>> >> > thoughts:
>> >> > * Regarding using URI, i think we should use that for resource
>> >> > sharing
>> >> > (binding multiple sockets on same port with different service name),
>> >> > we can
>> >> > also use that for socket type (maybe as fragment) I'm not sure we
>> >> > should
>> >> > support identifies (I'm not sure what the gain for that).
>> >>
>> >> It does make sense at times to have clients legibly identified, eg in
>> >> logs; it's not much extra cost, just a URI k/v pair
>> >>
>> >> > * We can transfer the protocol version on the
>> >> > "Sec-WebSocket-Protocol"
>> >> > and later support server that can support multiple client versions.
>> >>
>> >> that is a clever idea, I like it!
>> >>
>> >> > * should we only support strings (UTF8)? Because the framing for UTF8
>> >> > and binary is different
>> >>
>> >> I dont see any value in that atm, but I might be missing something;
>> >> again
>> >> I had that as a k/v option in the URI
>> >>
>> >> > * I don't think we should start with multiplexing, I think it very
>> >> > complicated to do it right.
>> >>
>> >> nope, KISS wins the day
>> >>
>> >> > Other than that, lets start :-).
>> >>
>> >> how do we start on the multipart framing writeup? You sketch it, or I
>> >> disassemble your code ;-?
>> >>
>> >> One issue we need to think through is the handling of identities - eg
>> >> if
>> >> due to zmq proxies several identities are assembled, delimited with a
>> >> zero-length frame; the options are either to pass those through via WS
>> >> as-is, or maybe handle them at proxy proper if that is to be the last
>> >> zeromq
>> >> endpoint having a visible identity
>> >>
>> >>
>> >> cheers - Michael
>> >>
>> >>
>> >> >
>> >> > Doron
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Tue, Jun 24, 2014 at 9:20 AM, Michael Haberler
>> >> > <mail17 at mah.priv.at>
>> >> > wrote:
>> >> > Hi Doron,
>> >> >
>> >> >
>> >> > Am 23.06.2014 um 14:30 schrieb Doron Somech <somdoron at gmail.com>:
>> >> >
>> >> > > Hi Michael,
>> >> > >
>> >> > > You are right, every message start with ascii code of 1 or 0 to
>> >> > > indicate if more to come, regarding the RFC, I think it will be
>> >> > > great.
>> >> > >
>> >> > > We should think about more stuff like port sharing (multiple
>> >> > > sockets
>> >> > > on same port each with different URI) and sub protocols validation
>> >> > > (for
>> >> > > PubSub, Request Reply).
>> >> >
>> >> > Assuming we write up a recommendation/method to map zeromq onto
>> >> > websockets as an RFC-style document, I see several aspects to the
>> >> > task:
>> >> >
>> >> > - define a mapping how multiframe messages are en/decoded, assuming
>> >> > the
>> >> > current ws connection handles a single connection (say a
>> >> > dealer/router or
>> >> > xsub/xpub pair); that would essentially cover non-mutilation of the
>> >> > frame
>> >> > structure, but not aspects like multiple destinations handled in one
>> >> > socket
>> >> > - do the whole ZMTP/multiple destination URI's dance on the JS side.
>> >> > - multiplexing several sockets over a single ws connection
>> >> > - define if the ws connect URI (which can be viewed as pre-connect,
>> >> > out
>> >> > of band information) is used to carry extra setup, like carry socket
>> >> > type,
>> >> > identity, make the proxy connect to several target URI's etc
>> >> >
>> >> > The first step IMO is essential; curious - does your scheme proxy
>> >> > zero-length frames properly? I'm not fully up to speed on ws specs if
>> >> > zero-length frames are passed properly.
>> >> > In my application scenario I dont have any upside for (2) and (3),
>> >> > and
>> >> > I'm a bit concerned about feature creep
>> >> > I think (4) would cover much of (2)
>> >> > (2) and (3) would probably require more complex framing than just (1)
>> >> > and (4), which would be my preferred goal
>> >> >
>> >> > I think the decision to take is 'full ZMTP JS-side' and more complex
>> >> > framing, versus one ws connection mapped onto a socket (which may
>> >> > connect to
>> >> > several URI's at the proxy), and simpler framing. I fear the framing
>> >> > methods
>> >> > would be incompatible if one started with the second goal and tried
>> >> > to
>> >> > achieve to the first thereafter, for what I see limited upside.
>> >> >
>> >> > What I propose is we formulate the framing procedure for the
>> >> > single-socket connection case for a start, and go from there.
>> >> >
>> >> > - Michael
>> >> >
>> >> >
>> >> >
>> >> > >
>> >> > > Regards,
>> >> > >
>> >> > > Doron
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Mon, Jun 23, 2014 at 2:53 PM, Michael Haberler
>> >> > > <mail17 at mah.priv.at>
>> >> > > wrote:
>> >> > > Hi Doron,
>> >> > >
>> >> > > Am 23.06.2014 um 10:25 schrieb Doron Somech <somdoron at gmail.com>:
>> >> > >
>> >> > > > Hi All,
>> >> > > >
>> >> > > > I'd like to introduce two new projects I'm working on:
>> >> > > >
>> >> > > > https://github.com/somdoron/JSMQ - ZeroMQ/NetMQ javascript client
>> >> > > > over WebSockets
>> >> > > > https://github.com/somdoron/NetMQ.WebSockets - WebSockets
>> >> > > > extension
>> >> > > > to NetMQ, uses stream socket type and provide a new socket object
>> >> > > > that has
>> >> > > > very similar interface to NetMQ socket object.
>> >> > > >
>> >> > > > Both available on nuget (include prerelease) and at a beta stage.
>> >> > > >
>> >> > > > You can read more about the projects at my blog:
>> >> > > > http://somdoron.com/2014/06/introducing-netmq-websockets-jsmq/
>> >> > >
>> >> > > very interesting since I'm on a related venture - relaying zmq
>> >> > > router/dealer and xsub/xpub to JS via libwebsockets (C++)
>> >> > >
>> >> > > I'm a complete JS retard, but if I understand your JSMQ layer right
>> >> > > it
>> >> > > essential wraps multipart frames (with leading 0/1 per frame ==
>> >> > > MORE flag)
>> >> > > over ws frames so they can be assembled/sent from normal zmq
>> >> > > multipart
>> >> > > frames in the proxy, and message structure retained?
>> >> > >
>> >> > > if this is so I'll adopt your scheme for the zmq/ws proxy I'm
>> >> > > working
>> >> > > on, because preservation of multiframe messages on the ws side is
>> >> > > an open
>> >> > > issue for me, and with your approach it would be more 'end-to-end',
>> >> > > the
>> >> > > zmq/ws proxy being a zeromq proxy proper in terms of socket
>> >> > > identity
>> >> > >
>> >> > > maybe this kind of ws framing warrants a bit of an RFC? happy to
>> >> > > co-author - could help interoperability downstream
>> >> > >
>> >> > > - Michael
>> >> > >
>> >> > > ps: where I am : http://goo.gl/4TfWBh
>> >> > >
>> >> > > nonobvious aspects are:
>> >> > > - a key feature is automatic JSON <-> protobuf conversion since we
>> >> > > use
>> >> > > protobuf-over-zmq internally, see: http://goo.gl/9OEcHY and
>> >> > > http://goo.gl/sY6sEI respectively
>> >> > > - I use URI arguments to drive proxy behavior: http://goo.gl/FVddoJ
>> >> > > -
>> >> > > rather flexible to tack on this or that option, uses liburiparser
>> >> > >   a client-side URI to connect to the proxy could look like so:
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > >
>> >> > > > Regards,
>> >> > > >
>> >> > > > Doron
>> >> > > > _______________________________________________
>> >> > > > zeromq-dev mailing list
>> >> > > > zeromq-dev at lists.zeromq.org
>> >> > > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> > >
>> >> > > _______________________________________________
>> >> > > zeromq-dev mailing list
>> >> > > zeromq-dev at lists.zeromq.org
>> >> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> > >
>> >> > > _______________________________________________
>> >> > > zeromq-dev mailing list
>> >> > > zeromq-dev at lists.zeromq.org
>> >> > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> > _______________________________________________
>> >> > zeromq-dev mailing list
>> >> > zeromq-dev at lists.zeromq.org
>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> > _______________________________________________
>> >> > zeromq-dev mailing list
>> >> > zeromq-dev at lists.zeromq.org
>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >>
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



More information about the zeromq-dev mailing list