[zeromq-dev] ZeroMQ, jzmq, and J2EE

[Marko Mikulicic]

On 14 April 2011 05:33, Joshua Foster <jhawk28 at gmail.com> wrote:

> Has anyone had success using zeromq in a J2EE application?
>

Yes, I used it to quickly (1 day work) bypass a scalability issue on by java
stack (jetty->cxf->solr) and
reach 800 query-per-sec rate using a nodejs proxy on a 8 core machine
previously 50 qps on pure CXF due to a lot of lock contention in various
layers
I also tried a thrift rpc between the nodejs proxy and the java app but I
obtained better results with the zmq approach.

The nodejs proxy creates a XREQ socket, binds it to well known port and for
each incoming http requests encodes the parsed soap arguments in a zmq
message.
The java application spawns a number of threads each of them with a REP
socket and connect to the nodejs xrep socket.

Advantages of this approach:

 1) quick to develop
 2) autoreconnect. I can restart both the proxy and the java container
without worrying about reconnection (unlike with thrift, where pooling and
reconnection had to be handled manually because connection pools are
application specific (see hector etc) and not available as generic tools
managing.
 3) low overhead
 4) no zmq devices to setup

Disadvantages:

 1) the simple xreq->rep socket layout does simple roundrobin and doesn't
balance the workers in an optimal way if the requests vary in length.
 2) when I hit a limit in the load that one single nodejs proxy could handle
I used "spark2" to spawn multiple nodejs http servers. Each of them had his
own
zeromq listening socket. I had to change the java code so that each socket
connected to each possible nodejs proxy port (in a given range). The
solution was quick and worked out of the box, but smells.
 3) require 2 native shared libraries (libzmq itself the java binding native
lib) + the nodejs binding

Software:

1) libzmq 2.1.4 from chris-lea ubuntu ppa
2) jzmq binding compiled from git
3) npm "zeromq at 0.4.0" package
3) npm "spark2 at 2.0.11" package

Zeromq saved my day, as it enabled me to quickly workaround a difficult to
debug scalability issue.

Cheers,
Marko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110414/412e71cf/attachment.htm>