[zeromq-dev] zmq & mssql queue to insert data

[Baranski, Antek]

The java plug-in lives in the production network my DB lives in the office network and the connectivity between the 2 is very much limited.

We could off course write an app on the receiving side that listen to the various 0MQ queues and insert the data that way, but I was hoping not having to create a server/app to do this and use MSSQL's queue mechanism but that would only work if I can somehow hookup the 0MQ & MSSQL queues which probably is a no-go judging by the responses so far.


Antek S. Baranski
Technical Director - Development Transformation
EA Sports, Canada
Work : 	604 - 456 - 1746 (x11746)
Cell : 	778 - 668 - 7174


-----Original Message-----
From: pieterh at gmail.com [mailto:pieterh at gmail.com] On Behalf Of Pieter Hintjens
Sent: Wednesday, August 05, 2009 12:34 PM
To: Baranski, Antek
Cc: Martin Sustrik; zeromq-dev at lists.zeromq.org
Subject: Re: [zeromq-dev] zmq & mssql queue to insert data

On Wed, Aug 5, 2009 at 7:14 PM, Baranski, Antek<abaranski at ea.com> wrote:

> Actually what I am trying to build is a real time telemetry gathering system.
>
> 0MQ is used as the collector / processor of various data streams/sources in the system. We built a simple Java "plug-in" for 0MQ that we can configure using XML and process data using XSLT and its this processed data stream that I need to insert into a DB.
>
> Any ideas on what the best way would be to go about this?

The simplest is perhaps to write to a log file that you process either
continuously or regularly, depending on the volume.  You can write a
batch processor in any random language then.

Any reason you don't do the database inserts immediately from the Java plug in?

-Pieter