[zeromq-dev] zmq & mssql queue to insert data

[Baranski, Antek]

Actually what I am trying to build is a real time telemetry gathering system.

0MQ is used as the collector / processor of various data streams/sources in the system. We built a simple Java "plug-in" for 0MQ that we can configure using XML and process data using XSLT and its this processed data stream that I need to insert into a DB.

Any ideas on what the best way would be to go about this?

Antek S. Baranski
Technical Director - Development Transformation
EA Sports, Canada
Work : 	604 - 456 - 1746 (x11746)
Cell : 	778 - 668 - 7174

-----Original Message-----
From: Martin Sustrik [mailto:sustrik at fastmq.com] 
Sent: Wednesday, August 05, 2009 1:08 AM
To: Baranski, Antek
Cc: 'zeromq-dev at lists.zeromq.org'
Subject: Re: [zeromq-dev] zmq & mssql queue to insert data

Hi Antek,

> Has anyone ever tried coupling zmq & mssql queue to insert data into a DB?

There's no native MSSQL coupling in 0MQ for the time being.

The problem you may encounter when trying to insert data into DB by hand 
is that DBs are generally much slower than 0MQ thus it'll become the 
bottleneck of the system.

Optionally, if what you need is just buffering large queues on disk to 
avoid out-of-memory problems (as opposed to recovery from power 
failures) you can set "swap" parameter when creating a queue. Swapping 
queue data to disk is much faster than inserting it into DB.

Martin