thanks, I had a look at gsn - That would be the middleware to gather the data from sensor networks... ..whereas my interest is in how to deal with the fire-hos

thanks, I had a look at gsn - That would be the middleware to gather the data from sensor networks... ..whereas my interest is in how to deal with the fire-hose of ~10Gb data per day, once you have 100Gb of it on disk and more streaming in all the time. I'm aware of commercial offerings - - KDB+ [its own semi-functional terse languages - K & Q ] - StreamBase [extended SQL] The open source ones ... - Aurora / Borealis ( MIT, Brandeis) - Medusa (MIT) - Streams (Stanford) - MonetDB (a Netherlands university) seem to embrace/extend SQL and/or XQuery and/or RDF Im thinking there must be a better way.