5 ms·
I would be interested to hear what you consider high volume (writes/second). I am supporting a manufacturing system and it sits at the moment at 7 000 writes/se
by testrun 12y ago
I would be interested to hear what you consider high volume (writes/second). I am supporting a manufacturing system and it sits at the moment at 7 000 writes/second (it is a normal time series, ie id,time,value,quality).
- Ixiaus 12y ago7000 writes per-second is pretty low for many high-volume time series needs. Though it's not usually write throughput that most of these technologies are worried about. It's usually compression using dsp methods, aggregate stream folding computations, etc... that matters.
- testrun 12y agoStill waiting what is considered as high. Compression etc are not part of the question.
- stingraycharles 12y agoI worked in adserving. On peak moments we were doing around 350,000 updates a second.
- gd1 12y agoJust queried my db, and 300k a second is a typical peak in CME futures (across all products). I'd imagine that options could be a lot more than that.
- Ixiaus 12y agoChill dude, I was being conversive. Your attitude makes me not even want to provide you with my own data now.
- hyperliner 12y agoThat took some restrain. Thanks for being nice. I got your point originally but some people think everything needs to be an argument or worse a fight.
- carlivar 12y agoEngineers wanting data isn't exactly surprising. I would not consider it attitude.
- smiler 12y agoAre you using SQL or ?
- testrun 12y agoISAM
- jandrewrogers 12y agoThe systems I deal with start at millions of writes per second and go up from there. I have heard of systems that do over a billion writes per second, though I have not breached that threshold personally (yet). From an IoT or sensor network standpoint, 7000 writes/second is an idle server.
- testrun 12y agowhat db are you using?
- bladecatcher 12y agoI don't have numbers handy. The real power of kdb+/q comes from the column oriented architecture and the extremely powerful vector functional language q. The language q is inspired by APL. I highly recommend you to check out this article to get a sense of the APL family of languages https://scottlocklin.wordpress.com/2013/07/28/ruins-of-forgotten-empires-apl-languages/ https://scottlocklin.wordpress.com/2013/07/28/ruins-of-forgo... If you want a database for blazing fast data-storage and retrieval, there are many options available. You start seeing the real benefits of kdb+/q when you use q to simplify very complex operations that aren't easily done in SQL. Also, the high level operators that q makes your code extremely terse. I've written complex backtesting systems that perform data mining on massive datasets - all in one page of very tight q code!