4 ms·

Yes, exactly. (Author)

eggy 10y ago
TS enhanced version of riak_core, got it. Most TS, or tick DBs, are columnar-based, memory-mapped, fast and light systems. Are there any benchmarks similar to STAC-M3, which is a year's worth of NYSE data run on different hardware to gauge kdb+'s effectiveness on different hardware configurations [1]? It's a great way to gauge performance and TCO. Does it do both memory (streaming data) and disk-based (historical) storage for big data set analytics in realtime? I'd be interested to see numbers there. A lot of people think kdb+ is only for finance. There is a conference coming up in May that will have talks on q (the language for the kdb+ database) about natural language processing and machine learning in q to name a few. Another is about using it at a power plant to most efficiently route power based upon realtime data [2]. I only got into kdb+ and q with the free, non-commercial 32-bit version. I usually use J and sometimes APL, which had MapReduce since at least the 80s for APL.Check out this post from 2009 [3]. I guess the 'new shiny' bit threw me in your chosen title. [1] https://stacresearch.com/news/2014/02/13/stac-reports-intel-ivy-bridge-ex-stac-m3-and-kdb https://stacresearch.com/news/2014/02/13/stac-reports-intel-... [2] https://kxcon2016.com/agenda/ https://kxcon2016.com/agenda/ [3] http://blog.data-miners.com/2009/04/mapreduce-hadoop-everything-old-is-new.html http://blog.data-miners.com/2009/04/mapreduce-hadoop-everyth...
- srpeck 10y ago
  You might find these benchmarks interesting: http://kparc.com/q4/readme.txt http://kparc.com/q4/readme.txt
  - eggy 10y ago
    I was inquiring about benchmarks for RiakTS, but your link was perfect. I am a J/APL dabbler, and quite recently learning kdb+/q (I prefer k). As much as I step away from these languages, I always find my way back to them in strange ways. I was studying music, and there was a great J article in Vector magazine written in August 2006 [1] that walks through scales, and other musical concepts in J. A Forth-based music software called Sporth [2] has a kona ugen in it, so you can generate scales or other musical items in kona, and then use them in the stack-based Sporth audio language. My interests in kdb+/q, k, J and APL are in applying them to mathematical investigations of music, visuals, doing data analysis, and then just code golfing, or toying around. They're so much fun! I need more time on large streaming datasets (Time Series data), than large disk-based datasets to really test latencies. I am building a box much better suited for it than my current machine. The goal is to stay in RAM as much as possible. [1] http://archive.vector.org.uk/art10010610 http://archive.vector.org.uk/art10010610 [2] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth
    - srpeck 10y ago
      You should definitely check out JohnEarnest/RodgerTheGreat's iKe, built on his open source k interpreter in JS. Fun examples: http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d6132bf961976524393a5517e5 http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d613... and http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04de8efc937f146b2b77904e http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04... https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok And related APL/J/K subreddit: https://www.reddit.com/r/apljk/ https://www.reddit.com/r/apljk/
      - eggy 10y ago
        I had stumbled upon John's work before. I am currently dabbling with a stack-based audio language called Sporth [1], and messing with the idea of somehow mashing it up with John's ike project. See, vector/array languages aren't just for FinTech or Time Series! [1] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth

eggy 10y ago

TS enhanced version of riak_core, got it. Most TS, or tick DBs, are columnar-based, memory-mapped, fast and light systems. Are there any benchmarks similar to STAC-M3, which is a year's worth of NYSE data run on different hardware to gauge kdb+'s effectiveness on different hardware configurations [1]? It's a great way to gauge performance and TCO. Does it do both memory (streaming data) and disk-based (historical) storage for big data set analytics in realtime? I'd be interested to see numbers there. A lot of people think kdb+ is only for finance. There is a conference coming up in May that will have talks on q (the language for the kdb+ database) about natural language processing and machine learning in q to name a few. Another is about using it at a power plant to most efficiently route power based upon realtime data [2]. I only got into kdb+ and q with the free, non-commercial 32-bit version. I usually use J and sometimes APL, which had MapReduce since at least the 80s for APL.Check out this post from 2009 [3]. I guess the 'new shiny' bit threw me in your chosen title. [1] https://stacresearch.com/news/2014/02/13/stac-reports-intel-ivy-bridge-ex-stac-m3-and-kdb https://stacresearch.com/news/2014/02/13/stac-reports-intel-... [2] https://kxcon2016.com/agenda/ https://kxcon2016.com/agenda/ [3] http://blog.data-miners.com/2009/04/mapreduce-hadoop-everything-old-is-new.html http://blog.data-miners.com/2009/04/mapreduce-hadoop-everyth...

srpeck 10y ago
You might find these benchmarks interesting: http://kparc.com/q4/readme.txt http://kparc.com/q4/readme.txt
- eggy 10y ago
  I was inquiring about benchmarks for RiakTS, but your link was perfect. I am a J/APL dabbler, and quite recently learning kdb+/q (I prefer k). As much as I step away from these languages, I always find my way back to them in strange ways. I was studying music, and there was a great J article in Vector magazine written in August 2006 [1] that walks through scales, and other musical concepts in J. A Forth-based music software called Sporth [2] has a kona ugen in it, so you can generate scales or other musical items in kona, and then use them in the stack-based Sporth audio language. My interests in kdb+/q, k, J and APL are in applying them to mathematical investigations of music, visuals, doing data analysis, and then just code golfing, or toying around. They're so much fun! I need more time on large streaming datasets (Time Series data), than large disk-based datasets to really test latencies. I am building a box much better suited for it than my current machine. The goal is to stay in RAM as much as possible. [1] http://archive.vector.org.uk/art10010610 http://archive.vector.org.uk/art10010610 [2] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth
  - srpeck 10y ago
    You should definitely check out JohnEarnest/RodgerTheGreat's iKe, built on his open source k interpreter in JS. Fun examples: http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d6132bf961976524393a5517e5 http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d613... and http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04de8efc937f146b2b77904e http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04... https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok And related APL/J/K subreddit: https://www.reddit.com/r/apljk/ https://www.reddit.com/r/apljk/
    - eggy 10y ago
      I had stumbled upon John's work before. I am currently dabbling with a stack-based audio language called Sporth [1], and messing with the idea of somehow mashing it up with John's ike project. See, vector/array languages aren't just for FinTech or Time Series! [1] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth