4 ms·

Minor quibble: the article is about RiakTS, their time-series enhanced version of riak_core. riak_core's main strength is that it does key-value in a distribut

by athenot 10y ago

Minor quibble: the article is about RiakTS, their time-series enhanced version of riak_core. riak_core's main strength is that it does key-value in a distributed/resilient manner, spreading values in multiple copies (at least 3) all over a cluster of servers. Kill one server, no problem. Need more capacity, add servers and it will rebalance itself. The TS part is just an optimization built on top of that, to make values which are near each other in time to be near each other in storage, for faster range-base retrieval.

siculars 10y ago
Yes, exactly. (Author)
- eggy 10y ago
  TS enhanced version of riak_core, got it. Most TS, or tick DBs, are columnar-based, memory-mapped, fast and light systems. Are there any benchmarks similar to STAC-M3, which is a year's worth of NYSE data run on different hardware to gauge kdb+'s effectiveness on different hardware configurations [1]? It's a great way to gauge performance and TCO. Does it do both memory (streaming data) and disk-based (historical) storage for big data set analytics in realtime? I'd be interested to see numbers there. A lot of people think kdb+ is only for finance. There is a conference coming up in May that will have talks on q (the language for the kdb+ database) about natural language processing and machine learning in q to name a few. Another is about using it at a power plant to most efficiently route power based upon realtime data [2]. I only got into kdb+ and q with the free, non-commercial 32-bit version. I usually use J and sometimes APL, which had MapReduce since at least the 80s for APL.Check out this post from 2009 [3]. I guess the 'new shiny' bit threw me in your chosen title. [1] https://stacresearch.com/news/2014/02/13/stac-reports-intel-ivy-bridge-ex-stac-m3-and-kdb https://stacresearch.com/news/2014/02/13/stac-reports-intel-... [2] https://kxcon2016.com/agenda/ https://kxcon2016.com/agenda/ [3] http://blog.data-miners.com/2009/04/mapreduce-hadoop-everything-old-is-new.html http://blog.data-miners.com/2009/04/mapreduce-hadoop-everyth...
  - srpeck 10y ago
    You might find these benchmarks interesting: http://kparc.com/q4/readme.txt http://kparc.com/q4/readme.txt
    - eggy 10y ago
      I was inquiring about benchmarks for RiakTS, but your link was perfect. I am a J/APL dabbler, and quite recently learning kdb+/q (I prefer k). As much as I step away from these languages, I always find my way back to them in strange ways. I was studying music, and there was a great J article in Vector magazine written in August 2006 [1] that walks through scales, and other musical concepts in J. A Forth-based music software called Sporth [2] has a kona ugen in it, so you can generate scales or other musical items in kona, and then use them in the stack-based Sporth audio language. My interests in kdb+/q, k, J and APL are in applying them to mathematical investigations of music, visuals, doing data analysis, and then just code golfing, or toying around. They're so much fun! I need more time on large streaming datasets (Time Series data), than large disk-based datasets to really test latencies. I am building a box much better suited for it than my current machine. The goal is to stay in RAM as much as possible. [1] http://archive.vector.org.uk/art10010610 http://archive.vector.org.uk/art10010610 [2] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth
      - srpeck 10y ago
        You should definitely check out JohnEarnest/RodgerTheGreat's iKe, built on his open source k interpreter in JS. Fun examples: http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d6132bf961976524393a5517e5 http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d613... and http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04de8efc937f146b2b77904e http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04... https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok And related APL/J/K subreddit: https://www.reddit.com/r/apljk/ https://www.reddit.com/r/apljk/
        10y ago

by athenot 10y ago

siculars 10y ago

Yes, exactly. (Author)

eggy 10y ago
TS enhanced version of riak_core, got it. Most TS, or tick DBs, are columnar-based, memory-mapped, fast and light systems. Are there any benchmarks similar to STAC-M3, which is a year's worth of NYSE data run on different hardware to gauge kdb+'s effectiveness on different hardware configurations [1]? It's a great way to gauge performance and TCO. Does it do both memory (streaming data) and disk-based (historical) storage for big data set analytics in realtime? I'd be interested to see numbers there. A lot of people think kdb+ is only for finance. There is a conference coming up in May that will have talks on q (the language for the kdb+ database) about natural language processing and machine learning in q to name a few. Another is about using it at a power plant to most efficiently route power based upon realtime data [2]. I only got into kdb+ and q with the free, non-commercial 32-bit version. I usually use J and sometimes APL, which had MapReduce since at least the 80s for APL.Check out this post from 2009 [3]. I guess the 'new shiny' bit threw me in your chosen title. [1] https://stacresearch.com/news/2014/02/13/stac-reports-intel-ivy-bridge-ex-stac-m3-and-kdb https://stacresearch.com/news/2014/02/13/stac-reports-intel-... [2] https://kxcon2016.com/agenda/ https://kxcon2016.com/agenda/ [3] http://blog.data-miners.com/2009/04/mapreduce-hadoop-everything-old-is-new.html http://blog.data-miners.com/2009/04/mapreduce-hadoop-everyth...
- srpeck 10y ago
  You might find these benchmarks interesting: http://kparc.com/q4/readme.txt http://kparc.com/q4/readme.txt
  - eggy 10y ago
    I was inquiring about benchmarks for RiakTS, but your link was perfect. I am a J/APL dabbler, and quite recently learning kdb+/q (I prefer k). As much as I step away from these languages, I always find my way back to them in strange ways. I was studying music, and there was a great J article in Vector magazine written in August 2006 [1] that walks through scales, and other musical concepts in J. A Forth-based music software called Sporth [2] has a kona ugen in it, so you can generate scales or other musical items in kona, and then use them in the stack-based Sporth audio language. My interests in kdb+/q, k, J and APL are in applying them to mathematical investigations of music, visuals, doing data analysis, and then just code golfing, or toying around. They're so much fun! I need more time on large streaming datasets (Time Series data), than large disk-based datasets to really test latencies. I am building a box much better suited for it than my current machine. The goal is to stay in RAM as much as possible. [1] http://archive.vector.org.uk/art10010610 http://archive.vector.org.uk/art10010610 [2] https://github.com/PaulBatchelor/Sporth https://github.com/PaulBatchelor/Sporth
    - srpeck 10y ago
      You should definitely check out JohnEarnest/RodgerTheGreat's iKe, built on his open source k interpreter in JS. Fun examples: http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d6132bf961976524393a5517e5 http://johnearnest.github.io/ok/ike/ike.html?gist=bbab46d613... and http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04de8efc937f146b2b77904e http://johnearnest.github.io/ok/ike/ike.html?gist=b741444d04... https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok/tree/gh-pages/ike https://github.com/JohnEarnest/ok https://github.com/JohnEarnest/ok And related APL/J/K subreddit: https://www.reddit.com/r/apljk/ https://www.reddit.com/r/apljk/
      - 10y ago