5 ms·

If anyone from the company's reading this: I'd take a guess that it's wanting "...to ensure that TiFlash processes the latest (not just fresh) data" is a major

If anyone from the company's reading this: I'd take a guess that it's wanting "...to ensure that TiFlash processes the latest (not just fresh) data" is a major problem. If you can tolerate even a few minutes-old data then I really think you could use some dumb kind of replication to another machine which would be much simpler. Also a billion rows of data isn't really so much, depending on what you're storing. (I'm trying to get 250 million rows (of 1 character) into a db to see how fast it counts when in memory. Still inserting) Edit: TSQL, counting 268 million rows all mem resident, 7-year old CPU, forcing it to use 1 core is 12 secs, allowing it to parallelise over 2 cores (all I have) is 8 secs. This is for comparison to "I did a simple test with five TiKV nodes, one TiFlash node, and one table with 250 million rows of data. I tried to count the table records. When I queried by index in TiKV, it took me more than 40 seconds to get the result. But when I queried in a single TiFlash node, it took only 10 seconds." I used single bytes per row but I know from prior testing that fat rows and thin rows count about the same speed.

gregwebs 6y ago
The TiFlash replication is sub-second and happens automatically. We are going to be publishing a more extensive paper on TiFlash, but the point is that the operational burden is equivalent to a read replica but you get the column-based storage speed of a data warehouse. I agree the data size doesn't seem extraordinarily big from some of the description. Later in the article it states a size of 2.4 TB. It is also true that in-memory databases should be faster. TiDB is designed for data that is too large to keep in memory. Note that having 5 TiKV nodes isn't going to make this single count query any faster than having 1 TiKV node. TiDB is designed for high availability, so a default TiKV setup has 3 nodes each with a copy of the data. TiKV can be scaled out horizontally to provide more compute power to handle a larger workload but it is not likely to make a single query on an idle machine any faster.
- jinqueeny 6y ago
  The link to the paper (on VLDB 2020) is now available: http://www.vldb.org/pvldb/vol13/p3072-huang.pdf http://www.vldb.org/pvldb/vol13/p3072-huang.pdf
ilovesoup 6y ago
Well, its a lot easier to use TiFlash than replicating your OLTP data to a read-optimized database (whatever it is). 1. Add TiFlash nodes to your TiDB cluster (you might need to config some ports and ip address just like other databases and then start the nodes) 2. ALTER TABLE your_table set TiFlash replica 1; That's it. You won't need all data in memory and replicating details are taken care of silently (schema change, fault-tolerance, load-balance, consistency and etc). I believe heterogeneous replicating even with minutes latency tolerance will not be as easy as above :) PS: the good part of keep TP/AP consistent is that user can use them in a single application without any surprise: it's just a different forms of a single logical data. PS2: I'm a dev of PingCAP.

gregwebs 6y ago

The TiFlash replication is sub-second and happens automatically. We are going to be publishing a more extensive paper on TiFlash, but the point is that the operational burden is equivalent to a read replica but you get the column-based storage speed of a data warehouse. I agree the data size doesn't seem extraordinarily big from some of the description. Later in the article it states a size of 2.4 TB. It is also true that in-memory databases should be faster. TiDB is designed for data that is too large to keep in memory. Note that having 5 TiKV nodes isn't going to make this single count query any faster than having 1 TiKV node. TiDB is designed for high availability, so a default TiKV setup has 3 nodes each with a copy of the data. TiKV can be scaled out horizontally to provide more compute power to handle a larger workload but it is not likely to make a single query on an idle machine any faster.

jinqueeny 6y ago
The link to the paper (on VLDB 2020) is now available: http://www.vldb.org/pvldb/vol13/p3072-huang.pdf http://www.vldb.org/pvldb/vol13/p3072-huang.pdf

ilovesoup 6y ago

Well, its a lot easier to use TiFlash than replicating your OLTP data to a read-optimized database (whatever it is). 1. Add TiFlash nodes to your TiDB cluster (you might need to config some ports and ip address just like other databases and then start the nodes) 2. ALTER TABLE your_table set TiFlash replica 1; That's it. You won't need all data in memory and replicating details are taken care of silently (schema change, fault-tolerance, load-balance, consistency and etc). I believe heterogeneous replicating even with minutes latency tolerance will not be as easy as above :) PS: the good part of keep TP/AP consistent is that user can use them in a single application without any surprise: it's just a different forms of a single logical data. PS2: I'm a dev of PingCAP.