5 ms·
Does it blow anyone else's mind that database servers are so powerful nowadays that you can just load half a billion rows into a SINGLE TABLE and pretty much ex
by jtchang 14y ago
Does it blow anyone else's mind that database servers are so powerful nowadays that you can just load half a billion rows into a SINGLE TABLE and pretty much expect things to work? Obviously there are caveats but all in all a nice chunk of RAM backed by a large SSD goes a long way.
- jpitz 14y agoMy telemetry table is >2B rows and growing. 5B is starting to worry me; .5B, not so much. This is even backed by ( not much ) spinning rust.
- spudlyo 14y agoHopefully you don't have an auto-increment int (2^31) as a PK, or you'll be worrying sooner than you think.
- rs 14y agoThat's one reason why switching auto-increments to bigint does help (a lot): http://dev.mysql.com/doc/refman/5.5/en/integer-types.html http://dev.mysql.com/doc/refman/5.5/en/integer-types.html
- Cyndre 14y agoThat is interesting to know, I have been setting my index's to bigint for a long time.
- jpitz 14y agoUgh. No. This uses a multi-column natural key. I have a tendency to avoid synthetic keys when I can.
- fleitz 14y agoI'm more impressed with how cheap it is. An SSD is on par with a 15K disk, but delivers 10X the perf, throw 10 SSDs in an array and you've got what used to cost $100,000 for about $5K
- Andys 14y agoBut SSDs that cost the same as a 15K disk do not have anywhere near the life time of the spinning disk due to write endurance limits. So load half a billion rows but don't update it too often!
- deleted 14y ago[deleted]
- fleitz 14y agoGiven the cost of power and space and vs. the number of additional drives necessary I'm not sure that longevity is worth it.
- henrikschroder 14y agoSSDs live long enough. You get magnitudes more IOPS performance out of a single SSD than a RAID of 15k disks, it consumes magnitudes less power, it costs less, and it needs less maintenance. It's a complete no-brainer these days to put an SSD in your database server.
- jemfinch 14y agoAs a long time skeptic of the NoSQL craze that seems to have (happily!) peaked, I have to say that no, I'm not very surprised by this. Databases (real databases) are using indexing technology that's been proven over not just years but decades of use and refinement. log2(500000000) is still only 28. Double that and it's only 29. And these databases' indexes are typically btrees with much higher branching factors than 2. One of my favorite articles on this is Dennis Forbes' "The Impact of SSDs on Database Performance and the Performance Paradox of Data Explodification" (http://www.yafla.com/dforbes/The_Impact_of_SSDs_on_Database_Performance_and_the_Performance_Paradox_of_Data_Explodification/ http://www.yafla.com/dforbes/The_Impact_of_SSDs_on_Database_...).
- gaius 14y agoI was impressed by stuff like this in the 90s ;-) What's impressive now is not the software but that the hardware is so cheap.