4 ms·
> I am skeptical that a networked database with multiple nodes can match the performance of a single master database such as MySQL, PostgreSQL, or SQL Server.
by darksaints 9y ago
> I am skeptical that a networked database with multiple nodes can match the performance of a single master database such as MySQL, PostgreSQL, or SQL Server.
It likely can't unless there is some black magic going on. Single node speed will always be faster. But once you get to that point where a single node chokes on the amount of data or query throughput that you have, you don't really have a choice anymore.
My personal plan is to start with Postgres RDS. Grow until RDS doesn't work anymore, then move to ultra beefy bare metal servers in colocation with AWS Direct Connect. If I ever outgrow an 4x24 core server with 3TB of memory on a RAIDed NVMe disk cluster, I might move to Citus.
For the various distributed database companies out there, I believe the one that will win in the marketplace is the one working or partnering to develop specialized hardware and networking, and then optimizing for it.
- p0rkbelly 9y agoWhy not bare metal on AWS? Going to DX is going to give you a conservative 10ms on your calls...
- darksaints 9y agoThat might be a good intermediate option, but they still don't have instances with 4x24 cpu, or >488GB ram.
- openasocket 9y agoDid you look at the x1 options? Up to 128 CPUs and 3,904GB of RAM on the x1e.32xlarge.
- darksaints 9y agoThose aren't bare metal. And in terms of database scalability, I would jump for bare metal before beefier servers, due to the fact that the disk is local and you don't have to deal with EBS noisy neighbors.
- openasocket 9y agoThe x1e.32xlarge has 4TB of SSD in addition to dedicated EBS bandwidth.
- saosebastiao 9y agoI haven't tried it, but IIRC that's only network bandwidth to the block store. Better than the alternative for sure, but I would assume disk access is still noisy unless you have a dedicated drive.
- openasocket 9y agoTrue, but you do have 4TB dedicated SSD
- welder 9y agoThe max IOPS for provisioned EBS is less than 10% the max IOPS of dedicated hardware SSDs.
- deleted 9y ago[deleted]
- riku_iki 9y ago> then move to ultra beefy bare metal servers in colocation with AWS Direct Connect. You will still need another beefy server in another datacenter with configured replication and failover, which is one of the problem Cockroach is trying to solve.
- cat199 9y ago> I believe the one that will win in the marketplace is the one working or partnering to develop specialized hardware and networking, and then optimizing for it. soo... oracle? :b
- styfle 9y agoThere is some black magic[0] going on with BedrockDB. They are using a single node at Expensify and getting 4 million queries per second. [0]: https://news.ycombinator.com/item?id=16118776 https://news.ycombinator.com/item?id=16118776
- manigandham 9y agoA distributed db running on a single node isn't distributed, so that is just showing the performance of SQLite on a single node, which is the point of the thread. Also bedrock has plenty of problems and is highly unrecommended.