3 ms·
I've always found doing logic and processing in the database to be much faster and much more efficient than doing it outside. The closer you can process data to
by joevandyk 14y ago
I've always found doing logic and processing in the database to be much faster and much more efficient than doing it outside. The closer you can process data to the data store, the faster it goes. Using materialized views can get you a long ways.
There's also no reason why you can't use multiple smaller databases. Use dblink. http://www.postgresql.org/docs/9.1/static/contrib-dblink-connect.html http://www.postgresql.org/docs/9.1/static/contrib-dblink-con...
I've never had a problem managing "schema and dependencies" on larger systems. Not sure what you are referring to there.
- gouranga 14y agoIt goes much faster I agree. However you are very short sighted. That black box doesn't go on forever. It will work fine for the first 50 users, seems ok for the next 100, 500 we're doing ok. 1000 BOOM! Then what do you do? I'd rather have a slightly higher latency and be able to scale to 100,000 users. Materialized views are not needed with CQRS architecture. ALL tables (or sets) are materialized views already. No joins are ever made. dblink is ugly and if you ask me a problem, not a solution (latency, network io, memory). I'm not sure what you classify as "larger" but I'm talking tens of Tb, 1500 tables and over 2 million LOC in the application.
- joevandyk 14y agoInstagram did just fine with postgresql with 27 million users. Many logical shards spread over a smaller amount of physical ones. http://www.databasesoup.com/2012/04/sharding-postgres-with-instagram.html http://www.databasesoup.com/2012/04/sharding-postgres-with-i...
- gouranga 14y agoInstagram isn't very complicated to be honest so I doubt that's a good scalability example.
- joevandyk 14y agoSkype's plan to support 1 billion users w/ postgres a few years ago? http://highscalability.com/skype-plans-postgresql-scale-1-billion-users http://highscalability.com/skype-plans-postgresql-scale-1-bi... and http://www.slideshare.net/adorepump/small-overview-of-skype-database-tools http://www.slideshare.net/adorepump/small-overview-of-skype-... Billions of records, 10,000 transactions per second. This was a couple of years ago. Dunno if Skype is complicated enough for you.
- erichocean 14y agoI get the sense he'll ratchet up what "complicated" means until he wins the argument. :)
- gouranga 14y agoNo just there is a difference between large and complicated.
- bingbing 14y agoEVE Online runs on a single SQL Server 2008 instance: http://wiki.eveonline.com/en/wiki/Tranquility#Database_Server http://wiki.eveonline.com/en/wiki/Tranquility#Database_Serve... serving hundreds of thousands of users
- gouranga 14y ago> At peak hours as of 2007 the database handles around 2-2.5k transactions per second, which generate roughly 40,000 input/output operations per second on the disks. That's not much...
- jacques_chester 14y agoThey're complex transactions. Inserts against dozens of tables, selects from dozens of tables. We're not talking about glorified KV lookups here.