4 ms·

Yup, but cost is relative to the problem you're trying to solve. When your app needs horizontal scaling DB with multiple regions, and you don't have to come up

Yup, but cost is relative to the problem you're trying to solve. When your app needs horizontal scaling DB with multiple regions, and you don't have to come up with your own sharding solution, suddenly $8000 a year sounds like a great price.

erikpukinskis 9y ago
General computer science-y kind of software philosophy question: Should we really be thinking of sharding as something you can outsource to the infrastructure layer? It just seems like having a stance about how the data in your specific domain naturally shards is probably going to pay off not even the long run, but immediately in terms of sanity checking your information architecture. And I wonder if in 2017 we haven't gotten to the point where a cloud application should just shard, because we're trying to think about software as something that runs on a transient instance with access to a subset of data, appearing and reappearing, not a giant box with everything on it. I get that Google engineers are darn close to abstracting away that "giant box with everything on it" behind an API, but I guess I'm asking if that's really how we should be thinking about our code.
- degenerate 9y ago
  I agree with your line of thinking. But at really big companies with hundreds of small teams, all working on separate projects, when the boss says to hook all that stuff together I'm sure a solution like Spanner looks really delicious and cost-effective. For you and me working with data we already understand and added to our project piece by piece, we can safely shard and compartmentalize.
- scott00 9y ago
  As with basically all software design questions, the answer is "it depends". Writing a LOB app with 4 expected users all located in the same location, and expected storage requirements of a gigabyte a year? Go with a traditional DB, add replication if you need high availability, make and test your backups, done. Writing the new Facebook weknoweverything app that captures smartphone audio and video continuously for all users at all times, automatically transcribes all conversations and produces AI-driven summaries of all video, all of which are saved forever and highly searchable? You're going to need a highly customized data storage architecture to have a prayer of keeping up with that data. Somewhere in between? Some of those are good candidates for Spanner. Some aren't.

erikpukinskis 9y ago

General computer science-y kind of software philosophy question: Should we really be thinking of sharding as something you can outsource to the infrastructure layer? It just seems like having a stance about how the data in your specific domain naturally shards is probably going to pay off not even the long run, but immediately in terms of sanity checking your information architecture. And I wonder if in 2017 we haven't gotten to the point where a cloud application should just shard, because we're trying to think about software as something that runs on a transient instance with access to a subset of data, appearing and reappearing, not a giant box with everything on it. I get that Google engineers are darn close to abstracting away that "giant box with everything on it" behind an API, but I guess I'm asking if that's really how we should be thinking about our code.

degenerate 9y ago
I agree with your line of thinking. But at really big companies with hundreds of small teams, all working on separate projects, when the boss says to hook all that stuff together I'm sure a solution like Spanner looks really delicious and cost-effective. For you and me working with data we already understand and added to our project piece by piece, we can safely shard and compartmentalize.
scott00 9y ago
As with basically all software design questions, the answer is "it depends". Writing a LOB app with 4 expected users all located in the same location, and expected storage requirements of a gigabyte a year? Go with a traditional DB, add replication if you need high availability, make and test your backups, done. Writing the new Facebook weknoweverything app that captures smartphone audio and video continuously for all users at all times, automatically transcribes all conversations and produces AI-driven summaries of all video, all of which are saved forever and highly searchable? You're going to need a highly customized data storage architecture to have a prayer of keeping up with that data. Somewhere in between? Some of those are good candidates for Spanner. Some aren't.