3 ms·

On this topic, where do you guys learn more about architecture of large conglomerates of web services and the latest trends? I read High Scalability and the oc

On this topic, where do you guys learn more about architecture of large conglomerates of web services and the latest trends? I read High Scalability and the occasional company blog. Is there anything else out there that might be even better? Blogs, books, forums, doesn't matter.

louyang 12y ago
Get a job at one of them, cant quite beat that.
- akurilin 12y ago
  I did already. There are multiple issues with that approach: you get only flavor of how to do things, unless you keep jumping teams over and over again for years. It's extremely inefficient way of learning this. You're also putting the burden on the folks in charge to know what's out there and pass it down to you, instead of actually doing your own research. Also, sometimes those people don't know what they're doing and they're struggling just as much as you would be in their position, so now you're learning from people who are making it up as they go, which you could have done equally as well by yourself.
dantiberian 12y ago
I've learned heaps from reading the papers that these companies publish. Google, Facebook, and Amazon all publish well written, readable, papers. If you're looking for somewhere to start, then the Dynamo paper is a classic: http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...
- gansai 12y ago
  just curious: "learned heaps" --> did it just improve your knowledgebase or it actually helped you practically to solve a problem which you were facing.? I am just curious, as I am a starter to reading these kind of papers. thanks.
  - dantiberian 12y ago
    A bit of both, some like the Dynamo paper helped me understand Cassandra better, while other like Spanner aren't directly applicable to my day to day work but introduced me to new ideas.
rawnlq 12y ago
This is a talk about the evolution of Dropbox's architecture from 2012: http://www.youtube.com/watch?v=PE4gwstWhmc http://www.youtube.com/watch?v=PE4gwstWhmc. It is incredibly detailed (down to the exact sql schema they use for file metadata etc).
RivieraKid 12y ago
+1 and I'm particularly interested in Amazon's architecture, I couldn't find much info about it (besides High Scalability). Can anyone from Amazon answer questions?

louyang 12y ago

Get a job at one of them, cant quite beat that.

akurilin 12y ago
I did already. There are multiple issues with that approach: you get only flavor of how to do things, unless you keep jumping teams over and over again for years. It's extremely inefficient way of learning this. You're also putting the burden on the folks in charge to know what's out there and pass it down to you, instead of actually doing your own research. Also, sometimes those people don't know what they're doing and they're struggling just as much as you would be in their position, so now you're learning from people who are making it up as they go, which you could have done equally as well by yourself.

dantiberian 12y ago

I've learned heaps from reading the papers that these companies publish. Google, Facebook, and Amazon all publish well written, readable, papers. If you're looking for somewhere to start, then the Dynamo paper is a classic: http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...

gansai 12y ago
just curious: "learned heaps" --> did it just improve your knowledgebase or it actually helped you practically to solve a problem which you were facing.? I am just curious, as I am a starter to reading these kind of papers. thanks.
- dantiberian 12y ago
  A bit of both, some like the Dynamo paper helped me understand Cassandra better, while other like Spanner aren't directly applicable to my day to day work but introduced me to new ideas.

rawnlq 12y ago

This is a talk about the evolution of Dropbox's architecture from 2012: http://www.youtube.com/watch?v=PE4gwstWhmc http://www.youtube.com/watch?v=PE4gwstWhmc. It is incredibly detailed (down to the exact sql schema they use for file metadata etc).

RivieraKid 12y ago

+1 and I'm particularly interested in Amazon's architecture, I couldn't find much info about it (besides High Scalability). Can anyone from Amazon answer questions?