3 ms·
Get a job at a big tech company (Google, Facebook, Microsoft, Yahoo, DropBox, Uber, etc.). Beyond a certain scale, every problem is distributed, and you'll be
by nostrademons 11y ago
Get a job at a big tech company (Google, Facebook, Microsoft, Yahoo, DropBox, Uber, etc.). Beyond a certain scale, every problem is distributed, and you'll be dealing with distributed systems and their pitfalls as a routine part of your job.
It's very difficult to tell what is of practical significance vs. what is just theoretically interesting without directly working with a real distributed system storing real data for real users. For example, master election is something you rarely face in the day-to-day: you build a system once (or just use Zookeeper) and then re-use it for all your systems. Merkel trees and vector clocks are fairly special-purpose - once in a while you'll find a system where they're critical, but a lot of the time you don't need them. Versioning problems and choices of ID schemas, however, come up all the time. Bloom filters are far more useful than the amount of theoretical or blogosphere attention given to them would indicate, sharding functions are important to know, and concurrency & buffering are critical skills.
- fizwhiz 11y agoMy personal experience at at large tech company is that there are usually designated teams thinking about these infrastructural problems so that the rest of the engineers don't have to worry about them. So as someone managing a "service" in a company that has a SOA model, sure, I'm technically a part of a distributed system but the deliberate abstractions minimize the "necessary" exposure to dealing with distributed systems. I'm in complete agreement with your thoughts on stuff with practical significance vs theoretically interesting topics (I'm far more interested in building some expertise around practically significant things first.) Are there any particular projects you'd recommend I follow to grow my knowledge in this space? Alternatively, would you have any recommendations on specific side-projects that could really motivate a good understanding of these concepts?