4 ms·

I loved this article, despite the negativity. A discussion that keeps recurring on HN is how you can get a lot done with simple, boring solutions. This is a pe

I loved this article, despite the negativity. A discussion that keeps recurring on HN is how you can get a lot done with simple, boring solutions. This is a perfect example of that in practice. What’s the simplest thing that’ll work? Do that, monitor the solution, and when you start hitting a limitation reevaluate what the next appropriate solution is. There are plenty of comments here deriding this as insanity when we have established best practices, but “best practices for me are not necessarily best practices thee”. People aren’t perfect, not every engineer comes fully-formed grokking the industry’s best solutions and their trade offs. This is what learning looks like. This is what an evolving system looks like. And this is how valuable software gets built every day. Who gives a fuck about scalability on a poc? “Let’s just put in a json file”. I Love it. Personally I start at the “use SQLite” step and go from there because SQLite is such amazing software. Thanks to this post I’m excited to try out Lifestream. You might rightly point out that Tailscale isn’t a side-project/poc anymore. But that’s exactly why they’re changing to SQLite and writing this post; this is their team learning to deal with larger problems and changing needs, and they’re sharing that journey with us. As long as they’re not _ignoring_ serious risks let them experiment. But try not to get too upset if they’re making different risk/effort trade-offs than you would. Even if it’s “objectively” the “wrong” thing. My rubric for technical sanity is: - are you taking (and testing) backups? - is your infra/data secure? - are you monitoring what’s happening? - can you recover from catastrophic failure in a acceptable time frame? - are you meeting your legal obligations? Cover those needs and you can be forgiven a lot suboptimal/experimental implementation details.

ramraj07 5y ago
The question is, is this the simple solution? Perhaps I and others have been spoiled by cloud offerings but having set up RDS once and never having to worry much about the db architecture after that for years as we scale has maybe spoiled us.
- rablackburn 5y ago
  I’ll be honest and admit my own bias to cheaper, self-managed tooling. I’ll admit that RDS definitely meets the criteria of simple. From what I understand RDS works great and you’re happy to pay what feels like the tiny cost…right up until it doesn’t anymore. You’re paying Amazon a premium for them to manage the tedious stuff. But sometimes dealing with the tedious is what teaches you how the stack works and makes you jmuch more prepared to solve those more complex problems down the road. The uncomfortable fact is businesses finding themselves in situations where they experience vendor-lock-in just because they’ve let their capability wither away. Clearly David’s point about avoiding MySQL and PostgreSQL due to fears of “vendor lock-in” resonates with me, even if I personally think that’s not quite where the threat is. …for Postgres anyway. I’ve heard enough stories that I never want to touch an oracle product in my career.
ignoramous 5y ago
> People aren’t perfect, not every engineer comes fully-formed grokking the industry’s best solutions and their trade offs. The thing is, David Crawshaw is being extremely honest and humble here in admitting their shortcomings and how they're learning from it... but an entire troupe of Silicon Valley engs seem to have lost sleep over it, for reasons beyond me.
- tptacek 5y ago
  Are these even shortcomings? They made JSON scale for 18 months, which is an achievement. They seamlessly switched to etcd, which is one of the "right things" for distributed state problems. Like us, they discovered that when you adopt a "distributed state" solution, you inherit all the API limitations those things come with because nothing really solves (or can solve) the "multiple writers multiple readers real-time consistency" problem. Nothing broke, but etcd was a pain to work with: anything you store in etcd, the developers that work with it have to understand the rituals of etcd. Wouldn't it be nice to have that just be a SQL database? Oh, look: that's easy to do: use sqlite, and ship WAL segments. There is negativity on this thread, but it strikes me as incredibly, even embarrassingly, ill-informed.
mettamage 5y ago
I favorited your post. It has taken out some insecurities that I had about best practices. I couldn't explicitly formulate them either. Thanks for the (professional/career) advice/therapy :)
deleted 5y ago
[deleted]

ramraj07 5y ago

The question is, is this the simple solution? Perhaps I and others have been spoiled by cloud offerings but having set up RDS once and never having to worry much about the db architecture after that for years as we scale has maybe spoiled us.

rablackburn 5y ago
I’ll be honest and admit my own bias to cheaper, self-managed tooling. I’ll admit that RDS definitely meets the criteria of simple. From what I understand RDS works great and you’re happy to pay what feels like the tiny cost…right up until it doesn’t anymore. You’re paying Amazon a premium for them to manage the tedious stuff. But sometimes dealing with the tedious is what teaches you how the stack works and makes you jmuch more prepared to solve those more complex problems down the road. The uncomfortable fact is businesses finding themselves in situations where they experience vendor-lock-in just because they’ve let their capability wither away. Clearly David’s point about avoiding MySQL and PostgreSQL due to fears of “vendor lock-in” resonates with me, even if I personally think that’s not quite where the threat is. …for Postgres anyway. I’ve heard enough stories that I never want to touch an oracle product in my career.

ignoramous 5y ago

> People aren’t perfect, not every engineer comes fully-formed grokking the industry’s best solutions and their trade offs. The thing is, David Crawshaw is being extremely honest and humble here in admitting their shortcomings and how they're learning from it... but an entire troupe of Silicon Valley engs seem to have lost sleep over it, for reasons beyond me.

tptacek 5y ago
Are these even shortcomings? They made JSON scale for 18 months, which is an achievement. They seamlessly switched to etcd, which is one of the "right things" for distributed state problems. Like us, they discovered that when you adopt a "distributed state" solution, you inherit all the API limitations those things come with because nothing really solves (or can solve) the "multiple writers multiple readers real-time consistency" problem. Nothing broke, but etcd was a pain to work with: anything you store in etcd, the developers that work with it have to understand the rituals of etcd. Wouldn't it be nice to have that just be a SQL database? Oh, look: that's easy to do: use sqlite, and ship WAL segments. There is negativity on this thread, but it strikes me as incredibly, even embarrassingly, ill-informed.

mettamage 5y ago

I favorited your post. It has taken out some insecurities that I had about best practices. I couldn't explicitly formulate them either. Thanks for the (professional/career) advice/therapy :)

deleted 5y ago

[deleted]