3 ms·
The way I've been putting it to people lately is: Never underestimate how hard a problem can grow by making it big. And also, at times, it is hard to appreciate
by tetha 4y ago
The way I've been putting it to people lately is: Never underestimate how hard a problem can grow by making it big. And also, at times, it is hard to appreciate how difficult something becomes if you haven't walked the path at least partially.
Like, from work, hosting postgres. At this point, I very much understand why a consultant once said - "You cannot make mistakes in a postgres 10GB or 100GB and a dozen transactions per second in size". And he's right, give it some hardware, don't touch knobs except for 1 or 2 and that's it. The average application accessing our postgres clusters is just too small to cause problems.
And then we have 2 postgres clusters with a dataset size of 1TB or 2TB peaking at like 300 - 400 transactions per second. That's not necessarily big or busy for what postgres can do, but it becomes noticeable that you have to do some things right at this point and some patterns just stop working hard.
And then there are people dealing with postgres instances 100 - 1000x bigger than this. And that's becoming tangibly awesome and frightening by now, using awesome in a more oldschool way there.
- mlrtime 4y agoNot only make it big, engineer it in a way that makes it profitable for the business. I'm sure there are many teams that could design such a network with nearly unlimited resources, but it is entirely different when you have profit margins.
- victor106 4y agoAs someone once said “Big is different”
- jiggawatts 4y agoI know how to operate a hose! How hard can it be to manage a large stream? It's "just" more water: https://www.youtube.com/watch?v=jxNM4DGBRMU https://www.youtube.com/watch?v=jxNM4DGBRMU