3 ms·

> The likely scenario is not that you haven't fully understood their constraints or requirements, it's that you're just smarter than they are. I never claimed

by hsn915 5y ago

> The likely scenario is not that you haven't fully understood their constraints or requirements, it's that you're just smarter than they are. I never claimed to be smarter. I just understand some things that I noticed a lot of people in the industry don't understand. My understanding is not even that great. But still, this is just one example that I keep running into over and over and over: People opting for a complicated infrastructure setup because that's what they think you should do. No one showed them how to make a stable reliable website that just runs on a single machine and handle thousands of concurrent connections. It's not hard. It's just that they've never seen it and assume it's basically impossible. There are areas about computing that I feel the same way about. For example, before Casey Muratori demoed his refterm implementation, I had no idea that it was possible to render a terminal at thousands of frames per second. I just assumed such a feat was technically impossible. Partly because no one has done it. But then he did it, and I was blown away. > Take that order of magnitude cheaper, single VPS server solution you're proposing and build something with it. Sounds like you'd make a lot of money. Building something and making money out of it are not the same thing. But thanks for the advice. I'm in the process of trying. I know for sure I can build the thing, but I don't know if it will make any money. We will see. > People who don't do / build explaining to the people who do how easy, simple, better their solutions would be. I do and have done. This kind of advice is exactly the kind of thing I know how to do because I have done it in the past using my trivial setup of a single process running on a cheap VPS. And I have also seen other teams struggle to get some feature nearly half-working on a complicated infrastructure setup with AWS and all the other buzzwords: Kibana, Elastic Search, Dynamo DB, Ansible, Terraform, Kubernetes ... what else? I can't even keep track of all this stuff that everyone keeps talking about even though hardly anyone needs at all. I've seen 4 or 5 companies try to build their service using this kind of setup, with the proposed advantange of "horizontal" and "auto" scaling. And you know what? They ALL struggled with poor performance, ALL THE TIME. It's really sad.

ctvo 5y ago
> No one showed them how to make a stable reliable website that just runs on a single machine and handle thousands of concurrent connections. What is a reliable website? What does this website do? If given a static constraint, like serve 2000 requests per second with 99.999% uptime, and enough time, I'm sure you can optimize it to be as efficient as you'd like. But that's not our exercise. Bespoke, custom solutions that are not the core of the business are not solutions. Repeat that a dozen times. MangaDex's business is not to be the most efficient website possible. Their business, I assume, is content and features for their users. They pick off the shelf technologies to do it because it's well documented, proven, and most importantly already built. They compose these technologies to solve their business problems. Often there's a mismatch or an overlap in functionality that introduces inefficiency (complexity, cost, performance), but that's a trade-off MangaDex and many other businesses make. We can judge how poorly or well they've made some trade-offs base on their business and overall requirements. You coming here and telling people that you can run it on a VPS ignores all of the above. And Casey has a YouTube channel where he makes a game his way I would assume because working to solve uninteresting business problems (and possibly dealing with co-workers who may pull down N project instead of building it themselves) wasn't a space he was interested in. There's a difference between a purely technical challenge, and working with complex, interacting systems like ... people and business requirements and laws and regulations and auditing and hiring and security and who the hell maintains this system when Bob quits. Conflating the two is the root of the comments like yours, I'd think. > It's not hard. It's just that they've never seen it and assume it's basically impossible. I'm sorry, but who thinks only serving 2000 requests per second is hard? Or do you assume they think it's hard because you misunderstood or are unaware of their 100s of other requirements that they need to solve for in addition to serving 2000 requests per second? I'm going on and on in this thread mainly because I'm tired of people assuming they know better than the people in the trenches making these decisions. You're assuming you're more knowledgable and skilled than them in their own problem space! They're obviously unaware that using Go to make a server would solve their problems otherwise why wouldn't they have done it? FYI I work (and have worked) for large tech companies (think silly acronym). I'm not even in this space, as in the type of problems I face are quite different, but I can respect the authors of that article enough to nod along, shrug, and not assume I know better.
- hsn915 5y ago
  > What is a reliable website? What does this website do? It performs well and doesn't randomly go down when someone posts a link to it on HN or tries to put in bad input. > I'm sure you can optimize it to be as efficient as you'd like. Wrong! I've said nothing about optimizing things. All I'm advocating is simple solutions that are proven to work. A web server in Go is far from efficient. An optimized server in C/C++ can probably perform 20x better than a Go server. If not more. However, a web server in Go makes far more reasonable use of system resources to achieve the desired goals. It's also pretty reliable. > Bespoke, custom solutions that are not the core of the business are not solutions. Repeat that a dozen times. I don't understand the point of this sentence. Are you saying that Kubernetes or Elastic Search or AWS or any of the other buzzwords are at the core of their business? Clearly they are not. > MangaDex's business is not to be the most efficient website possible. Their business, I assume, is content and features for their users. They pick off the shelf technologies to do it because it's well documented, proven, and most importantly already built. It's in the interest of their business to lower their cost of operations. Building on a complicated infrastructure when you don't need is incurring a lot of cost. Not just the monthly cost ($1500/mo) but the cost of the staff needed to understand and maintain this infrastructure. It's not the kind of thing that is easy to maintain. To be completely frank with you, I myself am not capable of understanding or maintaining such a system. And every company I've been almost had no one who understood how the system really works. Someone set things up sometime by following some tutorials. When things go wrong, people panic and go into fire fighting mode. They spend hours trying to make sense of what's going on, usually involving multiple people - because it's not a task that a single individual can handle. > They compose these technologies to solve their business problems. Often there's a mismatch or an overlap in functionality that introduces inefficiency (complexity, cost, performance), but that's a trade-off MangaDex and many other businesses make. We can judge how poorly or well they've made some trade-offs base on their business and overall requirements. You are talking as if these off the shelf technologies are reliable and easy to implement or integrate. From what I've seen, these solutions are a lot more complicated than what I'm proposing. Every place I've been to that tries to take this approach ends up burning too much money and resources trying to make their thing work. It's not as if these companies don't have to write code to make their product work. You still have to write code anyway. So, why not, instead of writing tons of glue code and configuration files to hopelessly integrate a hodge podge of tools and frameworks ... why not just write the simple code that just does the thing you want? > I'm going on and on in this thread mainly because I'm tired of people assuming they know better than the people in the trenches making these decisions. You're assuming you're more knowledgable and skilled than them in their own problem space! They're obviously unaware that using Go to make a server would solve their problems otherwise why wouldn't they have done it? The first company I've been to that was doing this kind of thing was spending upwards of $10k/mo on the most beefed up server that AWS provides to host the database server, and they still struggled to server more than 1000 users concurrently. According to you, I'm not in a position to give them suggestions or adivce about how to fix this problem!! > I'm sorry, but who thinks only serving 2000 requests per second is hard? Or do you assume they think it's hard because you misunderstood or are unaware of their 100s of other requirements that they need to solve for in addition to serving 2000 requests per second? What are the other 100 requirements that are not fulfilled by the thing I'm proposing?!

by hsn915 5y ago

ctvo 5y ago

> No one showed them how to make a stable reliable website that just runs on a single machine and handle thousands of concurrent connections. What is a reliable website? What does this website do? If given a static constraint, like serve 2000 requests per second with 99.999% uptime, and enough time, I'm sure you can optimize it to be as efficient as you'd like. But that's not our exercise. Bespoke, custom solutions that are not the core of the business are not solutions. Repeat that a dozen times. MangaDex's business is not to be the most efficient website possible. Their business, I assume, is content and features for their users. They pick off the shelf technologies to do it because it's well documented, proven, and most importantly already built. They compose these technologies to solve their business problems. Often there's a mismatch or an overlap in functionality that introduces inefficiency (complexity, cost, performance), but that's a trade-off MangaDex and many other businesses make. We can judge how poorly or well they've made some trade-offs base on their business and overall requirements. You coming here and telling people that you can run it on a VPS ignores all of the above. And Casey has a YouTube channel where he makes a game his way I would assume because working to solve uninteresting business problems (and possibly dealing with co-workers who may pull down N project instead of building it themselves) wasn't a space he was interested in. There's a difference between a purely technical challenge, and working with complex, interacting systems like ... people and business requirements and laws and regulations and auditing and hiring and security and who the hell maintains this system when Bob quits. Conflating the two is the root of the comments like yours, I'd think. > It's not hard. It's just that they've never seen it and assume it's basically impossible. I'm sorry, but who thinks only serving 2000 requests per second is hard? Or do you assume they think it's hard because you misunderstood or are unaware of their 100s of other requirements that they need to solve for in addition to serving 2000 requests per second? I'm going on and on in this thread mainly because I'm tired of people assuming they know better than the people in the trenches making these decisions. You're assuming you're more knowledgable and skilled than them in their own problem space! They're obviously unaware that using Go to make a server would solve their problems otherwise why wouldn't they have done it? FYI I work (and have worked) for large tech companies (think silly acronym). I'm not even in this space, as in the type of problems I face are quite different, but I can respect the authors of that article enough to nod along, shrug, and not assume I know better.

hsn915 5y ago
> What is a reliable website? What does this website do? It performs well and doesn't randomly go down when someone posts a link to it on HN or tries to put in bad input. > I'm sure you can optimize it to be as efficient as you'd like. Wrong! I've said nothing about optimizing things. All I'm advocating is simple solutions that are proven to work. A web server in Go is far from efficient. An optimized server in C/C++ can probably perform 20x better than a Go server. If not more. However, a web server in Go makes far more reasonable use of system resources to achieve the desired goals. It's also pretty reliable. > Bespoke, custom solutions that are not the core of the business are not solutions. Repeat that a dozen times. I don't understand the point of this sentence. Are you saying that Kubernetes or Elastic Search or AWS or any of the other buzzwords are at the core of their business? Clearly they are not. > MangaDex's business is not to be the most efficient website possible. Their business, I assume, is content and features for their users. They pick off the shelf technologies to do it because it's well documented, proven, and most importantly already built. It's in the interest of their business to lower their cost of operations. Building on a complicated infrastructure when you don't need is incurring a lot of cost. Not just the monthly cost ($1500/mo) but the cost of the staff needed to understand and maintain this infrastructure. It's not the kind of thing that is easy to maintain. To be completely frank with you, I myself am not capable of understanding or maintaining such a system. And every company I've been almost had no one who understood how the system really works. Someone set things up sometime by following some tutorials. When things go wrong, people panic and go into fire fighting mode. They spend hours trying to make sense of what's going on, usually involving multiple people - because it's not a task that a single individual can handle. > They compose these technologies to solve their business problems. Often there's a mismatch or an overlap in functionality that introduces inefficiency (complexity, cost, performance), but that's a trade-off MangaDex and many other businesses make. We can judge how poorly or well they've made some trade-offs base on their business and overall requirements. You are talking as if these off the shelf technologies are reliable and easy to implement or integrate. From what I've seen, these solutions are a lot more complicated than what I'm proposing. Every place I've been to that tries to take this approach ends up burning too much money and resources trying to make their thing work. It's not as if these companies don't have to write code to make their product work. You still have to write code anyway. So, why not, instead of writing tons of glue code and configuration files to hopelessly integrate a hodge podge of tools and frameworks ... why not just write the simple code that just does the thing you want? > I'm going on and on in this thread mainly because I'm tired of people assuming they know better than the people in the trenches making these decisions. You're assuming you're more knowledgable and skilled than them in their own problem space! They're obviously unaware that using Go to make a server would solve their problems otherwise why wouldn't they have done it? The first company I've been to that was doing this kind of thing was spending upwards of $10k/mo on the most beefed up server that AWS provides to host the database server, and they still struggled to server more than 1000 users concurrently. According to you, I'm not in a position to give them suggestions or adivce about how to fix this problem!! > I'm sorry, but who thinks only serving 2000 requests per second is hard? Or do you assume they think it's hard because you misunderstood or are unaware of their 100s of other requirements that they need to solve for in addition to serving 2000 requests per second? What are the other 100 requirements that are not fulfilled by the thing I'm proposing?!