8 ms·

I disagree. Even on hacker news, people rarely express such absurd things with so little confidence. You fail to take into account many of the following: * Ext

I disagree. Even on hacker news, people rarely express such absurd things with so little confidence. You fail to take into account many of the following: * Extremely high volume. Uber has indicated elsewhere that they receive upwards of a few hundred thousand requests per second on just one service. Please show me the logistics stack that did this in the 70s. * Yes, building the first version of something is extremely cheap and easy. But being able to improve it becomes harder and harder. Especially given high volume, modern companies need sophisticated analytical tools that provides reliable data to both technical and non technical staff. Please show me the analytics stack that was able to ingest, store, and analyze terabytes of business data in realtime from the 70s. * Reliability. Modern web applications need to fail gracefully and be debugged quickly. Please show me the logistics and routing stack that was capable of extremely high uptime while being deployed constantly and serving hundreds of thousands of requests per second from the 70s. * Extensibility. Businesses need to extend to new markets. Moves like this often invalidate past assumptions. In order to support business flexibility, modern engineers deliberately invests considerable time into building decoupled components that can be reused as platforms instead of stuffed into a monolithic codebase. Please show me the operations and routing stack that could easily be reconfigured to enable such products as Amazon Web Services, Uber's external API, the google maps API, or Uber EATS—from the 70s. To make this more concrete, I worked on a routing stack at another company which probably works similarly to Uber's ETA systems. When considering these things, it's important to keep in mind the dependency tree of each new problem set and the work required to make those dependencies work reliably at big scale. To give you an idea of what this area alone entails: 1. Machine learning. - wiring together of and improving algorithms: linear regression to begin with, then random forest, then neural networks. - ensuring data required for learning is reliably available and correctly computed. - tools to launch, deploy, test these models. 2. Working with map data in memory many times larger than what fits onto the smallest consumer laptop. - how do you handle updates of data? - what if you want to use different data sets in different places, because they're more accurate? - how do you debug errors in the data without visual tools (hint: it's really hard and time consuming)? - how do you optimize loading this data into memory without requiring hours to deploy your application? - where do you even store this data? 3. Requests per second in the hundreds of thousands and latency requirements (in order to ensure the app responds quickly) hovering around 10ms. - how do you profile complex distributed applications? - what optimizations are available to make graph search faster (hint: A* isn't fast enough)? - how hard is it to implement these optimizations? 4. Data science and data science tools - Visualizations! - again, reliable data pipelines That's about what one team works on over the course of a year. Note the dependencies we have here: 1. We assume access to cloud infrastructure that doesn't require us to do all of our own devops. 2. We assume mature and automatically scaling data infrastructure: that kafka and storm have been set up and tuned to a degree that we don't have to worry about it. In reality, kafka alone requires a team of at least a dozen at linkedin to keep up with the maintenance, operations, and optimization burden of keeping up with scale. 3. We assume mature and scalable service oriented architecture tooling—if a call to another service is slow, I should be able to see on a dashboard what service is slow, how frequently, it's slow, why it's slow (if it depends on another service) etc. and countless other things I could spend days enumerating for you but I guess it'd be wasted on you because you're pretty convinced you already solved these problems in the 70s, so why am i wasting my breath

dang 10y ago
It's important to counter the trivializing sort of dismissal that people often post to HN (the old "I could build Twitter in a weekend" and whatnot). We want the culture to move more toward thoughtful, substantive critique. So your detailed argument here, based on experience, is valuable. Please don't spoil it by becoming uncivil like this: > I guess it'd be wasted on you because you're pretty convinced you already solved these problems in the 70s, so why am i wasting my breath With that your comment does more harm than good: it poisons the atmosphere and detracts from your substantive contribution. You're definitely not "wasting your breath" even if you fail to persuade the other person not to be snarkily dismissive, because the real audience for a comment like yours is everybody else: i.e. the rest of us who are curious about how (in this case) Uber operates and why things might be the way they are. That audience needs to see both good information about the challenges involved (as opposed to this-has-been-trivial-since-the-70s) and a good example of how to patiently respond to a trivializing comment with a thoughtful one. It's bad if, instead, you give us a reason to wince and an example of replying to a dismissive comment with a rude one.
- 10921809211 10y ago
  This is an non-exhaustive list of technologies that the article mentions: "Terraform, Schemaless, Riak, Cassandra, Hadoop, Redis, Twemproxy, Celery, Kafka, Elasticsearch, Logstash, Kibana, Docker, Mesos, Aurora, HAProxy, Hyperbahn, Ringpop, TChannel, Nginx, Thrift, Protobuf, Phabricator, OpenGrok, Packer, Vagrant, Boto, Unison, Jenkins, Clusto, Puppet, Grafana, Storm, Spark, React, SVG, Canvas 2D, Gufaru, DropWizard" How are you going to discuss all that in a substantive manner? All anyone can do in the limited time frame of a HN discussion is draw parallels to previous job experiences or previous user experiences. My own experiences are more aligned with the sentiment expressed in the "dismissal" comment.
  - dang 10y ago
    Mine too, but that doesn't make it a good comment. In fact its first paragraph is almost a parody of the know-it-all internet comment. There are a zillion ways to make the same kind of argument thoughtfully. Talking about one's own concrete experiences helps. So does not acting like you know everything about somebody else's situation. Bloat is a problem, and so (in my view) is the kitchen-sink software culture of hauling in libraries and frameworks without thought for overall complexity. But we need to be able to talk about this at a higher level than other-people-are-idiots-compared-to-me. A much higher level.

dang 10y ago

It's important to counter the trivializing sort of dismissal that people often post to HN (the old "I could build Twitter in a weekend" and whatnot). We want the culture to move more toward thoughtful, substantive critique. So your detailed argument here, based on experience, is valuable. Please don't spoil it by becoming uncivil like this: > I guess it'd be wasted on you because you're pretty convinced you already solved these problems in the 70s, so why am i wasting my breath With that your comment does more harm than good: it poisons the atmosphere and detracts from your substantive contribution. You're definitely not "wasting your breath" even if you fail to persuade the other person not to be snarkily dismissive, because the real audience for a comment like yours is everybody else: i.e. the rest of us who are curious about how (in this case) Uber operates and why things might be the way they are. That audience needs to see both good information about the challenges involved (as opposed to this-has-been-trivial-since-the-70s) and a good example of how to patiently respond to a trivializing comment with a thoughtful one. It's bad if, instead, you give us a reason to wince and an example of replying to a dismissive comment with a rude one.

10921809211 10y ago
This is an non-exhaustive list of technologies that the article mentions: "Terraform, Schemaless, Riak, Cassandra, Hadoop, Redis, Twemproxy, Celery, Kafka, Elasticsearch, Logstash, Kibana, Docker, Mesos, Aurora, HAProxy, Hyperbahn, Ringpop, TChannel, Nginx, Thrift, Protobuf, Phabricator, OpenGrok, Packer, Vagrant, Boto, Unison, Jenkins, Clusto, Puppet, Grafana, Storm, Spark, React, SVG, Canvas 2D, Gufaru, DropWizard" How are you going to discuss all that in a substantive manner? All anyone can do in the limited time frame of a HN discussion is draw parallels to previous job experiences or previous user experiences. My own experiences are more aligned with the sentiment expressed in the "dismissal" comment.
- dang 10y ago
  Mine too, but that doesn't make it a good comment. In fact its first paragraph is almost a parody of the know-it-all internet comment. There are a zillion ways to make the same kind of argument thoughtfully. Talking about one's own concrete experiences helps. So does not acting like you know everything about somebody else's situation. Bloat is a problem, and so (in my view) is the kitchen-sink software culture of hauling in libraries and frameworks without thought for overall complexity. But we need to be able to talk about this at a higher level than other-people-are-idiots-compared-to-me. A much higher level.