3 ms·

To me software seems like an arms race because while it’s expensive to develop, the marginal cost to run it is very low. As long as companies continue to find w

To me software seems like an arms race because while it’s expensive to develop, the marginal cost to run it is very low. As long as companies continue to find way to increase their competitiveness through software, there will always be more work for software engineers as more and more of them are consumed by maintenance of existing software. That’s why the 2000s outsourcing wave didn’t really impact the market for software engineers in the US despite the apocalyptic rhetoric at the time. If Company A has a productive local dev team, their competitor B has to either hire their own or hire a bigger outsourced team to stay competitive. Then company A has to hire even more to meet exec’s growth targets and it becomes a competitive spiral unless economic conditions change (like recently with elimination of ZIRP). I think in the near future, LLMs will put that arms race on overdrive and all of us will have to adapt. They’re here to stay, for better or worse. I handle it by using AI tools for understanding codebases and code reviews. Properly set up, LLMs can even give decent refactoring advice - i.e. by feeding it a callgraph - but the UX isn’t there yet. I use cli apps and custom scripts to do a lot of codebase work with LLMs like claude-sync and aider. I’d start by trying out aider/Cursor to see whether they can help you manage the code. Use aider’s repomap or Cursor’s repo indexing features and ask the LLM questions about the codebase. Use the Cursor code review feature, which has caught a lot of subtle bugs for me (I ignored the feature for my first month and was pleasantly surprised when I first used it in lieu of debugging a PR). Experiment with asking the LLM to refactor the codebase since now their token windows are large enough to fit ten thousand plus lines. Maybe try Github Copilot Chat if you can get beta access, since they do codebase indexing too. My advice boils down to “fight fire with fire”. In the hands of competent engineers, LLMs are a lot more powerful than the slop you’re seeing. (For context I’m working at the intersection of C++/Rust/Qt for which training data is basically nonexistent)

jcgrillo 2y ago
> marginal cost to run it is very low I think this isn't particularly true, in general. I'll give an example of how this assumption can bite an early stage startup rather badly. Company X rapidly develops some idea, raises seed round, hires some devs. At this point they have no customers or traffic so infrastructure costs are low and software development costs dominate. Moreover, they have $100k of free Amazon bucks to play with so they're not really thinking about infrastructure costs. Then they get a customer or two, and all of a sudden infrastructure costs are huge and the small dev team has to reorient in a panic towards figuring out how to make their thing efficient--basically an unplanned total rewrite. The opportunity cost of that could actually sink the company or force them to raise a round on bad terms. In the public cloud, there are many ways you can very quickly rack up a huge bill if you don't do things efficiently. Some examples: 1. Let's just use AWS Lambda for everything! 2. Let's use AWS managed kuberneetus! 3. Let's write a bunch of small files to S3! 4. Let's not think too hard about what our data looks like on the wire, and just ignore data transfer fees! These costs may look small at first, but they probably scale more rapidly than your revenue does. There's absolutely no substitute for just doing it consistently right, from the very beginning, and not deviating.
- throwup238 2y ago
  The marginal cost of each of those is very low, especially stuff like S3 and Lambda. You used to have to rent enough dedicated servers or VMs to handle your traffic and before that buy racks of storage and servers to run software. Now it's literally paid by the minute/gigabyte as you use it instead of having to invest in it upfront. Just because companies are bad at cost management and bloat their stacks in clouds doesn't mean that the marginal cost is high, just that their absolute budget is. The marginal cost of compute and the marginal cost of each new customer using it are different issues.
  - jcgrillo 2y ago
    Yes, but the marginal cost isn't actually that low that you can use these things willy-nilly without actually calculating and thinking about it. So the idea that "dev time is expensive, computers are cheap" just isn't true. So a system which automatically generates a bunch of inefficient, poorly wrought crap quickly for low effort will make you pay for it in different ways. So it's not obviously advantageous, and it seems (to me at least) the disadvantages are greater.
    - throwup238 2y ago
      I never said that you don't have to think about it nor did I say that "dev time is expensive, computers are cheap." What I said was that the "while [software is] expensive to develop, marginal cost to run it is very low." Those are completely different statements, although I could have been more specific. Low marginal cost is built into the cloud products like Lambda, S3, and autoscaling VMs. That's why they're popular, even though the servers running those services are much more expensive. Edit: I think I see the impedance mismatch here. Yes the marginal cost of spinning up another RDB instance or big VM is relatively high, but unless you’re spinning up one for each customer, the marginal cost per user is much lower. Sometimes the marginal cost is recalculated like when scaling up to more big DB instances, but the cost per unit is very low compared to say a factory that has to buy machines up front and train machinists when they need to grow. If a software company has enough servers already, the marginal cost of a new customer is often just the extra electricity used by existing compute

jcgrillo 2y ago

> marginal cost to run it is very low I think this isn't particularly true, in general. I'll give an example of how this assumption can bite an early stage startup rather badly. Company X rapidly develops some idea, raises seed round, hires some devs. At this point they have no customers or traffic so infrastructure costs are low and software development costs dominate. Moreover, they have $100k of free Amazon bucks to play with so they're not really thinking about infrastructure costs. Then they get a customer or two, and all of a sudden infrastructure costs are huge and the small dev team has to reorient in a panic towards figuring out how to make their thing efficient--basically an unplanned total rewrite. The opportunity cost of that could actually sink the company or force them to raise a round on bad terms. In the public cloud, there are many ways you can very quickly rack up a huge bill if you don't do things efficiently. Some examples: 1. Let's just use AWS Lambda for everything! 2. Let's use AWS managed kuberneetus! 3. Let's write a bunch of small files to S3! 4. Let's not think too hard about what our data looks like on the wire, and just ignore data transfer fees! These costs may look small at first, but they probably scale more rapidly than your revenue does. There's absolutely no substitute for just doing it consistently right, from the very beginning, and not deviating.

throwup238 2y ago
The marginal cost of each of those is very low, especially stuff like S3 and Lambda. You used to have to rent enough dedicated servers or VMs to handle your traffic and before that buy racks of storage and servers to run software. Now it's literally paid by the minute/gigabyte as you use it instead of having to invest in it upfront. Just because companies are bad at cost management and bloat their stacks in clouds doesn't mean that the marginal cost is high, just that their absolute budget is. The marginal cost of compute and the marginal cost of each new customer using it are different issues.
- jcgrillo 2y ago
  Yes, but the marginal cost isn't actually that low that you can use these things willy-nilly without actually calculating and thinking about it. So the idea that "dev time is expensive, computers are cheap" just isn't true. So a system which automatically generates a bunch of inefficient, poorly wrought crap quickly for low effort will make you pay for it in different ways. So it's not obviously advantageous, and it seems (to me at least) the disadvantages are greater.
  - throwup238 2y ago
    I never said that you don't have to think about it nor did I say that "dev time is expensive, computers are cheap." What I said was that the "while [software is] expensive to develop, marginal cost to run it is very low." Those are completely different statements, although I could have been more specific. Low marginal cost is built into the cloud products like Lambda, S3, and autoscaling VMs. That's why they're popular, even though the servers running those services are much more expensive. Edit: I think I see the impedance mismatch here. Yes the marginal cost of spinning up another RDB instance or big VM is relatively high, but unless you’re spinning up one for each customer, the marginal cost per user is much lower. Sometimes the marginal cost is recalculated like when scaling up to more big DB instances, but the cost per unit is very low compared to say a factory that has to buy machines up front and train machinists when they need to grow. If a software company has enough servers already, the marginal cost of a new customer is often just the extra electricity used by existing compute