5 ms·

This is often explained as “cycles are cheap, programmers are expensive”. For the sake of argument, suppose LLMs are able to replace programmers for “laborious

This is often explained as “cycles are cheap, programmers are expensive”. For the sake of argument, suppose LLMs are able to replace programmers for “laborious but straightforward work” like writing native versions of your app for every platform, and optimizing the hell out of it. Do we then expect all the apps we work with to become snappy again? Or are there red-queen-like dynamics that will prevent it?

thrtythreeforty 2y ago
I doubt it. The core logic remains unchanged: why have 1 feature that works well when you can have 10 features that don't work?
- jessriedel 2y ago
  Because features require more human overhead to understand and manage than porting/optimizing does, by assumption of this hypothetical. Product managers and high-level architects aren’t free.
- preyneyv 2y ago
  This is exactly the issue. We can do things faster => we can do more things worse!
hansonkd 2y ago
Snappy software requires the design of the system to be in sync with the engineering. Also the team needs to be aligned on snappiness being a priority. Product designers not understanding how adding certain UI elements, animations or requesting certain features is the main cause from my experience. The fastest way to speed things up in certain cases is to just remove some elements. I would expect the situation to get worse rather than better if LLMs can deliver any feature a product designer wants without limitation and if the product designer doesn't care about "snappiness"
- jessriedel 2y ago
  The assumption of the hypothetical is not that they can deliver any feature, which I consider (maybe you disagree) more open-ended and less automatable than porting and optimization. If you think that’s an unrealistic assumption, please feel free to propose another!
  - hansonkd 2y ago
    I'm saying that it doesn't matter how much porting and optimization the llms do if the person designing the product doesn't care about efficiency. Currently designers have pretty limitless ability to add features even without LLMs. LLMs will just compound that. The only way to get efficient code is to have the right mindset from the get go. Not as an after thought that you hope the LLM will optimize.
    - jessriedel 2y ago
      Ok. Personally, I dispute that the designers don’t care about efficiency. I think it’s just expensive.
Retr0id 2y ago
I think "software is a gas" is the better explanation. ("Software always expands to fit whatever container it is stored in") Microsoft has enough engineer-hours on hand to make snappy UIs if they wanted to, but it's just not a priority for them. More/cheaper engineer hours doesn't really change that prioritization.
- jessriedel 2y ago
  Microsoft would hire a ton more engineers and look very different if engineer price went to ~zero. So what does that look like? If you can make an app snappy with $100 in GPU time, why wouldn’t they?
  - Retr0id 2y ago
    Because they'd rather spend that $100 in GPU time on adding a new feature. Repeat that a few times and now your app is 10x more complex and costs 1000x more to perf-optimise.
    - jessriedel 2y ago
      The hypothetical is only interesting if we draw a distinction between easy and hard tasks. (“AI can do anything a human can do behind a keyboard” is AGI complete.) For this hypothetical I asserted that performance optimization and porting were automatable, but that doesn’t include any feature. If you think there’s a more interesting/realistic/well-defined assumption, please suggest it!
qwertox 2y ago
There is a future where this is within the realm of the possible. All it takes is one good AI capable of doing this which can then be duplicated many times to work on this task. An AI engineer which knows how languages compile down to machine code and understands the intention of the blueprint, which the slow app would be. 5 to 10 years, maybe? I mean, we currently think in the context of programming languages as written text, but AI has shown that it's capable of doing speech-to-speech "tasks" (for lack of a better word), where tokens are not only present in the form of text tokens, but also audio tokens, which skips the TTS step. It might become able to do video-to-assembly tasks, where it sees what an electron app does, maybe also reads the code, and then knows what needs to be done to convert this into optimized assembly code.
alpaca128 2y ago
> cycles are cheap ...when it's other people's cycles & time. If Google can squeeze the last 0.1% out of their servers, they will. But there's no incentive to make the experience better for users unless it both has a measurable impact and there's a viable competitor.
NoboruWataya 2y ago
That's one possible outcome, lets call it outcome A. The others that occur to me are that (B) software stays the same but gets cheaper, or (C) software stays the same and is still expensive but profits increase. I imagine companies would much prefer outcome C if they can get away with it, and will do everything in their power to achieve it. But they are subject to market pressures so may be forced into one of the other outcomes. What outcome do you think the market will force? Personally I think we will end up with a mix of outcomes B and C.

thrtythreeforty 2y ago

I doubt it. The core logic remains unchanged: why have 1 feature that works well when you can have 10 features that don't work?

jessriedel 2y ago
Because features require more human overhead to understand and manage than porting/optimizing does, by assumption of this hypothetical. Product managers and high-level architects aren’t free.
preyneyv 2y ago
This is exactly the issue. We can do things faster => we can do more things worse!

hansonkd 2y ago

Snappy software requires the design of the system to be in sync with the engineering. Also the team needs to be aligned on snappiness being a priority. Product designers not understanding how adding certain UI elements, animations or requesting certain features is the main cause from my experience. The fastest way to speed things up in certain cases is to just remove some elements. I would expect the situation to get worse rather than better if LLMs can deliver any feature a product designer wants without limitation and if the product designer doesn't care about "snappiness"

jessriedel 2y ago
The assumption of the hypothetical is not that they can deliver any feature, which I consider (maybe you disagree) more open-ended and less automatable than porting and optimization. If you think that’s an unrealistic assumption, please feel free to propose another!
- hansonkd 2y ago
  I'm saying that it doesn't matter how much porting and optimization the llms do if the person designing the product doesn't care about efficiency. Currently designers have pretty limitless ability to add features even without LLMs. LLMs will just compound that. The only way to get efficient code is to have the right mindset from the get go. Not as an after thought that you hope the LLM will optimize.
  - jessriedel 2y ago
    Ok. Personally, I dispute that the designers don’t care about efficiency. I think it’s just expensive.

Retr0id 2y ago

I think "software is a gas" is the better explanation. ("Software always expands to fit whatever container it is stored in") Microsoft has enough engineer-hours on hand to make snappy UIs if they wanted to, but it's just not a priority for them. More/cheaper engineer hours doesn't really change that prioritization.

jessriedel 2y ago
Microsoft would hire a ton more engineers and look very different if engineer price went to ~zero. So what does that look like? If you can make an app snappy with $100 in GPU time, why wouldn’t they?
- Retr0id 2y ago
  Because they'd rather spend that $100 in GPU time on adding a new feature. Repeat that a few times and now your app is 10x more complex and costs 1000x more to perf-optimise.
  - jessriedel 2y ago
    The hypothetical is only interesting if we draw a distinction between easy and hard tasks. (“AI can do anything a human can do behind a keyboard” is AGI complete.) For this hypothetical I asserted that performance optimization and porting were automatable, but that doesn’t include any feature. If you think there’s a more interesting/realistic/well-defined assumption, please suggest it!

qwertox 2y ago

There is a future where this is within the realm of the possible. All it takes is one good AI capable of doing this which can then be duplicated many times to work on this task. An AI engineer which knows how languages compile down to machine code and understands the intention of the blueprint, which the slow app would be. 5 to 10 years, maybe? I mean, we currently think in the context of programming languages as written text, but AI has shown that it's capable of doing speech-to-speech "tasks" (for lack of a better word), where tokens are not only present in the form of text tokens, but also audio tokens, which skips the TTS step. It might become able to do video-to-assembly tasks, where it sees what an electron app does, maybe also reads the code, and then knows what needs to be done to convert this into optimized assembly code.

alpaca128 2y ago

> cycles are cheap ...when it's other people's cycles & time. If Google can squeeze the last 0.1% out of their servers, they will. But there's no incentive to make the experience better for users unless it both has a measurable impact and there's a viable competitor.

NoboruWataya 2y ago

That's one possible outcome, lets call it outcome A. The others that occur to me are that (B) software stays the same but gets cheaper, or (C) software stays the same and is still expensive but profits increase. I imagine companies would much prefer outcome C if they can get away with it, and will do everything in their power to achieve it. But they are subject to market pressures so may be forced into one of the other outcomes. What outcome do you think the market will force? Personally I think we will end up with a mix of outcomes B and C.