6 ms·

Baking the base models on to ROM makes a lot of economic sense. Less so for consumers though, because it'd mean the phone is out of date in 3 months when a bet

by onion2k 2mo ago

Baking the base models on to ROM makes a lot of economic sense. Less so for consumers though, because it'd mean the phone is out of date in 3 months when a better model comes along.

ronsor 2mo ago
It's a perfect reason to get consumers to buy a new phone every year again! They got bored of the camera.
- miohtama 2mo ago
  This stopped working some years ago. Now you sell the same phone with higher price tag.
- ben_w 2mo ago
  Right now the models are doubling in performance (by the METR time horizon metric at least) every 4 months, so 3 doublings in a year; conversely, I hear (not my field) it takes around a year to make a prototype IC and another year to turn that into mass production, i.e. if the next (late-2026 model) iPhone has a chip like this, it will likely be with, at best, a late-2024 set of weights. I think you can get open-weights models today that have performance equivalent to the SOTA-late-2024 while fitting in the RAM of a (high end) 2025-26 phone. At some point the music will stop on training bigger models, and when that happens it will make sense to have ROM weights (or 100% analog circuits given how noise-resistant LLMs are), but we'll know when that is because the investment bubble funding the training of new models will have burst.
  - dgrabla 2mo ago
    wait, analog circuits? can you elaborate this?
    - inigyou 2mo ago
      Circuits that aren't restricted to two particular voltage levels.
    - ben_w 2mo ago
      Transistors can be used to amplify signals, they are not limited to acting as binary switches. If you use analog rather than digital, using transistors in this way means you can replace however many transistors it would have taken for multiplying two n-bit numbers with just one; I understand capacitors can be used for accumulation, but don't know how many additional components that needs as I'm an electronics noob. The reason we don't do this in general (any more) is that for long chains between input and output it has been much too difficult to avoid accumulation of errors. LLMs happen to be extremely resilient to errors like this, which is also why we can use e.g. 4-bit weights.
  - close04 2mo ago
    At some point it’s got to be good enough for the normal “phone stuff” that appeal to most users. So they wouldn’t suffer from FOMO because they didn’t wait for the next model. Every phone gimmick went through the same evolution curve until it passed the “good enough” point and eventually plateaued.
    - ben_w 2mo ago
      Yes, but irrelevant. While these models are improving at the present rate, the manufacturer can save money at no loss of feature-bullet-point-on-website by letting you download a model after you bought the thing and running it on normal hardware. The rate of change to the models has to be slower than the hardware roll-out to be worth a hardware solution. If "good enough" happens before then, that just means the user gets a software solution.
      - close04 2mo ago
        You might be right but hard to tell without analyzing costs and benefits. Is a cutting edge model for phone stuff worth the slower performance and battery drain for example? The rate of change by itself doesn’t tell you the whole story because of costs and diminishing returns. So what if your model is twice as good if it’s 10x the cost and it saves you 1ms? Everything else about phones reached “good enough for a phone” levels in years, and then got minimal generational improvements.
chorizo 2mo ago
And the customers can wait for the new phone released next year. These are edge models - the average customer doesn’t need the latest frontier model. Just needs to be good enough for the features you promised.
otabdeveloper4 2mo ago
That's a software engineering problem. They just need to figure out how to fine-tune for alignment and tool usage. That's the only thing the normie consumer cares for really.
fsloth 2mo ago
I don’t think average user _needs_ to solve frontier challenges. ”Call to Jane”, ”turn on the lights” and ”what’s the weather this afternoon” is more like it I would guess. Ofc if the model has some critical bugs that’s another matter.
- dzhiurgis 2mo ago
  Your examples worked on phones for over a decade. Maybe baking in a model that is "certified" to have some unconditioned truths + rest is pulled from external models/store could make sense. But AFAIK that doesn't exist and I'm not sure it can possibly be made. Perhaps society as a whole at least can work on an open corpus of training data, but I'm not holding my breath on this.
  - logicallee 2mo ago
    >Your examples worked on phones for over a decade. Nope. And not only not a decade ago, right now. If you have an Android or iPhone, you can give it clear and easy to understand instructions that Gemma 4 could complete[1] if it had tool calls on it, and that 100.00% of Claude, ChatGPT, Grok, Kimi, you name it, could understand and all complete if they had the access. The phones will fail to complete it. I just tried Siri. I said "hey Siri", waited for Siri to come up, and then I asked one of the exact sentences you replied to: "what's the weather this afternoon?" It thought for around 20 seconds, and said "Something went wrong. Please try again."[2] I have Wifi, I have mobile Internet, I have free storage space, I have up to date software. What went wrong is that phones have never properly connected agents, not ten years ago, not last year, not this year, and probably not next year. But don't settle for what Google could do in 1999 by hotlinking the keyword "weather" in any query to the weather being shown in the results. Tell your phone (any phone): "Please call back the last number that called me that is not an unlisted number, regardless of who it came from." 0 out of any phone will complete that today, tomorrow, a year from now, five years from now, ever, because phone makers are not going to let them do that. Meanwhile, 100% of all frontier agents could complete it if they had tool calls on the phone. Which they don't, and won't ever, thanks to the duopoly. Okay, that's a bit dismissive, I would love to be wrong! [1] after any voice recognition to text - which does work really well on both Android and iPhone! [2] screenshot: https://ibb.co/21rtDnfV https://ibb.co/21rtDnfV

by onion2k 2mo ago

Baking the base models on to ROM makes a lot of economic sense. Less so for consumers though, because it'd mean the phone is out of date in 3 months when a better model comes along.

ronsor 2mo ago

It's a perfect reason to get consumers to buy a new phone every year again! They got bored of the camera.

miohtama 2mo ago
This stopped working some years ago. Now you sell the same phone with higher price tag.
ben_w 2mo ago
Right now the models are doubling in performance (by the METR time horizon metric at least) every 4 months, so 3 doublings in a year; conversely, I hear (not my field) it takes around a year to make a prototype IC and another year to turn that into mass production, i.e. if the next (late-2026 model) iPhone has a chip like this, it will likely be with, at best, a late-2024 set of weights. I think you can get open-weights models today that have performance equivalent to the SOTA-late-2024 while fitting in the RAM of a (high end) 2025-26 phone. At some point the music will stop on training bigger models, and when that happens it will make sense to have ROM weights (or 100% analog circuits given how noise-resistant LLMs are), but we'll know when that is because the investment bubble funding the training of new models will have burst.
- dgrabla 2mo ago
  wait, analog circuits? can you elaborate this?
  - inigyou 2mo ago
    Circuits that aren't restricted to two particular voltage levels.
  - ben_w 2mo ago
    Transistors can be used to amplify signals, they are not limited to acting as binary switches. If you use analog rather than digital, using transistors in this way means you can replace however many transistors it would have taken for multiplying two n-bit numbers with just one; I understand capacitors can be used for accumulation, but don't know how many additional components that needs as I'm an electronics noob. The reason we don't do this in general (any more) is that for long chains between input and output it has been much too difficult to avoid accumulation of errors. LLMs happen to be extremely resilient to errors like this, which is also why we can use e.g. 4-bit weights.
- close04 2mo ago
  At some point it’s got to be good enough for the normal “phone stuff” that appeal to most users. So they wouldn’t suffer from FOMO because they didn’t wait for the next model. Every phone gimmick went through the same evolution curve until it passed the “good enough” point and eventually plateaued.
  - ben_w 2mo ago
    Yes, but irrelevant. While these models are improving at the present rate, the manufacturer can save money at no loss of feature-bullet-point-on-website by letting you download a model after you bought the thing and running it on normal hardware. The rate of change to the models has to be slower than the hardware roll-out to be worth a hardware solution. If "good enough" happens before then, that just means the user gets a software solution.
    - close04 2mo ago
      You might be right but hard to tell without analyzing costs and benefits. Is a cutting edge model for phone stuff worth the slower performance and battery drain for example? The rate of change by itself doesn’t tell you the whole story because of costs and diminishing returns. So what if your model is twice as good if it’s 10x the cost and it saves you 1ms? Everything else about phones reached “good enough for a phone” levels in years, and then got minimal generational improvements.

chorizo 2mo ago

And the customers can wait for the new phone released next year. These are edge models - the average customer doesn’t need the latest frontier model. Just needs to be good enough for the features you promised.

otabdeveloper4 2mo ago

That's a software engineering problem. They just need to figure out how to fine-tune for alignment and tool usage. That's the only thing the normie consumer cares for really.

fsloth 2mo ago

I don’t think average user _needs_ to solve frontier challenges. ”Call to Jane”, ”turn on the lights” and ”what’s the weather this afternoon” is more like it I would guess. Ofc if the model has some critical bugs that’s another matter.

dzhiurgis 2mo ago
Your examples worked on phones for over a decade. Maybe baking in a model that is "certified" to have some unconditioned truths + rest is pulled from external models/store could make sense. But AFAIK that doesn't exist and I'm not sure it can possibly be made. Perhaps society as a whole at least can work on an open corpus of training data, but I'm not holding my breath on this.
- logicallee 2mo ago
  >Your examples worked on phones for over a decade. Nope. And not only not a decade ago, right now. If you have an Android or iPhone, you can give it clear and easy to understand instructions that Gemma 4 could complete[1] if it had tool calls on it, and that 100.00% of Claude, ChatGPT, Grok, Kimi, you name it, could understand and all complete if they had the access. The phones will fail to complete it. I just tried Siri. I said "hey Siri", waited for Siri to come up, and then I asked one of the exact sentences you replied to: "what's the weather this afternoon?" It thought for around 20 seconds, and said "Something went wrong. Please try again."[2] I have Wifi, I have mobile Internet, I have free storage space, I have up to date software. What went wrong is that phones have never properly connected agents, not ten years ago, not last year, not this year, and probably not next year. But don't settle for what Google could do in 1999 by hotlinking the keyword "weather" in any query to the weather being shown in the results. Tell your phone (any phone): "Please call back the last number that called me that is not an unlisted number, regardless of who it came from." 0 out of any phone will complete that today, tomorrow, a year from now, five years from now, ever, because phone makers are not going to let them do that. Meanwhile, 100% of all frontier agents could complete it if they had tool calls on the phone. Which they don't, and won't ever, thanks to the duopoly. Okay, that's a bit dismissive, I would love to be wrong! [1] after any voice recognition to text - which does work really well on both Android and iPhone! [2] screenshot: https://ibb.co/21rtDnfV https://ibb.co/21rtDnfV