3 ms·
A phone makes a very crappy AI inference rig. It's battery powered and can't even really run at 100% utilization on an ongoing basis due to how challenging the
by zozbot234 6mo ago
A phone makes a very crappy AI inference rig. It's battery powered and can't even really run at 100% utilization on an ongoing basis due to how challenging the thermals are.
- fridder 6mo agoat the moment yes. The one possible silver lining with all of the current hardware crunch is that it _should_ force some hardware advancements. The last couple years hardware has been kinda boring. My m1max is still zippy as all hell and doesn't really need to be upgraded, unless I am committing to local AI inference.
- zozbot234 6mo agoI kinda assume phones are going to be battery powered for the foreseeable future. "Gaming" phones with better cooling do exist, but they are a tiny niche. Most local AI users will want to serve their inference needs through a very different kind of system.
- array_key_first 6mo agoYes, but the battery tech itself is improving. We're already seeing new phones approach 8000 mAH internal batteries, which is large enough that you can splurge on compute and still have some left over at the end of the day.
- virgil_disgr4ce 6mo ago> it _should_ force some hardware advancements I'm very curious what kind of hardware advancements you're imagining. Because we're already kind of near a physical wall regarding heat dissipation on phones. I mean hey, maybe foundational physics will surprise the world with a radical breakthrough that disappears heat into a black hole or something, but I sure wouldn't hold my breath
- csheaff 6mo agowater cooled pant pockets
- willchis 6mo agoThe iPant? Or a Samsung WCPP1?
- goatlover 6mo agoLaunch everyone's phones into space.
- jimbokun 6mo agoHeat dissipation is even harder in space
- throwaway5752 6mo agoeGPU cradles, presumably, for people with intense local model execution requirements until it can be made to work in the device? This is exactly like the POS dongles Square had until tap to pay was more widespread?
- geon 6mo agoMore likely it would force software advancements. Current models are horribly inefficient.