I don't know if it applies to older gen, but the team behind mlc-llm are suggesting that at least with steamdeck's APU you can go beyond the cap, hopefully it a

I don't know if it applies to older gen, but the team behind mlc-llm are suggesting that at least with steamdeck's APU you can go beyond the cap, hopefully it applies to other APUs: https://blog.mlc.ai/2023/08/09/Making-AMD-GPUs-competitive-for-LLM-inference#running-on-steamdeck-using-vulkan-with-unified-memory https://blog.mlc.ai/2023/08/09/Making-AMD-GPUs-competitive-f... Regarding the choice of device, I'm regularly in places where 24h electricity/internet is not guaranteed, so renting cloud gpus: nope, bulky gaming laptop that chews through battery in a couple of hours: nope (even though I came really close to getting one with a RTX 4090 for 2600€, but I'm done with space heaters), frame.work 13? RAM to slow... Mobile devices with at least 64GB of lpddr5 with decent battery consumption? The choice is quite limited.