4 ms·
Really cool to see this! Local is the real future of AI. I got really excited about this and fired it up on my petite little M2 Macbook Air only for it to grin
by mavsman 3y ago
Really cool to see this! Local is the real future of AI.
I got really excited about this and fired it up on my petite little M2 Macbook Air only for it to grind it to a halt. Think the old days when you had a virus on your PC and you'd move the mouse then wait 45 seconds to see the cursor move. It honestly made me feel nostalgic. I guess I have to taper performance expectations with this Air, though this is the first time it's happened.
- quickthrower2 3y agoJust wait 10 years when computers have a dedicated AI-PU and you don't have to worry about freezing anything up to talk to your bot.
- kridsdale1 3y agoThe M1 line of Macs does have that unit.
- joaogui1 3y agoI wonder if edge tpus (sold here https://coral.ai/products/ https://coral.ai/products/) could help, though I guess the community would have to optimize for them instead of for standard hardware
- jmorgan 3y agoHow much memory do you have in your Macbook? The 7B models seem to work well with at least 16GB of unified memory, but I’ve seen Macs with 8GB really struggle.
- mavsman 3y agoIndeed it's just a poor little 8GB of RAM.