4 ms·
Use local OSS models then? They aren’t as good and you need beefy hardware (either Apple silicon or nvidia GPUs). But they are totally workable, and you avoid y
by seanmcdirmid 9mo ago
Use local OSS models then? They aren’t as good and you need beefy hardware (either Apple silicon or nvidia GPUs). But they are totally workable, and you avoid your dislikes directly.
- davidw 9mo ago"Not as good and costs a lot in hardware" still sounds like I'm at a disadvantage.
- seanmcdirmid 9mo ago$3000 is not that much for hardware (like a refurbished MBP Max with decent amount of RAM), and you'd be surprised how much more useful a thing that is slightly worse than the expensive thing is when you don't have anxiety about token usage.
- davidw 9mo ago$3000 might not be much to a wealthy software engineer in the US, but to, say, a college student in Portugal, it's a big expense. Open source software democratized software in a huge way.
- seanmcdirmid 9mo agoOk, from that perspective we are still a few years from when a college student in Portugal can run local OSS models on their own hardware...but we aren't a few decades away from that, at least.