It runs on a 2022 MBP with m1 chip. Stuff like llama is okay, but the quality does not match with claude-sonnet 3, much less 3.5. The delay and set up time can

It runs on a 2022 MBP with m1 chip. Stuff like llama is okay, but the quality does not match with claude-sonnet 3, much less 3.5. The delay and set up time can be quite significant. In the end, it's just more worth it to pay the $0.07 or so to run it from a proprietary model. And something like Cursor has .cursorrules and privacy modes and their own enterprise grade stuff. So privacy is not as much as an issue, but you got to do the homework to see if it works for you. The killer feature for locally hosted AI seems to be that it's consistent. No sudden patches breaking everything, no paranoia whenever quality spikes after demo day and then drops 3 months after. No worrying that prices are going from $20 to $200/month. AWS Bedrock is also pretty good and secure if you want to use AWS, and it comes with a suite of other tools like evaluation and guardrails.