4 ms·
Let's see which company becomes the first to sell "coding appliances": hardware with a model good enough for normal coding. If Mistral is so permissive they co
by qwertox 11mo ago
Let's see which company becomes the first to sell "coding appliances": hardware with a model good enough for normal coding.
If Mistral is so permissive they could be the first ones, provided that hardware is then fast/cheap/efficient enough to create a small box that can be placed in an office.
Maybe in 5 years.
- baq 11mo agollm in a box connected via usb is the dream. ...so it won't ever happen, it'll require wifi and will only be accessible via the cloud, and you'll have to pay a subscription fee to access the hardware you bought. obviously.
- brazukadev 11mo agomy bet is a deepseek box
- bakies 11mo agoI bought a framework desktop hoping to do this.
- sosodev 11mo agoAnd it can do it, right? I think AMD AI Max line the first realistic offering for this type of thing. The Apple offerings are interesting but the lack of x86, Linux, and general compatibility make it hard sell imo.
- bakies 11mo agoHaven't started relying on it for anything but it does have decent performance at like 130b parameters. I also haven't tried to tune it at all. I saw some recommendations somewhere but I installed talos on it and didn't want to figure any of that out.
- sosodev 11mo agoI just got my framework mainboard today. I haven't had a chance to set it up yet but from the research I've been doing it seems like Minimax M2 might be the best coding model for it at the moment. Similar performance to Devstral2 with only 10b active params.
- giancarlostoro 11mo agoMy Macbook Pro with an M4 Pro chip can handle a number of these models (I think it has 16GB of VRAM) with reasonable performance, my bottleneck continuously is the token caps. I assume someone with a much more powerful Mac Studio could run way more than I can, considering they get access to about 96GB of VRAM out of the system RAM iirc.