3 ms·
Is there something specifically with Kimi that's better here? As far as I know Kimi pricing is about the same as Sonnet 5 -- what happens if you use that model
by nharada 2mo ago
Is there something specifically with Kimi that's better here? As far as I know Kimi pricing is about the same as Sonnet 5 -- what happens if you use that model and Fable instead? Or Grok 4.5 which is even cheaper?
- ralusek 2mo agoOne benefit of an open source one is that you can, as a large corporation, run it "locally" within your own data center. Even fine tune it.
- culi 2mo agoThere are very few companies that would ever be able to afford to run it themselves. But it does give you the security that it's technically possible. Puts some limits on stuff like Moonshot changing their terms/conditions/policies
- adamisnotroman 2mo agoTechnically, "open weight" but yeah
- lenerdenator 2mo agoThe problem with hosting these right now is that it's not a known quantity like lots of traditional business software is. You can more-or-less approximate what your build out spend for a data center hosting database software is going to be. Your book of business will require a given amount of revenue to pay it off, but once that's known, it's off to the races. With AI, things are moving so fast and new business models are being tried all the time. You would be competing with some of the wealthiest companies in the world for data center hardware capable of hosting these models in a usable state. We've gone with Claude somehow hosted through GCP Vertex AI where I'm at.
- claytonjy 2mo agoHow big is this market, self-hosting a model that requires 64 GPUs, H100 or better, with good interconnects between nodes? I suspect the overlap of those that can afford it, and those that have the talent to manage it, is a fairly thin slice of the Venn diagram. Even the large corps are gonna be getting it from the inference vendors, or more likely Bedrock and friends.
- Maxious 2mo agoDell will sell you a PowerEdge XE7740/XE7745 "AI Factory" with 32 H200s https://infohub.delltechnologies.com/en-au/t/dell-ai-factory-with-nvidia-gpus-and-open-source-software-stack/ https://infohub.delltechnologies.com/en-au/t/dell-ai-factory... They "booked $24 billion in AI server orders this quarter as its customer base broadened to more than 5,000" https://finance.yahoo.com/markets/stocks/articles/dells-ai-factory-booming-backlog-213100173.html?guccounter=1 https://finance.yahoo.com/markets/stocks/articles/dells-ai-f...
- zbendefy 2mo agoDont have much experience how well they perform but quantized models can run on ~4 H100 or A100 which should be true for kimi k3 and qwen 3.8 as well.
- DennisP 2mo agoThe article says Kimi is better at some things than Fable. That's probably not true of Sonnet.