4 ms·
Ya, i dont know of anyone wanting to run very large AI models in a windows environment. Or, frankly, on a laptop. Why not just VPN into a dedicated server?
by sandworm101 5mo ago
Ya, i dont know of anyone wanting to run very large AI models in a windows environment. Or, frankly, on a laptop. Why not just VPN into a dedicated server?
- whywhywhywhy 5mo agoHow much does a dedicated server with 128GB vram cost a month.
- sandworm101 5mo agoLess than this laptop.
- forthefuture 5mo agoYou can get an H200 (141GB) here for $2,700/mo: https://deploybase.ai/articles/h200-price https://deploybase.ai/articles/h200-price I could be wrong but my understanding is that 24/7 dedicated servers are wildly economically unviable. The reason cloud tends to cost less than local today (other than the subsidization) is because you aren't running models 24/7. So like 6 hours of cloud per weekday might beat the yearly cost of building local machines, but it's not in the same universe if you're running 24/7, as evidenced by two months of H200 rental costing more than the DGX Spark this Laptop is built out of.
- undersuit 5mo agoHow well will the local LLM run when your laptop is in your bag while you're walking around?
- mort96 5mo agoI mean not that much? You buy the hardware once and then it’s just running for many years
- satvikpendem 5mo agoI do. I can take my laptop anywhere I want, for example to a coffee shop and run a coding model while eating a croissant without worrying about an internet connection, as the term local model implies.
- taffydavid 5mo agoAnd you can warm up the croissant by just placing it on the trackpad while you wait for the LLM to finish
- satvikpendem 5mo agoExactly, it's a built in reheater.
- nozzlegear 5mo agoThe coffee shop doesn't have wifi?
- thewebguyd 5mo agoWith BUILD happening tomorrow, I suspect Microsoft is going to have some stuff about local AI there with MS Foundry on Windows/Foundry Local. The timing of this announcement a day before BUILD is obviously intentional. Suddenly all the Windows K2 stuff makes sense, but I doubt it'll be enough. Its too little too late for Microsoft.
- ActionHank 5mo agoThey've spent more than a decade cratering their OS. I doubt they will be able to turn it around with one code-named project.
- ActionHank 5mo agoNo one is doing serious work on Windows anymore, those who are have yet to realise the clownshow they are collectively a part of.