3 ms·
The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always bu
by paxys 7mo ago
The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.
- kkralev 7mo ago[flagged]
- wmf 7mo agojust want to run a 7-8b model locally This is already solved by running LM Studio on a normal computer.
- zozbot234 7mo agoOllama or llama.cpp are also common alternatives. But a 8B model isn't going to have much real-world knowledge or be highly reliable for agentic workloads, so it makes sense that people will want more than that.
- zach_vantio 7mo agothe compute density is insane. but giving a 70B model actual write access locally for agentic workloads is a massive liability. they still hallucinate too much. raw compute without strict state control is basically just a blast radius waiting to happen.
- nine_k 7mo agoWould a hedge fund that does not want to trust to a public AI cloud just buy chassis, mobos, GPUs, etc, and build an equivalent themselves? I suspect they value their time differently.
- paxys 7mo agoWhy do you think a hedge fund can't hire a couple of IT guys? Most of the larger ones have technical operations that would put big tech to shame.
- ViscountPenguin 7mo agoMedium sized hedge funds are a good portion of the market, and only really want to hire just enough tech people to keep the quant pipelines running.
- signal_v1 7mo ago[dead]
- p1esk 7mo agoThey wouldn’t build anything - they would order from Dell or Supermicro.
- qubex 7mo agoThey’re kickstarting a TINY device that is pocketable and aimed at consumers. I’ve backed it (full disclosure).
- jgrizou 7mo agohttps://www.kickstarter.com/projects/tiinyai/tiiny-ai-pocket-lab https://www.kickstarter.com/projects/tiinyai/tiiny-ai-pocket...
- griffinmb 7mo agoThis is not the same company. The OP Tiny Corp accused them of Trademark infringement on Twitter, due to exactly this kind of misconception.
- qubex 7mo agoI might’ve been conned: https://bay41.com/posts/tiiny-ai-pocket-lab-review/ https://bay41.com/posts/tiiny-ai-pocket-lab-review/
- Fr0styMatt88 7mo agoYou can still cancel your pledge. I’ve just done that myself.
- ankaz 7mo ago[dead]
- mihaaly 7mo agoWe may be surprised how illefficient companies are in organizing the creation of sophisticated things (including processes) for themselves, to use (so for the cost center column). Higher management figures out things to do in strategic level, in brief, and pushes on "soldiers", who kick it through in the least time (cheapest of the cheapest, for the sake of the quarterlies) EXACTLY the way management told it. Because they have to, their job is to make happen the company objectives given, the way it is given. Pushing out crap in the shape of the thing expected. Larger organiztaion can use these kind of things the most. Even if they don't do that.