4 ms·

I don’t get the need. With gaming there’s a real benefit to having the card close to a display. There’s enough benefit that you don’t mind it being unused 20 ho

I don’t get the need. With gaming there’s a real benefit to having the card close to a display. There’s enough benefit that you don’t mind it being unused 20 hours a day. There’s relatively little benefit to having training happen a few feet away rather than a data center. Solid chance it sits unused most of the time, and when you really need it you run into capacity issues, so you’d need to predict your future needs carefully or be happy waiting for a job to finish. AI training feels like transport. You rent the capacity/vehicle you need on demand, benefit from yearly upgrades. Very few people are doing so much training that they need a local powerhouse, upgraded every year or so. Even sharing the hardware in a pool seems more rational. Pay 200/month for access to a semi private cluster rather than having it sit on your desk.

echelon 2y ago
A video editor wants the tool sitting on their desk. Not pay-per-gen SaaS where the results are garbage nine out of ten times. The comfy ecosystem is rife with people that want local tools.
- returnInfinity 2y ago
  6000 / 24 = 250 usd If you replace your GPU every 2 years, its 250 usd per month If the price halves, still its 125 usd. Even if price halves and use for 5 years, its 50 usd per month.
  - echelon 2y ago
    People are already spending far more than that on Runway and Kling: > I spent $745 in Kling credits to bring the Princess Mononoke trailer to life. https://www.reddit.com/r/aivideo/comments/1fvchbf/i_spent_745_in_kling_credits_to_bring_the/ https://www.reddit.com/r/aivideo/comments/1fvchbf/i_spent_74... Most of that is in failed generations.
ClemFandango22 2y ago
I'm not too familiar with the AI space but I wonder if this is an effort from NVIDIA to combine their AI and Gaming markets. Did this come from a conversational question stating, 'How do we sell discrete cards through our existing manufacturing partnerships to both gaming enthusiasts and AI enthusiasts?'. I do wonder how comfortable they are pivoting back to being a consumer hardware company if AI becomes a more competitive space, or if the 'hype' subsides. Pure speculation and I'm probably off the mark.
- richardw 2y ago
  Yeah quite possible. They have distribution, brand, customers used to paying a lot of money. Can go far just selling to the portion of existing customers who would like "the best local setup for AI".
kadushka 2y ago
I see the DIGITs box as mainly for inference, not training. It allows me to load a fairly large model (e.g. 70B llama or 12B flux) and run it locally at decent speeds.
- richardw 2y ago
  Then surely far simpler custom chips are the eventual model, like happened with crypto? Groq, Etched etc. In that universe, Nvidia has absolutely no moat and a thousand chips are coming.
  - kadushka 2y ago
    Maybe. For me to consider buying some unknown hardware it needs be a lot better than Nvidia. Like double the speed, half the power, and half the price. And the software better be rock solid and tested on all popular models. Many startups have been trying for several years now, and eventually someone will succeed, but it's not easy. Even AMD hasn't been able to pull it off.
gunian 2y ago
privacy for machine learning to be locally usable besides a chat interface in a meaningful way I would have to give it access to pretty much all my digital data sending that over the network feels very idk icky because it's not just photos or emails
sliken 2y ago
A local NAS makes sense for some use cases. Privacy, local control, and network bandwidth adds up to having a NAS for some use cases. I see similar coming for AI. Tagging local photos, reading/summarizing private documents, helping you code (without uploaded it to 3rd parties), using uncensored LLMs, maybe even playing NPCs once games support API for LLMs. It's going to take awhile, but I wouldn't be surprised if NASs start supporting AI accelerators to provide local endpoints for AI.

echelon 2y ago

A video editor wants the tool sitting on their desk. Not pay-per-gen SaaS where the results are garbage nine out of ten times. The comfy ecosystem is rife with people that want local tools.

returnInfinity 2y ago
6000 / 24 = 250 usd If you replace your GPU every 2 years, its 250 usd per month If the price halves, still its 125 usd. Even if price halves and use for 5 years, its 50 usd per month.
- echelon 2y ago
  People are already spending far more than that on Runway and Kling: > I spent $745 in Kling credits to bring the Princess Mononoke trailer to life. https://www.reddit.com/r/aivideo/comments/1fvchbf/i_spent_745_in_kling_credits_to_bring_the/ https://www.reddit.com/r/aivideo/comments/1fvchbf/i_spent_74... Most of that is in failed generations.

ClemFandango22 2y ago

I'm not too familiar with the AI space but I wonder if this is an effort from NVIDIA to combine their AI and Gaming markets. Did this come from a conversational question stating, 'How do we sell discrete cards through our existing manufacturing partnerships to both gaming enthusiasts and AI enthusiasts?'. I do wonder how comfortable they are pivoting back to being a consumer hardware company if AI becomes a more competitive space, or if the 'hype' subsides. Pure speculation and I'm probably off the mark.

richardw 2y ago
Yeah quite possible. They have distribution, brand, customers used to paying a lot of money. Can go far just selling to the portion of existing customers who would like "the best local setup for AI".

kadushka 2y ago

I see the DIGITs box as mainly for inference, not training. It allows me to load a fairly large model (e.g. 70B llama or 12B flux) and run it locally at decent speeds.

richardw 2y ago
Then surely far simpler custom chips are the eventual model, like happened with crypto? Groq, Etched etc. In that universe, Nvidia has absolutely no moat and a thousand chips are coming.
- kadushka 2y ago
  Maybe. For me to consider buying some unknown hardware it needs be a lot better than Nvidia. Like double the speed, half the power, and half the price. And the software better be rock solid and tested on all popular models. Many startups have been trying for several years now, and eventually someone will succeed, but it's not easy. Even AMD hasn't been able to pull it off.

gunian 2y ago

privacy for machine learning to be locally usable besides a chat interface in a meaningful way I would have to give it access to pretty much all my digital data sending that over the network feels very idk icky because it's not just photos or emails

sliken 2y ago

A local NAS makes sense for some use cases. Privacy, local control, and network bandwidth adds up to having a NAS for some use cases. I see similar coming for AI. Tagging local photos, reading/summarizing private documents, helping you code (without uploaded it to 3rd parties), using uncensored LLMs, maybe even playing NPCs once games support API for LLMs. It's going to take awhile, but I wouldn't be surprised if NASs start supporting AI accelerators to provide local endpoints for AI.