3 ms·

Looks like a solution in search of a problem.

bayindirh 3y ago
AI workloads are currently going through their "developers are expensive, hardware is cheap(ish)" era. Nobody is trying very hard to fit models to GPUs, and optimize utilization and whatnot. Instead they throw in more GPUs. So B100 will fit right into the bill.
- hardware2win 3y ago
  >"developers are expensive, hardware is cheap(ish)" Is it even true? H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax
  - bayindirh 3y ago
    Generally it's thought in the hourly cost, plus how much productivity it adds to your workflow. IOW, you pay yearly price of an engineer, plus some hourly cost, but it generally adds more productivity rather than optimizing what you have at hand. > H100 is worth 1 year of salary of mid engineer in eastern eu If you can have them supplied to you, in the first place.
- dpacmittal 3y ago
  There's a lot of work being put in to make LLMs run on consumer hardware, albeit it's being done by the open source community and not the companies themselves.
  - rapsey 3y ago
    Apple is most definitely working on running them on iPhones.
    - bayindirh 3y ago
      They are already running some of the Siri stuff completely offline and on device, AFAIK. Also, they have tons of image related AI infrastructure running on iPhones already.
  - bayindirh 3y ago
    I mean, for the training part. Inference is easy (relative to training). Even my 3 year old camera can do real-time eye/face-tracking for humans and mammals via a real-time neural network running on a dedicated low power DSP.
  - willis936 3y ago
    The consumer hardware is made by the same company and there's little competition. Rather than undercut the price of commercial hardware, what commercial applications for GPUs has done is raise the price floor of consumer GPUs. The companies are not undercutting themselves. It's profit city.
    - bayindirh 3y ago
      I think AI accelerators in ARM based systems will be start to eat that green company from below. Even Intel's bottom of the barrel N95/N100 has accelerators now.
3abiton 3y ago
The problem is EU's winter. Pair this bad boy with a heat pump and watch it roar.

AI workloads are currently going through their "developers are expensive, hardware is cheap(ish)" era. Nobody is trying very hard to fit models to GPUs, and optimize utilization and whatnot. Instead they throw in more GPUs. So B100 will fit right into the bill.

hardware2win 3y ago
>"developers are expensive, hardware is cheap(ish)" Is it even true? H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax
- bayindirh 3y ago
  Generally it's thought in the hourly cost, plus how much productivity it adds to your workflow. IOW, you pay yearly price of an engineer, plus some hourly cost, but it generally adds more productivity rather than optimizing what you have at hand. > H100 is worth 1 year of salary of mid engineer in eastern eu If you can have them supplied to you, in the first place.
dpacmittal 3y ago
There's a lot of work being put in to make LLMs run on consumer hardware, albeit it's being done by the open source community and not the companies themselves.
- rapsey 3y ago
  Apple is most definitely working on running them on iPhones.
  - bayindirh 3y ago
    They are already running some of the Siri stuff completely offline and on device, AFAIK. Also, they have tons of image related AI infrastructure running on iPhones already.
- bayindirh 3y ago
  I mean, for the training part. Inference is easy (relative to training). Even my 3 year old camera can do real-time eye/face-tracking for humans and mammals via a real-time neural network running on a dedicated low power DSP.
- willis936 3y ago
  The consumer hardware is made by the same company and there's little competition. Rather than undercut the price of commercial hardware, what commercial applications for GPUs has done is raise the price floor of consumer GPUs. The companies are not undercutting themselves. It's profit city.
  - bayindirh 3y ago
    I think AI accelerators in ARM based systems will be start to eat that green company from below. Even Intel's bottom of the barrel N95/N100 has accelerators now.

3abiton 3y ago

The problem is EU's winter. Pair this bad boy with a heat pump and watch it roar.