4 ms·

> 4 seconds to run on my Mac Studio, which cost $8,300 USD Jesus, is it common for developers to have such expensive computers these days?

modernpink 3y ago
Considering the advances in computational hardware over the past few decades plus the corresponding (and not unrelated) real increase in developer salaries, it is unreasonably cheap.
- mistrial9 3y ago
  plus it phones-home for safety
simonw 3y ago
Not exactly common, but it's not unsurprising. $8,000 computers have got really, really good! If you made a living as a plumber you would spend a lot more than that on tools and a pickup truck.
ElectricalUnion 3y ago
> Jesus, is it common for developers to have such expensive computers these days? Computers are really cheap now. A PDP-8, the first really successful minicomputer (read very cheap minicomputer), was around 18,500 USD, in 1965's USD, or 170,000 USD in 2023's USD. For a historic comparison, the price of a introductory minimal system for an actual "mainframe class computer" of the same vintage, a IBM System/360 Model 30, was 133,000 USD in 1965's USD, or around 1,225,000 USD in 2023's USD. Those 8300 USD cited are very cheap. A person in the bleeding edge of the private AI sector is expected to handle several Nvidia H100 80GB, each with a individual cost around 40,000 USD per unit. Those 8300 USD cited are peanuts in comparison.
- bekantan 3y ago
  I don't mind paying, but I want to have a Linux workstation. What would be x86 alternative in that price range (if any)? Xeons with HBM are more expensive IIRC
  - jart 3y ago
    I know it's arm64 but you can hack Mac Studio and put Linux on it. Putting Linux on Apple Silicon makes it go so much faster than MacOS if you're doing command line development work. With x86 there really isn't anything you can buy as far as I know that'll go as fast as apple silicon for cpu inference. It's because the RAM isn't integrated into the CPU system on chip. For example, if you get a high-end x86 system with 5000+ MT/s then you can expect maybe ~15 tokens per second at CPU inference, but a high end Mac Studio does CPU inference at ~36 tokens per second. Not that it matters though if you're planning to get a high-end Nvidia card for your x86 computer. Nvidia GPUs go very fast and so does Apple Metal.
    - ElectricalUnion 3y ago
      If you have really unlimited budget, unconditional love for Intel and x86 and don't care about ludicrous power draw at all, Intel has a silly Sapphire Rapids Xeon Max part with 64GiB of 1TB/s HBM. It goes really fast (same magnitude of bandwith as A100s) if your model fits in that cache entirely.
Art9681 3y ago
Used wisely that $8,300 computer will pay for itself and a lot more. But I wouldn't gamble it either. Those that have a plan to multiply its value dont need my encouragement.

modernpink 3y ago

Considering the advances in computational hardware over the past few decades plus the corresponding (and not unrelated) real increase in developer salaries, it is unreasonably cheap.

mistrial9 3y ago
plus it phones-home for safety

simonw 3y ago

Not exactly common, but it's not unsurprising. $8,000 computers have got really, really good! If you made a living as a plumber you would spend a lot more than that on tools and a pickup truck.

ElectricalUnion 3y ago

> Jesus, is it common for developers to have such expensive computers these days? Computers are really cheap now. A PDP-8, the first really successful minicomputer (read very cheap minicomputer), was around 18,500 USD, in 1965's USD, or 170,000 USD in 2023's USD. For a historic comparison, the price of a introductory minimal system for an actual "mainframe class computer" of the same vintage, a IBM System/360 Model 30, was 133,000 USD in 1965's USD, or around 1,225,000 USD in 2023's USD. Those 8300 USD cited are very cheap. A person in the bleeding edge of the private AI sector is expected to handle several Nvidia H100 80GB, each with a individual cost around 40,000 USD per unit. Those 8300 USD cited are peanuts in comparison.

bekantan 3y ago
I don't mind paying, but I want to have a Linux workstation. What would be x86 alternative in that price range (if any)? Xeons with HBM are more expensive IIRC
- jart 3y ago
  I know it's arm64 but you can hack Mac Studio and put Linux on it. Putting Linux on Apple Silicon makes it go so much faster than MacOS if you're doing command line development work. With x86 there really isn't anything you can buy as far as I know that'll go as fast as apple silicon for cpu inference. It's because the RAM isn't integrated into the CPU system on chip. For example, if you get a high-end x86 system with 5000+ MT/s then you can expect maybe ~15 tokens per second at CPU inference, but a high end Mac Studio does CPU inference at ~36 tokens per second. Not that it matters though if you're planning to get a high-end Nvidia card for your x86 computer. Nvidia GPUs go very fast and so does Apple Metal.
  - ElectricalUnion 3y ago
    If you have really unlimited budget, unconditional love for Intel and x86 and don't care about ludicrous power draw at all, Intel has a silly Sapphire Rapids Xeon Max part with 64GiB of 1TB/s HBM. It goes really fast (same magnitude of bandwith as A100s) if your model fits in that cache entirely.

Art9681 3y ago

Used wisely that $8,300 computer will pay for itself and a lot more. But I wouldn't gamble it either. Those that have a plan to multiply its value dont need my encouragement.