4 ms·

Other than the NPU, it’s not really a game changer; here’s a 512GB AMD deepseek build for $2000: https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully

by hedora 2y ago

Other than the NPU, it’s not really a game changer; here’s a 512GB AMD deepseek build for $2000: https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/ https://digitalspaceport.com/how-to-run-deepseek-r1-671b-ful...

flakiness 2y ago
The low energy use can be a game changer if you live in a crappy apartment with limited power capacity. I gave up my big GPU box dream because of that.
aurareturn 2y ago
between 4.25 to 3.5 TPS (tokens per second) on the Q4 671b full model. 3.5 - 4.25 tokens/s. You're torturing yourself. Especially with a reasoning model. This will run it at 40 tokens/s based on rough calculation. Q4 quant. 37b active parameters. 5x higher price for 10x higher performance.
- hinkley 2y ago
  Also you don't have to deal with Windows. Which people who do not understand Apple are very skilled at not noticing. If you've ever used git, svn, or an IDE side by side on corporate Windows versus Apple I don't know why you would ever go back.
  - hatthew 2y ago
    Is there a reason one couldn't use linux?
  - bigyabai 2y ago
    The PC doesn't have to run Windows either. Strictly speaking, professional applications see MacOS support as an Apple-sanctioned detriment. > If you've ever used git, svn, or an IDE side by side I still reach for Windows, even though it's a dogshit OS. I would rather use WSL to write and deploy a single app, as opposed to doing my work in a Linux VM or (god forbid) writing and debugging multiple versions just to support my development runtime. If I'm going to use an ad-encumbered commercial service-slop OS, I might as well pick the one that doesn't actively block my work.
  - brailsafe 2y ago
    It's also just clearly a powerful and interesting tinkering project, which there are valid arguments for, but this can just chill out on your desk as an elegant general productivity machine. What it wouldn't do that the tinkering project could do is be upgraded, act as a powerful gaming pc, or cause migraines from constant fan noise. The custom build would work great though, and even moreso in a server room and as well continues to reveal by comparison how excessively Apple prices it's components.

by hedora 2y ago

flakiness 2y ago

The low energy use can be a game changer if you live in a crappy apartment with limited power capacity. I gave up my big GPU box dream because of that.

aurareturn 2y ago

between 4.25 to 3.5 TPS (tokens per second) on the Q4 671b full model. 3.5 - 4.25 tokens/s. You're torturing yourself. Especially with a reasoning model. This will run it at 40 tokens/s based on rough calculation. Q4 quant. 37b active parameters. 5x higher price for 10x higher performance.

hinkley 2y ago
Also you don't have to deal with Windows. Which people who do not understand Apple are very skilled at not noticing. If you've ever used git, svn, or an IDE side by side on corporate Windows versus Apple I don't know why you would ever go back.
- hatthew 2y ago
  Is there a reason one couldn't use linux?
- bigyabai 2y ago
  The PC doesn't have to run Windows either. Strictly speaking, professional applications see MacOS support as an Apple-sanctioned detriment. > If you've ever used git, svn, or an IDE side by side I still reach for Windows, even though it's a dogshit OS. I would rather use WSL to write and deploy a single app, as opposed to doing my work in a Linux VM or (god forbid) writing and debugging multiple versions just to support my development runtime. If I'm going to use an ad-encumbered commercial service-slop OS, I might as well pick the one that doesn't actively block my work.
- brailsafe 2y ago
  It's also just clearly a powerful and interesting tinkering project, which there are valid arguments for, but this can just chill out on your desk as an elegant general productivity machine. What it wouldn't do that the tinkering project could do is be upgraded, act as a powerful gaming pc, or cause migraines from constant fan noise. The custom build would work great though, and even moreso in a server room and as well continues to reveal by comparison how excessively Apple prices it's components.