3 ms·

A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now a

by dabinat 3mo ago

A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.

Tenoke 3mo ago
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks. Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
- mathisfun123 3mo ago
  > Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in. lol this is so wrong it's funny - equities go up in price, commodity goods go down in price. the two markets are literally diametrically opposed.
  - matt_heimer 3mo ago
    I'd have a better portfolio right now if I invested in RAM instead of equities.
    - saltcured 3mo ago
      RAM is more like agricultural products (with short shelf-life) than commodities like fossil fuels, mineral ores, etc. You can manage an inventory or speculate on production, but you cannot really hold a "portfolio" of it in any sensible way. So, you should get into RAM futures if you believe this is more than a transient arbitrage sort of situation. All extant RAM will become obsolete as the demand shifts to newer, fancier versions.
      - Tenoke 3mo ago
        >but you cannot really hold a "portfolio" of it in any sensible way. xAI effectively did and lucked out to cover their losses and more with it.
      - fc417fc802 3mo ago
        > All extant RAM will become obsolete Yes but on what timescale? Replacing the > 5 year old sticks in my laptop would currently cost well over half what the machine ran me when it was brand new.
- sarjann 3mo ago
  No, equities naturally trend up with economic growth. RAM is up because of a supply shock, as new capacity comes on prices will drop, it’s a commodity.
  - Tenoke 3mo ago
    Yes, in the very long term, but in the medium term where the listed capacities even matter we are not close to that. Really, the cost per gb of vram on flagships hasnt went down since 1080ti and thats not accouting for the recent increases which will likely last for years.
    - whaleofatw2022 3mo ago
      There are some minor exceptions but agreed otherwise. Manufacturers have been excessively using VRAM limitations as a price point distinguisher.
  - hahahaa 3mo ago
    If it flys, floats or FLOPs it is better to rent.
  - dofm 3mo ago
    But it could get worse for two years yet.
- hypercube33 3mo ago
  Asus rog 13 128gb is still sub $3k last I checked and ticks the boxes. If you get tired of AI it's a kick ass tablet except the weight and can run AAA games for now decently enough.
  - fodkodrasz 3mo ago
    Is that a notebook? As the thermals don't really compare for a notebook and these high performance minipcs.
- dabinat 3mo ago
  > Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in. I said less insane, not sane. If prices go down 20% but are still up 150% over a few years ago, that would still be an improvement from now. Or it could work the other way: if new hardware comes out in 2027 such that the tokens/$ ratio works out better, that would also be less insane.
deleted 3mo ago
[deleted]
hadlock 3mo ago
Apple rumor mill is suggesting that we may see M5 Mac Studio announced in September at that event. Apple just increased their pricing across the board, so I'm not holding my breath that they will be reasonably priced. They also cancelled development of their M6 in favor of the future M7 which I suspect will be a major, AI-focused upgrade compared to the M4->M5 upgrade
- dofm 3mo ago
  They cancelled the higher-end M6 models (including the Pro, this time). There will be a basic M6.
Bombthecat 3mo ago
Apple already announced that will increase prices.. doubt that Mac studio will stay cheap
kristianp 3mo ago
The Mac Studio M3 Ultra with 96GB of RAM with 1Tb SSD costs $6799, but has triple the memory bandwidth [1]. It's almost twice the price of Strix Halo and when the M5 Ultras come out it will be interesting to see how much they go for. I expect a lot of demand for them! There was a post recently, that showed the DGX spark is twice as fast as the M3 Ultra at prompt processing, but half as fast at output tokens [2]. They used gps-oss-120 for that test with a small context. [1] https://gpuquicklist.com/apus?models=GB10%20Grace%20Blackwell,Mac%20Studio%20M3%20Ultra%2096GB,AI%20Max+%20395%20128GB https://gpuquicklist.com/apus?models=GB10%20Grace%20Blackwel... [2] https://aimultiple.com/dgx-spark-alternatives https://aimultiple.com/dgx-spark-alternatives , https://news.ycombinator.com/item?id=48732679 https://news.ycombinator.com/item?id=48732679
- cthalupa 3mo ago
  I’ve got a 128gb m5 max mbp and two sparks. For my real-world use cases, a single spark running DS4 Flash will have fully responded by the time my Mac has even started generating tokens. I figured I’d have more generation heavy work when I also bought the Mac, but it has done very little work running LLMs since I got the first Spark. I mostly run them clustered for DS4 and am quite happy with the performance, and the cost isn’t that much more for two than the MBP while giving me double the unified memory. I’ll probably pick up a third to run multiple smaller models. I don’t understand why people would buy a halo over a spark at comparable prices, particularly because if you want to cluster, the cx7 be beats the shit out of them when it comes to latency and throughput
sixothree 3mo ago
MacBook Pro can come configured with 128GB M5 Max.

by dabinat 3mo ago

Tenoke 3mo ago

I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks. Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.

mathisfun123 3mo ago
> Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in. lol this is so wrong it's funny - equities go up in price, commodity goods go down in price. the two markets are literally diametrically opposed.
- matt_heimer 3mo ago
  I'd have a better portfolio right now if I invested in RAM instead of equities.
  - saltcured 3mo ago
    RAM is more like agricultural products (with short shelf-life) than commodities like fossil fuels, mineral ores, etc. You can manage an inventory or speculate on production, but you cannot really hold a "portfolio" of it in any sensible way. So, you should get into RAM futures if you believe this is more than a transient arbitrage sort of situation. All extant RAM will become obsolete as the demand shifts to newer, fancier versions.
    - Tenoke 3mo ago
      >but you cannot really hold a "portfolio" of it in any sensible way. xAI effectively did and lucked out to cover their losses and more with it.
    - fc417fc802 3mo ago
      > All extant RAM will become obsolete Yes but on what timescale? Replacing the > 5 year old sticks in my laptop would currently cost well over half what the machine ran me when it was brand new.
sarjann 3mo ago
No, equities naturally trend up with economic growth. RAM is up because of a supply shock, as new capacity comes on prices will drop, it’s a commodity.
- Tenoke 3mo ago
  Yes, in the very long term, but in the medium term where the listed capacities even matter we are not close to that. Really, the cost per gb of vram on flagships hasnt went down since 1080ti and thats not accouting for the recent increases which will likely last for years.
  - whaleofatw2022 3mo ago
    There are some minor exceptions but agreed otherwise. Manufacturers have been excessively using VRAM limitations as a price point distinguisher.
- hahahaa 3mo ago
  If it flys, floats or FLOPs it is better to rent.
- dofm 3mo ago
  But it could get worse for two years yet.
hypercube33 3mo ago
Asus rog 13 128gb is still sub $3k last I checked and ticks the boxes. If you get tired of AI it's a kick ass tablet except the weight and can run AAA games for now decently enough.
- fodkodrasz 3mo ago
  Is that a notebook? As the thermals don't really compare for a notebook and these high performance minipcs.
dabinat 3mo ago
> Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in. I said less insane, not sane. If prices go down 20% but are still up 150% over a few years ago, that would still be an improvement from now. Or it could work the other way: if new hardware comes out in 2027 such that the tokens/$ ratio works out better, that would also be less insane.

deleted 3mo ago

[deleted]

hadlock 3mo ago

Apple rumor mill is suggesting that we may see M5 Mac Studio announced in September at that event. Apple just increased their pricing across the board, so I'm not holding my breath that they will be reasonably priced. They also cancelled development of their M6 in favor of the future M7 which I suspect will be a major, AI-focused upgrade compared to the M4->M5 upgrade

dofm 3mo ago
They cancelled the higher-end M6 models (including the Pro, this time). There will be a basic M6.

Bombthecat 3mo ago

Apple already announced that will increase prices.. doubt that Mac studio will stay cheap

kristianp 3mo ago

The Mac Studio M3 Ultra with 96GB of RAM with 1Tb SSD costs $6799, but has triple the memory bandwidth [1]. It's almost twice the price of Strix Halo and when the M5 Ultras come out it will be interesting to see how much they go for. I expect a lot of demand for them! There was a post recently, that showed the DGX spark is twice as fast as the M3 Ultra at prompt processing, but half as fast at output tokens [2]. They used gps-oss-120 for that test with a small context. [1] https://gpuquicklist.com/apus?models=GB10%20Grace%20Blackwell,Mac%20Studio%20M3%20Ultra%2096GB,AI%20Max+%20395%20128GB https://gpuquicklist.com/apus?models=GB10%20Grace%20Blackwel... [2] https://aimultiple.com/dgx-spark-alternatives https://aimultiple.com/dgx-spark-alternatives , https://news.ycombinator.com/item?id=48732679 https://news.ycombinator.com/item?id=48732679

cthalupa 3mo ago
I’ve got a 128gb m5 max mbp and two sparks. For my real-world use cases, a single spark running DS4 Flash will have fully responded by the time my Mac has even started generating tokens. I figured I’d have more generation heavy work when I also bought the Mac, but it has done very little work running LLMs since I got the first Spark. I mostly run them clustered for DS4 and am quite happy with the performance, and the cost isn’t that much more for two than the MBP while giving me double the unified memory. I’ll probably pick up a third to run multiple smaller models. I don’t understand why people would buy a halo over a spark at comparable prices, particularly because if you want to cluster, the cx7 be beats the shit out of them when it comes to latency and throughput

sixothree 3mo ago

MacBook Pro can come configured with 128GB M5 Max.