6 ms·

The only reason I can see Apple do this is if it enables them to sell entry level devices with vastly more ram than the competition can afford. Say entry level

by spockz 5mo ago

The only reason I can see Apple do this is if it enables them to sell entry level devices with vastly more ram than the competition can afford. Say entry level MacBook Pro with 256GiB ram to facilitate running frontier level local models. If that is an edge they want to have.

sokoloff 5mo ago
In what world would 256GB of RAM be “entry level”?
- m4x 5mo ago
  A world where Apple has invested in their own fabs, so they can sell devices with drastically more RAM than their competition at entry-level prices.
  - sokoloff 5mo ago
    Why would they sell a device with 256GB of RAM as the lowest-spec device rather than making 8 32GB or 16 16GB machines as their entry-level? Apple’s not exactly famous for their low pricing on spec upgrades nor competing based on being the price leader…
    - ethbr1 5mo ago
      If 256GB of RAM enables them to run on-device AI models that (for reasons) are a key feature differentiator? Personally, I think there's no way memory heavy inference moves on-device (vs cloud) due to the economics, but it's not impossible technology + platforms go that way for currently unforeseeable reasons.
      - sokoloff 5mo ago
        Right. I’m not arguing that Apple wouldn’t offer a 256GB model if they could make money doing it; I’m puzzled as to why they wouldn’t offer several lower-spec models as the entry-level into and then progressive upgrades within that line, since only some people need that 256GB feature differentiator of running frontier-level models on their MacBook Pro.
        ethbr1 5mo ago
        And I'm saying, if 256GB of memory is a requirement for running customer-expected local models (and local models are preferred for some reason).
      - everforward 5mo ago
        I think there’s a realistic chance consumer inference moves on-device. I think it really depends on marketing. My non-tech friends and family would probably be served perfectly fine by local models today, if they had a working web search tool. Their queries are often “soft” and don’t have an exact answer. My mom and aunt used it to pick a hairstyle, my mom used it to get an image of what a room would look like with particular drapes in it, etc. Stuff I think mid-sized local models like Gemma or smaller Qwens could do without issue. They just don’t have a device that will run them. Businesses won’t move. They need a huge context so they can stuff a bunch of Confluence pages in it and 300 tools and it needs to read an entire codebase and yada yada. The hardware depreciation and electricity will probably make it a net zero or even cost more than paying for API access.
        ethbr1 5mo ago
        The economic argument in favor of cloud inference: higher utilization is always going to have a ROI for inference hardware. But maybe that hardware becomes so commoditized that it's not difficult to obtain / stuff in a box.
        everforward 5mo ago
        My argument is predicated on the assumption that mainstream hardware manufacturers will copy the way Apple and Framework have made system memory usable for inference. In that world, a) we are already at or close to having enough memory in local devices to do inference locally, and b) that memory isn't inference-specific and can be utilized for other things. Most devices come with enough memory to do some level of inference, and some come with plenty (eg a gaming desktop probably has 32GB+ of RAM in it). You aren't going to run Kimi on it, but I think the reality for a lot of consumer inference is that it doesn't need to be. It's going to be a lot of things that are soft, and easily answered by a search API, so the LLM really just needs to be able to skim and summarize. Going a step further, we may even see some kind of hybrid approach where a local OpenRouter kind of thing decides whether the task is soft enough to do locally with models that fit in RAM or if it needs to be farmed out to a PaaS provider.
      - trollbridge 5mo ago
        Think past on-device inference... imagine what on-device training could do. And that would need a lot of RAM.
  - hylaride 5mo ago
    I'm no business expert and Apple is of course in a unique position, but owning your own fabs has rarely worked out long term. They require eye watering amounts of CAPEX that needs to be amortized over a timeframe that's longer than apple's products. Today's bleeding edge fabs become tomorrow's "cheap" fabs that pump out chips that don't need to be bleeding edge for the components that go into everyday products like microwaves, cars, etc. One of the reasons Intel fell behind is that they couldn't give access to their competitors for business reasons, and therefore could never scale as high as TSMC could. There are many other reasons, but accounting is a huge one. Unless there is a huge ROI or something else we don't otherwise know, I don't see Apple adding such expensive deprecating assets onto their books as chip fabs.
    - FarmerPotato 5mo ago
      A more realistic scenario that surely must have been considered is this: They make a strategic investment in the DRAM maker. It's the same idea as buying an equity stake or entering a JV with a flat-panel maker. Or deploying cash to provide the latest EUV tooling to TSMC. Large famous-brand computer maker has no expertise in running a fab. I could punch holes in this idea right away. But it seems better than "tilting up your own fab".
  - mrandish 5mo ago
    The point of my GP post was that due to being one of the world's biggest and longest-term buyers, Apple is already paying very close to actual manufacturing costs + amortized capex because RAM is an undifferentiated commodity. Owning the factory themselves doesn't reduce the actual manufacturing cost + amortized capex that Apple would have to pay their own factory. Apple is already buying RAM at the lowest possible margins. It's similar math to deciding whether to spend your own cash or get a loan. If the loan's interest rate is low enough, it's better take the loan and put your cash to work where it can return a higher margin. And at the incredibly low margins Apple pays for RAM, keeping that cash in long-term investments will actually earn more money than putting it into building RAM factories. If Apple could go back in time 3.5 years and decide to build their own factory, that would put them in a great position today. But deciding to do it now won't increase their supply 3.5 years from now more than just increasing their long-term orders with existing suppliers. Those suppliers will start building new factories based on Apple's increased orders and they'll do it faster and cheaper than Apple can because they don't have to build some factories in the U.S. for political reasons or worry as much about environmental regulation, permitting and ensuring Apple employees in Penang get benefits similar to employees in Cupertino.
    - m4x 5mo ago
      Isn't one of the points of the article that memory manufacturers leave demand unmet for their own financial safety? In which case, nobody (including Apple) is paying close to manufacturing costs. There isn't enough memory to go around and prices are extremely inflated. You're talking about the "best" things Apple could do with their money, in terms of investment returns, but I think that misses the point that Apple literally can't buy enough memory at any price.
      - mrandish 5mo ago
        > that misses the point that Apple literally can't buy enough memory at any price. They can't buy enough today. I think I already explained this as well as I can in my second paragraph above. You seem to not be appreciating that every decision in this business has a 3.5+ year latency. And 3.5 years ago when Apple made the decisions they're living with now, Micron was selling their furniture (euphemistically speaking). Sinking billions into building RAM fabs would not only have gotten you laughed out of the boardroom, Apple shareholders would have rioted (for all the logical reasons I laid out in my prior two posts). The only scenario where the prudently balanced decision Apple made to not vertically integrate RAM manufacturing looks bad is the very unlikely scenario that actually happened this one time - but that 'bad look' is only temporary because the situation will probably look different 12 months from now (which is less than a third-of-a-fab-decision away)). Back when I first became a strategist for a public F500 tech company whose products you probably use often, a very wise man took me to dinner and counseled me to never post mortem my decisions on the outcomes, but only on the decision process. In other words, knowing what I knew when I made the decision, did I choose the option most statistically likely to get the desired result - regardless if it worked or failed that time? In highly uncertain games, some bets don't pay off. In my work, if 60% of my decisions came out positively, I'd have been the greatest of all time (they weren't that quite that high). If strategists don't think this way we end up leading our companies to doom by "chasing black swans" (aka unlikely outlier outcomes). > memory manufacturers leave demand unmet That's not their plan or desire - it's just the inevitable result of trying to predict both future demand and future capacities of unbuilt fabs on new nodes (every new node is an adventure). Both are rough estimates with bell-curve probabilities and wide error bars. You might imagine the Chief RAM Strategist drags the center of the bell curve so the big bump in the middle is right over their best guess of what the fab yields and market demand will actually be three years from now - perfectly slicing the odds in the probabilistic middle. But they don't. Instead, being a smart, experienced strategist, they slide the bell curve a little bit to the low-side - because very profitably selling everything you can possibly make and regretting the 5 or 10% of demand you left unmet is a far, FAR better (and more survivable) problem to have than sitting on 5 or 10% unsold excess capacity from expensive fab space you paid to build but can't monetize. That extra capacity will often force you to take lower margins on ALL your capacity to ensure you sell through the extra 10%. This is how you speed run becoming an ex-strategist, sometimes of an ex-company. In short, being wrong on the low side is the best kind of wrong you can be. But being wrong on the high side is the worst kind of wrong, and it's not linear. Every extra percent you go over costs significantly more than the last. If they do their job perfectly, it looks to consumers like "manufacturers leave demand unmet." If they do their job less than perfectly but pretty well, it looks like "RAM's too damn expensive and the idiots created a shortage - what were those morons thinking?" If they do their job catastrophically it looks to consumers like "Woohoo, RAMs dirt cheap. Look at me! I've got so many sticks I'm bathing in them like Scrooge McDuck!"
- giwook 5mo ago
  That is the timeline I want to live in. Probably.
- ceejayoz 5mo ago
  In the same world 8GB used to be unfathomably huge not that long ago? Hell, an 8GB hard drive was unfathomable when I was a kid in the 90s. I remember getting a 30 megabyte drive for our Mac LC.
  - mmmlinux 5mo ago
    And that Mac LC only had a max of 10MB of ram.
  - trollbridge 5mo ago
    The first computer I ever used had 512K, which was a great deal back then, and by the time I was old enough to learn to type it had 10MB disk too. My childhood best friend and neighbour had the same kind of computer except they only had something like 384K of memory and I tried to convince them their computer was broken when it didn't count up all the way.
    - mrandish 5mo ago
      > The first computer I ever used had 512K Mine had 4K of RAM and an 800 KHz CPU - and I was living in the future, man. No way I could use that much memory. After all I had to type in whatever program I wanted to run every time I turned it on. Then I got a manual audio cassette recorder and thought "Woah, I don't think it gets better than this!"
      - isr 5mo ago
        My first (vic20) had 3.5KB of ram. When I added a whopping 8KB memory expansion cartridge, I genuinely wondered if I could possibly get any of my BASIC programs to use it all. And when I upgraded from tapes to the 1541 floppy drive (5.25" floppies), it was like "thats it - I have INFINITE possibilities now" Simpler times :)
        mrandish 5mo ago
        > the 1541 floppy drive Ah, the legendary 1541. We used to joke it was the only floppy drive on any 8-bit micro that was actually slower than some dial-up modems. It wasn't actually slower, but it was really slow compared to most other 8-bit computer's drives (unless you added after-market accelerators).
- trollbridge 5mo ago
  The same world where a MacBook Pro G4 12" came base level with 256MB of RAM. The same world where a classic Mac came with 128KB of RAM. We might someday live in a world where entry-level is 256TB of RAM.

by spockz 5mo ago

sokoloff 5mo ago

In what world would 256GB of RAM be “entry level”?

m4x 5mo ago
A world where Apple has invested in their own fabs, so they can sell devices with drastically more RAM than their competition at entry-level prices.
- sokoloff 5mo ago
  Why would they sell a device with 256GB of RAM as the lowest-spec device rather than making 8 32GB or 16 16GB machines as their entry-level? Apple’s not exactly famous for their low pricing on spec upgrades nor competing based on being the price leader…
  - ethbr1 5mo ago
    If 256GB of RAM enables them to run on-device AI models that (for reasons) are a key feature differentiator? Personally, I think there's no way memory heavy inference moves on-device (vs cloud) due to the economics, but it's not impossible technology + platforms go that way for currently unforeseeable reasons.
    - sokoloff 5mo ago
      Right. I’m not arguing that Apple wouldn’t offer a 256GB model if they could make money doing it; I’m puzzled as to why they wouldn’t offer several lower-spec models as the entry-level into and then progressive upgrades within that line, since only some people need that 256GB feature differentiator of running frontier-level models on their MacBook Pro.
      - ethbr1 5mo ago
        And I'm saying, if 256GB of memory is a requirement for running customer-expected local models (and local models are preferred for some reason).
    - everforward 5mo ago
      I think there’s a realistic chance consumer inference moves on-device. I think it really depends on marketing. My non-tech friends and family would probably be served perfectly fine by local models today, if they had a working web search tool. Their queries are often “soft” and don’t have an exact answer. My mom and aunt used it to pick a hairstyle, my mom used it to get an image of what a room would look like with particular drapes in it, etc. Stuff I think mid-sized local models like Gemma or smaller Qwens could do without issue. They just don’t have a device that will run them. Businesses won’t move. They need a huge context so they can stuff a bunch of Confluence pages in it and 300 tools and it needs to read an entire codebase and yada yada. The hardware depreciation and electricity will probably make it a net zero or even cost more than paying for API access.
      - ethbr1 5mo ago
        The economic argument in favor of cloud inference: higher utilization is always going to have a ROI for inference hardware. But maybe that hardware becomes so commoditized that it's not difficult to obtain / stuff in a box.
        everforward 5mo ago
        My argument is predicated on the assumption that mainstream hardware manufacturers will copy the way Apple and Framework have made system memory usable for inference. In that world, a) we are already at or close to having enough memory in local devices to do inference locally, and b) that memory isn't inference-specific and can be utilized for other things. Most devices come with enough memory to do some level of inference, and some come with plenty (eg a gaming desktop probably has 32GB+ of RAM in it). You aren't going to run Kimi on it, but I think the reality for a lot of consumer inference is that it doesn't need to be. It's going to be a lot of things that are soft, and easily answered by a search API, so the LLM really just needs to be able to skim and summarize. Going a step further, we may even see some kind of hybrid approach where a local OpenRouter kind of thing decides whether the task is soft enough to do locally with models that fit in RAM or if it needs to be farmed out to a PaaS provider.
    - trollbridge 5mo ago
      Think past on-device inference... imagine what on-device training could do. And that would need a lot of RAM.
- hylaride 5mo ago
  I'm no business expert and Apple is of course in a unique position, but owning your own fabs has rarely worked out long term. They require eye watering amounts of CAPEX that needs to be amortized over a timeframe that's longer than apple's products. Today's bleeding edge fabs become tomorrow's "cheap" fabs that pump out chips that don't need to be bleeding edge for the components that go into everyday products like microwaves, cars, etc. One of the reasons Intel fell behind is that they couldn't give access to their competitors for business reasons, and therefore could never scale as high as TSMC could. There are many other reasons, but accounting is a huge one. Unless there is a huge ROI or something else we don't otherwise know, I don't see Apple adding such expensive deprecating assets onto their books as chip fabs.
  - FarmerPotato 5mo ago
    A more realistic scenario that surely must have been considered is this: They make a strategic investment in the DRAM maker. It's the same idea as buying an equity stake or entering a JV with a flat-panel maker. Or deploying cash to provide the latest EUV tooling to TSMC. Large famous-brand computer maker has no expertise in running a fab. I could punch holes in this idea right away. But it seems better than "tilting up your own fab".
- mrandish 5mo ago
  The point of my GP post was that due to being one of the world's biggest and longest-term buyers, Apple is already paying very close to actual manufacturing costs + amortized capex because RAM is an undifferentiated commodity. Owning the factory themselves doesn't reduce the actual manufacturing cost + amortized capex that Apple would have to pay their own factory. Apple is already buying RAM at the lowest possible margins. It's similar math to deciding whether to spend your own cash or get a loan. If the loan's interest rate is low enough, it's better take the loan and put your cash to work where it can return a higher margin. And at the incredibly low margins Apple pays for RAM, keeping that cash in long-term investments will actually earn more money than putting it into building RAM factories. If Apple could go back in time 3.5 years and decide to build their own factory, that would put them in a great position today. But deciding to do it now won't increase their supply 3.5 years from now more than just increasing their long-term orders with existing suppliers. Those suppliers will start building new factories based on Apple's increased orders and they'll do it faster and cheaper than Apple can because they don't have to build some factories in the U.S. for political reasons or worry as much about environmental regulation, permitting and ensuring Apple employees in Penang get benefits similar to employees in Cupertino.
  - m4x 5mo ago
    Isn't one of the points of the article that memory manufacturers leave demand unmet for their own financial safety? In which case, nobody (including Apple) is paying close to manufacturing costs. There isn't enough memory to go around and prices are extremely inflated. You're talking about the "best" things Apple could do with their money, in terms of investment returns, but I think that misses the point that Apple literally can't buy enough memory at any price.
    - mrandish 5mo ago
      > that misses the point that Apple literally can't buy enough memory at any price. They can't buy enough today. I think I already explained this as well as I can in my second paragraph above. You seem to not be appreciating that every decision in this business has a 3.5+ year latency. And 3.5 years ago when Apple made the decisions they're living with now, Micron was selling their furniture (euphemistically speaking). Sinking billions into building RAM fabs would not only have gotten you laughed out of the boardroom, Apple shareholders would have rioted (for all the logical reasons I laid out in my prior two posts). The only scenario where the prudently balanced decision Apple made to not vertically integrate RAM manufacturing looks bad is the very unlikely scenario that actually happened this one time - but that 'bad look' is only temporary because the situation will probably look different 12 months from now (which is less than a third-of-a-fab-decision away)). Back when I first became a strategist for a public F500 tech company whose products you probably use often, a very wise man took me to dinner and counseled me to never post mortem my decisions on the outcomes, but only on the decision process. In other words, knowing what I knew when I made the decision, did I choose the option most statistically likely to get the desired result - regardless if it worked or failed that time? In highly uncertain games, some bets don't pay off. In my work, if 60% of my decisions came out positively, I'd have been the greatest of all time (they weren't that quite that high). If strategists don't think this way we end up leading our companies to doom by "chasing black swans" (aka unlikely outlier outcomes). > memory manufacturers leave demand unmet That's not their plan or desire - it's just the inevitable result of trying to predict both future demand and future capacities of unbuilt fabs on new nodes (every new node is an adventure). Both are rough estimates with bell-curve probabilities and wide error bars. You might imagine the Chief RAM Strategist drags the center of the bell curve so the big bump in the middle is right over their best guess of what the fab yields and market demand will actually be three years from now - perfectly slicing the odds in the probabilistic middle. But they don't. Instead, being a smart, experienced strategist, they slide the bell curve a little bit to the low-side - because very profitably selling everything you can possibly make and regretting the 5 or 10% of demand you left unmet is a far, FAR better (and more survivable) problem to have than sitting on 5 or 10% unsold excess capacity from expensive fab space you paid to build but can't monetize. That extra capacity will often force you to take lower margins on ALL your capacity to ensure you sell through the extra 10%. This is how you speed run becoming an ex-strategist, sometimes of an ex-company. In short, being wrong on the low side is the best kind of wrong you can be. But being wrong on the high side is the worst kind of wrong, and it's not linear. Every extra percent you go over costs significantly more than the last. If they do their job perfectly, it looks to consumers like "manufacturers leave demand unmet." If they do their job less than perfectly but pretty well, it looks like "RAM's too damn expensive and the idiots created a shortage - what were those morons thinking?" If they do their job catastrophically it looks to consumers like "Woohoo, RAMs dirt cheap. Look at me! I've got so many sticks I'm bathing in them like Scrooge McDuck!"
giwook 5mo ago
That is the timeline I want to live in. Probably.
ceejayoz 5mo ago
In the same world 8GB used to be unfathomably huge not that long ago? Hell, an 8GB hard drive was unfathomable when I was a kid in the 90s. I remember getting a 30 megabyte drive for our Mac LC.
- mmmlinux 5mo ago
  And that Mac LC only had a max of 10MB of ram.
- trollbridge 5mo ago
  The first computer I ever used had 512K, which was a great deal back then, and by the time I was old enough to learn to type it had 10MB disk too. My childhood best friend and neighbour had the same kind of computer except they only had something like 384K of memory and I tried to convince them their computer was broken when it didn't count up all the way.
  - mrandish 5mo ago
    > The first computer I ever used had 512K Mine had 4K of RAM and an 800 KHz CPU - and I was living in the future, man. No way I could use that much memory. After all I had to type in whatever program I wanted to run every time I turned it on. Then I got a manual audio cassette recorder and thought "Woah, I don't think it gets better than this!"
    - isr 5mo ago
      My first (vic20) had 3.5KB of ram. When I added a whopping 8KB memory expansion cartridge, I genuinely wondered if I could possibly get any of my BASIC programs to use it all. And when I upgraded from tapes to the 1541 floppy drive (5.25" floppies), it was like "thats it - I have INFINITE possibilities now" Simpler times :)
      - mrandish 5mo ago
        > the 1541 floppy drive Ah, the legendary 1541. We used to joke it was the only floppy drive on any 8-bit micro that was actually slower than some dial-up modems. It wasn't actually slower, but it was really slow compared to most other 8-bit computer's drives (unless you added after-market accelerators).
trollbridge 5mo ago
The same world where a MacBook Pro G4 12" came base level with 256MB of RAM. The same world where a classic Mac came with 128KB of RAM. We might someday live in a world where entry-level is 256TB of RAM.