5 ms·

No, they aren't. They're desperate. We've realized for some time that not only are language model technologies going to dominate our future, but that they requ

No, they aren't. They're desperate. We've realized for some time that not only are language model technologies going to dominate our future, but that they require massive compute to run. We've also been talking about Apple - their silence, but also their hardware capabilities to run LLMs on consumer devices. Every other player (OpenAI, Facebook, Microsoft, etc) is faced with a billion dollar datacenter investment that still provides a worse experience (latency & privacy). Apple has a huge advantage because they get to pay nothing for a superior compute infrastructure that comes with a better privacy and latency story. As soon as they release something, it's going to be good, and it's going to be free, because it runs fast on consumer hardware and consumer electricity. A billion-dollar datacenter that's upgraded every N years can't compete with that. Microsoft knows this, and they're trying to get out in front of whatever Apple comes up with. That's why they're telling manufacturers to install these "NPU" things - it's a direct equivalent to the iPhone/iPad/Mac compute capabilities. So no, Microsoft probably knew this would be the reaction. But it's better to get dragged through the mud for lack of trust than for being left behind the cutting edge. In the former case you're just a slimeball; in the latter case you're a moronic slimeball.

deleted 2y ago
[deleted]
causal 2y ago
Agree on the Apple strengths, disagree that MS is scared. PCs just aren't where they make most of their money anymore. Enterprise is extremely lucrative for them, and their headstart in the AI space may be just what they need to close the gap with AWS.
- CuriouslyC 2y ago
  Azure is bad. Google is the one that's going to leverage AI to catch up with AWS, GCP is actually good and they've already partnered with Huggingface.
  - bruce511 2y ago
    Features aside, I simply don't trust Google to keep GCP running in the long term. Their propensity to kill projects (even ones which are successful), plus their gratuitous price hikes (aka Google Maps) has made me Goggle negative. Sure, they might run GCP forever. Sure they may offer some sort of customer service at some point. Sure they may decide to keep pricing at sane levels. The key work in all that is "might" - and that requires my _trust_. And for better, or worse, my trust simply isn't there.
    - CuriouslyC 2y ago
      I 100% feel you on the google trust. I think it'll end up being central to a lot of their other efforts so I'm expecting it to stick around, but hybrid cloud solutions are just a smart bet anyhow.
    - magicalhippo 2y ago
      For us Google wasn't even on the table, due to what you mentioned. It was AWS or Azure. AWS didn't have a data center in our country and our customers are required by law to keep their data in the country, so easy win for Azure.
  - yesdocs 2y ago
    Why is azure ‘bad’? Is it because you dislike Microsoft? In my experience Azure is really good and getting better. Even if AWS is more popular, Azure is gaining traction, is more intuitive and easier to use, and in my experience: cheaper
    - CuriouslyC 2y ago
      they've had long running security issues and performance edge cases that make relying on them a bit of a time bomb.
    - Hikikomori 2y ago
      Had to do multi cloud with Azure, worked on network infrastructure, vnet, peering, firewall, vpn gateway, etc. Documentation was almost worthless, leaving a lot of room for interpretation. Encountered and filed more bugs trying to get it up and running than I did for years of using Aws. Support was mediocre. Non existent ipv6 support. This was a few years ago and in comparison to Aws.
- Zhyl 2y ago
  What would you do if you had several billion devices that you had root access to, used directly by humans and isn't making you any money in sales these days? The 2024 answer is to use it to capture data to train AI. Microsoft could lose 9/10 of their market share (which they won't) and still have millions of devices harvesting training data for them.
  - lalalandland 2y ago
    If GDPR is at all questionable, Microsoft is not able to push this to EU countries. The risk for a business to get in GDPR trouble is to high.
jerf 2y ago
"We've realized for some time that not only are language model technologies going to dominate our future" Close, but not quite. What we realize right now is that AI has pumped, pumped, pumped a whole lot of stock valuations sky high, and the first faint glimmers of the market asking that some actual money be delivered in proportion to the sky high valuations are starting to appear. (Note carefully the "in proportion to"; it is not enough to make a bit of cash. These valuations are enormous.) The stocks so pumped need results more-or-less right now, since propagating anything they have right now out into the market to actually gather revenue is going to take some time. I expect a couple of other Hail Mary plays like this to show up too. And again, let me underline, emphasize, and highlight, this isn't about whether AI can make "any" money. It's about whether it can justify those staggeringly high stock valuations in, say, the next year or so. This is a fairly uphill battle. Even if you are a True Believer than AI will be an economic revolution in literally the next 12 months or so, it's hard not to look at the market and come to the conclusion that right now every AI player is being valuated as the inevitable victor and the inevitable beneficiary of 100% (or more) of the AI gain, e.g., nVidia is being valuated as if there will never, ever be any competition for AI hardware despite all the incentives, each AI user valuated as if they are the inevitable winner in their respective domains, etc. They can't all win simultaneously like that, it's just impossible. (On the plus side, it's really just stock valuations. I don't expect an AI winter this time, the tools are much more valuable than they were last time. I mean, I would still very firmly say that what we have right now is grotesquely over hyped, but there's still quite a bit of "there" there. The previous AI hype cycles didn't have anywhere near as much "there" there.)
- AlexandrB 2y ago
  The costs are also largely being ignored at the moment. Both the obvious marginal costs (electricity, hardware for compute, dev time) and the hidden costs (flood of spam and misinformation).
  - Yoric 2y ago
    Also, the environmental cost. It's hidden, but won't be forever.
  - 2y ago
pjkundert 2y ago
Suddenly, distributed private inference, compute, storage and bandwidth mediated by Holochain at global scale will become … valuable.
bee_rider 2y ago
It seems like a somewhat far fetched explanation, but then I can’t think of a better one. They come up with this farcical application that nobody wants, to justify the hardware requirement, so that at least the hardware will be out there for when Apple shows them what the actual application is.
_uhtu 2y ago
> We've realized for some time that... language model technologies going to dominate our future Citation Needed.
- deleted 2y ago
  [deleted]
SideburnsOfDoom 2y ago
> language model technologies going to dominate our future I know many people who are _far from convinced_ of that. But that's not the relevant point: Microsoft, Google et all clearly are. And watching them collectively, desperately stampede after the mirage, at great cost, is certainly something.
oldpersonintx 2y ago
Apple is in terrible shape on software, AI is driving this home the only tier-one, network-effects app they control is iMessage search? someone else maps? someone else video? someone else audio? someone else and now they will go to someone else for AI what did they do in the meantime? VR that bellyflopped
- AlexandrB 2y ago
  Not sure why this is relevant. Apple's forte was always the hardware (and lately the greater focus on privacy). And you could easily say the same about Microsoft: search? someone else maps? someone else video? someone else audio? someone else Arguably Apple is at least better positioned than MS for some of these because people actually use Apple Maps and Apple Music. When was the last time someone whipped out Bing Maps?
- boxed 2y ago
  You forgot iOS. App APIs are also network effect things.
amelius 2y ago
Just wait until TSMC asks for 30% of Apple's revenue or they'll kick Apple out of their Fabstore. Fabrication is where the tech advantage lies, not hardware design IP. Anyone nowadays can program VHDL, see open source hardware. The challenge is actually fabricating it.
yyggvbb 2y ago
I think Microsoft software teams are already working behind their legal teams. Their business model is more about licensing than making products. Likely they’ll force some kind of regulatory situation that’s advantage to them. For example no AI capable OSs for government work, or something like that.
Miraste 2y ago
Apple doesn't have as much of an advantage as it appears. Their Neural Engine are fast and getting better, but modern machine learning models need another resource: fast RAM. Unfortunately for Apple, their whole product stack is segmented based on RAM. There are millions and millions of Apple devices with lightning-fast processors and 8GB or even 4GB RAM. These won't be able to do anything like Copilot+, unless they make calls to a datacenter like Microsoft is doing. Their existing local models are hit-or-miss already. FaceTime's audio transcription is laughable. Whether this is due to memory requirements, I don't know, but it doesn't bode well for further models.
luma 2y ago
> Apple has a huge advantage because they get to pay nothing for a superior compute infrastructure that comes with a better privacy and latency story. Right now, AI is a pay-$B-to-play and you need the massive datacenter compute for training and skilled researchers to develop. Apple silicon is neat, but let's not pretend like they have any credible story to tell about AI, and their recent deal with OpenAI suggests that they've been forced to punt. > As soon as they release something, it's going to be good, and it's going to be free, because it runs fast on consumer hardware and consumer electricity. Apple recently released their first public LLM, OpenELM. The largest released model is OpenELM-3B. The Instruct version of this model scores a 24.8 on the MMLU benchmark, which is a pick-one-of-four multiple choice test. Apple's biggest and best model somehow manages to get less than 25% on a 4 option multiple choice exam. For reference, MS released a similar size model Phi3-mini-4k which scored a 69 on that same exam. Yes, Apple has some nice local hardware to run models, but they don't have the models. They don't have the models because they don't have the people and they don't have the compute.
surgical_fire 2y ago
> As soon as they release something, it's going to be good Huh? How is that a given?
- luma 2y ago
  It is absolutely not true, they already released a model (OpenELM) and it compares extremely poorly versus similar size models from the likes of MS. This chart is correct and to scale: https://i.imgur.com/5mjlesU.png https://i.imgur.com/5mjlesU.png
deleted 2y ago
[deleted]
ca_tech 2y ago
The realized value of LLMs is going to be output from a data to AI pipeline. From raw data to actions or insights. This is Microsoft's play to control the entire process. They are attempting to abstract away all the "tooling" needed to manage and process data. It falls down because there is no discretion left to the consumer. Objectively, the computer is already processing all the data you are interacting with. Subjectively it is assumed its ephemeral. Computer forensics proves this wrong. So I think to refine my opening statement, the realized value of LLMs is going to be the output from a "curated" data to AI pipeline. Which Microsoft is not providing with this solution.
Wytwwww 2y ago
> Apple - their silence, but also their hardware capabilities to run LLMs > billion dollar datacenter investment that still provides a worse experience It's not at all clear to me that those very small local models would be competing directly with large LLMs run in datacentres for the same use cases. If that's going to be the case MS/etc. would also require significantly less compute power that's is currently expected, so Nvidia would be the real losers in that case (unless they shift to consumer HW). Also does Apple have such a huge advantage compared to the newer AMD/Intel/Google/Qualcomm(?) chips with NPUs? Yes Apple started investing sooner but all the phones/tablets and the overwhelming majority of laptops are crippled by extremely low memory capacity (in the context of running LLMs). So are they really that much ahead? e.g. what real advantages does the latest iPhone have compared to a Pixel 8 Pro (sure it's still a niche device, with a seemingly inferior an chip (but more memory) but based on what we know Google seems to be ahead in AI development and will make their solution available on all high end Android devices). > As soon as they release something, it's I doubt that's a given, their software has long ceased to be highly particularly exceptional and on the hardware side the only actually innovative product they have released for years has been a flop with a very unclear future.

deleted 2y ago

[deleted]

causal 2y ago

Agree on the Apple strengths, disagree that MS is scared. PCs just aren't where they make most of their money anymore. Enterprise is extremely lucrative for them, and their headstart in the AI space may be just what they need to close the gap with AWS.

CuriouslyC 2y ago
Azure is bad. Google is the one that's going to leverage AI to catch up with AWS, GCP is actually good and they've already partnered with Huggingface.
- bruce511 2y ago
  Features aside, I simply don't trust Google to keep GCP running in the long term. Their propensity to kill projects (even ones which are successful), plus their gratuitous price hikes (aka Google Maps) has made me Goggle negative. Sure, they might run GCP forever. Sure they may offer some sort of customer service at some point. Sure they may decide to keep pricing at sane levels. The key work in all that is "might" - and that requires my _trust_. And for better, or worse, my trust simply isn't there.
  - CuriouslyC 2y ago
    I 100% feel you on the google trust. I think it'll end up being central to a lot of their other efforts so I'm expecting it to stick around, but hybrid cloud solutions are just a smart bet anyhow.
  - magicalhippo 2y ago
    For us Google wasn't even on the table, due to what you mentioned. It was AWS or Azure. AWS didn't have a data center in our country and our customers are required by law to keep their data in the country, so easy win for Azure.
- yesdocs 2y ago
  Why is azure ‘bad’? Is it because you dislike Microsoft? In my experience Azure is really good and getting better. Even if AWS is more popular, Azure is gaining traction, is more intuitive and easier to use, and in my experience: cheaper
  - CuriouslyC 2y ago
    they've had long running security issues and performance edge cases that make relying on them a bit of a time bomb.
  - Hikikomori 2y ago
    Had to do multi cloud with Azure, worked on network infrastructure, vnet, peering, firewall, vpn gateway, etc. Documentation was almost worthless, leaving a lot of room for interpretation. Encountered and filed more bugs trying to get it up and running than I did for years of using Aws. Support was mediocre. Non existent ipv6 support. This was a few years ago and in comparison to Aws.
Zhyl 2y ago
What would you do if you had several billion devices that you had root access to, used directly by humans and isn't making you any money in sales these days? The 2024 answer is to use it to capture data to train AI. Microsoft could lose 9/10 of their market share (which they won't) and still have millions of devices harvesting training data for them.
- lalalandland 2y ago
  If GDPR is at all questionable, Microsoft is not able to push this to EU countries. The risk for a business to get in GDPR trouble is to high.

jerf 2y ago

"We've realized for some time that not only are language model technologies going to dominate our future" Close, but not quite. What we realize right now is that AI has pumped, pumped, pumped a whole lot of stock valuations sky high, and the first faint glimmers of the market asking that some actual money be delivered in proportion to the sky high valuations are starting to appear. (Note carefully the "in proportion to"; it is not enough to make a bit of cash. These valuations are enormous.) The stocks so pumped need results more-or-less right now, since propagating anything they have right now out into the market to actually gather revenue is going to take some time. I expect a couple of other Hail Mary plays like this to show up too. And again, let me underline, emphasize, and highlight, this isn't about whether AI can make "any" money. It's about whether it can justify those staggeringly high stock valuations in, say, the next year or so. This is a fairly uphill battle. Even if you are a True Believer than AI will be an economic revolution in literally the next 12 months or so, it's hard not to look at the market and come to the conclusion that right now every AI player is being valuated as the inevitable victor and the inevitable beneficiary of 100% (or more) of the AI gain, e.g., nVidia is being valuated as if there will never, ever be any competition for AI hardware despite all the incentives, each AI user valuated as if they are the inevitable winner in their respective domains, etc. They can't all win simultaneously like that, it's just impossible. (On the plus side, it's really just stock valuations. I don't expect an AI winter this time, the tools are much more valuable than they were last time. I mean, I would still very firmly say that what we have right now is grotesquely over hyped, but there's still quite a bit of "there" there. The previous AI hype cycles didn't have anywhere near as much "there" there.)

AlexandrB 2y ago
The costs are also largely being ignored at the moment. Both the obvious marginal costs (electricity, hardware for compute, dev time) and the hidden costs (flood of spam and misinformation).
- Yoric 2y ago
  Also, the environmental cost. It's hidden, but won't be forever.
- 2y ago

pjkundert 2y ago

Suddenly, distributed private inference, compute, storage and bandwidth mediated by Holochain at global scale will become … valuable.

bee_rider 2y ago

It seems like a somewhat far fetched explanation, but then I can’t think of a better one. They come up with this farcical application that nobody wants, to justify the hardware requirement, so that at least the hardware will be out there for when Apple shows them what the actual application is.

_uhtu 2y ago

> We've realized for some time that... language model technologies going to dominate our future Citation Needed.

deleted 2y ago
[deleted]

SideburnsOfDoom 2y ago

> language model technologies going to dominate our future I know many people who are _far from convinced_ of that. But that's not the relevant point: Microsoft, Google et all clearly are. And watching them collectively, desperately stampede after the mirage, at great cost, is certainly something.

oldpersonintx 2y ago

Apple is in terrible shape on software, AI is driving this home the only tier-one, network-effects app they control is iMessage search? someone else maps? someone else video? someone else audio? someone else and now they will go to someone else for AI what did they do in the meantime? VR that bellyflopped

AlexandrB 2y ago
Not sure why this is relevant. Apple's forte was always the hardware (and lately the greater focus on privacy). And you could easily say the same about Microsoft: search? someone else maps? someone else video? someone else audio? someone else Arguably Apple is at least better positioned than MS for some of these because people actually use Apple Maps and Apple Music. When was the last time someone whipped out Bing Maps?
boxed 2y ago
You forgot iOS. App APIs are also network effect things.

amelius 2y ago

Just wait until TSMC asks for 30% of Apple's revenue or they'll kick Apple out of their Fabstore. Fabrication is where the tech advantage lies, not hardware design IP. Anyone nowadays can program VHDL, see open source hardware. The challenge is actually fabricating it.

yyggvbb 2y ago

I think Microsoft software teams are already working behind their legal teams. Their business model is more about licensing than making products. Likely they’ll force some kind of regulatory situation that’s advantage to them. For example no AI capable OSs for government work, or something like that.

Miraste 2y ago

Apple doesn't have as much of an advantage as it appears. Their Neural Engine are fast and getting better, but modern machine learning models need another resource: fast RAM. Unfortunately for Apple, their whole product stack is segmented based on RAM. There are millions and millions of Apple devices with lightning-fast processors and 8GB or even 4GB RAM. These won't be able to do anything like Copilot+, unless they make calls to a datacenter like Microsoft is doing. Their existing local models are hit-or-miss already. FaceTime's audio transcription is laughable. Whether this is due to memory requirements, I don't know, but it doesn't bode well for further models.

luma 2y ago

> Apple has a huge advantage because they get to pay nothing for a superior compute infrastructure that comes with a better privacy and latency story. Right now, AI is a pay-$B-to-play and you need the massive datacenter compute for training and skilled researchers to develop. Apple silicon is neat, but let's not pretend like they have any credible story to tell about AI, and their recent deal with OpenAI suggests that they've been forced to punt. > As soon as they release something, it's going to be good, and it's going to be free, because it runs fast on consumer hardware and consumer electricity. Apple recently released their first public LLM, OpenELM. The largest released model is OpenELM-3B. The Instruct version of this model scores a 24.8 on the MMLU benchmark, which is a pick-one-of-four multiple choice test. Apple's biggest and best model somehow manages to get less than 25% on a 4 option multiple choice exam. For reference, MS released a similar size model Phi3-mini-4k which scored a 69 on that same exam. Yes, Apple has some nice local hardware to run models, but they don't have the models. They don't have the models because they don't have the people and they don't have the compute.

surgical_fire 2y ago

> As soon as they release something, it's going to be good Huh? How is that a given?

luma 2y ago
It is absolutely not true, they already released a model (OpenELM) and it compares extremely poorly versus similar size models from the likes of MS. This chart is correct and to scale: https://i.imgur.com/5mjlesU.png https://i.imgur.com/5mjlesU.png

deleted 2y ago

[deleted]

ca_tech 2y ago

The realized value of LLMs is going to be output from a data to AI pipeline. From raw data to actions or insights. This is Microsoft's play to control the entire process. They are attempting to abstract away all the "tooling" needed to manage and process data. It falls down because there is no discretion left to the consumer. Objectively, the computer is already processing all the data you are interacting with. Subjectively it is assumed its ephemeral. Computer forensics proves this wrong. So I think to refine my opening statement, the realized value of LLMs is going to be the output from a "curated" data to AI pipeline. Which Microsoft is not providing with this solution.

Wytwwww 2y ago

> Apple - their silence, but also their hardware capabilities to run LLMs > billion dollar datacenter investment that still provides a worse experience It's not at all clear to me that those very small local models would be competing directly with large LLMs run in datacentres for the same use cases. If that's going to be the case MS/etc. would also require significantly less compute power that's is currently expected, so Nvidia would be the real losers in that case (unless they shift to consumer HW). Also does Apple have such a huge advantage compared to the newer AMD/Intel/Google/Qualcomm(?) chips with NPUs? Yes Apple started investing sooner but all the phones/tablets and the overwhelming majority of laptops are crippled by extremely low memory capacity (in the context of running LLMs). So are they really that much ahead? e.g. what real advantages does the latest iPhone have compared to a Pixel 8 Pro (sure it's still a niche device, with a seemingly inferior an chip (but more memory) but based on what we know Google seems to be ahead in AI development and will make their solution available on all high end Android devices). > As soon as they release something, it's I doubt that's a given, their software has long ceased to be highly particularly exceptional and on the hardware side the only actually innovative product they have released for years has been a flop with a very unclear future.