4 ms·

I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me i

by ezst 2y ago

I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me in any way nor supports any work-related tasks). Worse, most people have no use for that (and recent surveys even show predominant hostility towards AI creep). We shouldn't be paying extra for that, it should be opt-in, and then it would become clear (by looking at the sales and how few are willing to pay a premium for "AI") how overblown and unnecessary this is.

DrillShopper 2y ago
Corporatized gains in the market from hype Socialized losses in increased carbon emissions, upheaval from job loss, and higher prices on hardware. The more they say the future will be better the more that it looks like the status quo.
renewiltord 2y ago
I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money because they won't optimize die size. Disgusting.
- nl 2y ago
  Is this a parody? Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?
  - throwaway48476 2y ago
    Maybe through a game of telephone they confused die size and node size?
  - thfuran 2y ago
    Yes, they're making fun of the comment they replied to.
    - singlepaynews 2y ago
      Would you do me the favor of explaining the joke? I get the premise—nobody cares about die size, but the comment being mocked seems perfectly innocuous to me? They want a laptop without an NPU b/c according to link we get more out of CPU anyways? What am I missing here?
      - michaelt 2y ago
        It has been the norm for several decades to have hardware features that go unused. The realities of mass manufacturing and supply chains and whatnot mean it's cheaper to get a laptop with a webcam I don't use, a fingerprint reader I don't use, and an SD card reader I don't use. It's cheaper to get a CPU with integrated graphics I don't use, a trusted execution environment I don't use, remote management features I don't use. It's cheaper to get a discrete GPU with RGB LEDs I don't use, directx support I don't use, four outputs when I only need one. It's cheaper to get a motherboard with integrated wifi than one without.
  - tedunangst 2y ago
    No, no, no, you just don't get it. The only thing Dell will sell me is a laptop 324mm wide, which is totally appalling, but if they offered me a laptop that's 320mm wide, I'd immediately buy it. In my line of work, which is totally serious business, every millimeter counts.
- _zoltan_ 2y ago
  News flash: you're in the niche of the niche. People don't care about die size. I'd be willing to bet that the amount of money they are missing out on is miniscule and is by far offset by people's money who care about other stuff. Like you know, performance and battery life, just to stick to your examples.
  - mattnewton 2y ago
    That’s exactly what the poster is arguing- they are being sarcastic.
    - shermantanktop 2y ago
      It whooshed over my head too. That’s the danger of sarcasm…it’s a cooperative form of humor but the other party might not get it.
- waveBidder 2y ago
  your satire is off base enough that people don't understand it's satire.
  - heavyset_go 2y ago
    The Poe's Law means it's working.
  - 0xDEAFBEAD 2y ago
    Says a lot about HN that so many believed he was genuine.
- fijiaarone 2y ago
  Yeah, I know what you mean. I hate lugging around a big CPU core.
- ezst 2y ago
  I'm fine with the mockery, I genuinely hadn't realized that "wanting to pay for what one needs" was such a hot and controversial take.
  - ginko 2y ago
    The extra cost of the area spent on npu cores is pretty hard to quantify. I guess removing it would allow for higher yields and number of chips per wafer but then you’d need to set up tooling for two separate runs (one with npu and one without) Add to that that most of the cost is actually the design of the chip and it’s clear why manufacturers just always add the extra features. Maybe they could sell a chip with the NPU permanently disabled but I guess that wouldn’t be what you want either? Fwiw there should be no power downside to having an unused unit. It’ll just not be powered.
    - ezst 2y ago
      The argument boils down to "since it's there, better to keep it because making a version without it would defeat economies of scale and not save much, if at all", and that's a sensible take… under the assumption that there's a general demand for NPUs, which I contest. In practice, everyone is paying a premium for NPUs that only a minority desires, and only a fraction of that minority essentially does "something" with it. This thread really helps to show that the use-cases are few, non-essential, and that the general application landscape hasn't adopted NPUs and has very little incentive to do so (because of the alien programming model, because of hardware compat across vendors, because of the ecosystem being a moving target with little stability in sight, and because of the high-effort/low-reward in general). I do want to be wrong, of course. Tech generally is exciting because it offers new tools to crack old problems, opening new venues and opportunities in the process. Here it looks like we have a solution in search for a problem that was set by marketing departments.
      - Miraste 2y ago
        Modern SoCs already have all kinds of features with use-cases that are few and non-essential. Granted they don't take as much space as NPUS, but manufacturers are betting that if NPUs are available, software will evolve to use them regularly. If it doesn't, they'll probably go away in a few generations. But at a minimum, Microsoft and Apple seem highly committed to using them.
mardifoufs 2y ago
NPUs were a thing (and a very common one in mobile CPUs too) way before the LLM craze.
kalleboo 2y ago
> most people have no use for that Apple originally added their NPUs before the current LLM wave to support things like indexing your photo library so that objects and people are searchable. These features are still very popular. I don't think these NPUs are fast enough for GenAI anyway.
- wmf 2y ago
  MS Copilot and "Apple Intelligence" are running a small language model and image generation on the NPU so that should count as "GenAI".
  - kalleboo 2y ago
    It's still in beta so we'll see how things go but I saw someone testing what Apple Intelligence ran on-device vs sent off to the "private secure cloud" and even stuff like text summaries were being sent to the cloud.
- grugagag 2y ago
  I wish I could turn that off on my phone.
jcgrillo 2y ago
I just got an iphone and the whole photos thing is absolutely garbage. All I wanted to do was look through my damn photos and find one I took recently but it started playing some random music and organized them in no discernible order.. like it wasn't the reverse time sorted.. Idk what kind of fucked up "creative process" came up with that bullshit but I sure wish they'd unfuck it stat. The camera is real good though.
- james_marks 2y ago
  There’s an album called “Recents” that’s chronological and scrolled to the end. “Recent” seems to mean everything; I’ve got 6k+ photos, I think since the last fresh install, which is many devices ago. Sounds like the view you’re looking for and will stick as the default once you find it, but you do have to bat away some BS at first.
- coldpie 2y ago
  There is a chronological view tucked in there somewhere, but they really do hide it behind the other crap. Once you manage to get into it, it usually stays that way, but sometimes it kicks me back out to the random nonsense view and I have to take a few minutes to find chronological again.
- nioj 2y ago
  I think the music part is related to the setting called "Show Featured Content" in the Photos app settings.
  - jcgrillo 2y ago
    Yeah that's how I ended up making it stop but if this is what these NPUs are being used for just.. why?
barfingclouds 2y ago
Agreed. I use perplexity and chat gpt daily, but if I understand correctly that’s all done off device which is fine with me. I don’t need to generate weird images, or have my phone rewrite emails for me, or summarize stuff.

by ezst 2y ago

DrillShopper 2y ago

Corporatized gains in the market from hype Socialized losses in increased carbon emissions, upheaval from job loss, and higher prices on hardware. The more they say the future will be better the more that it looks like the status quo.

renewiltord 2y ago

I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money because they won't optimize die size. Disgusting.

nl 2y ago
Is this a parody? Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?
- throwaway48476 2y ago
  Maybe through a game of telephone they confused die size and node size?
- thfuran 2y ago
  Yes, they're making fun of the comment they replied to.
  - singlepaynews 2y ago
    Would you do me the favor of explaining the joke? I get the premise—nobody cares about die size, but the comment being mocked seems perfectly innocuous to me? They want a laptop without an NPU b/c according to link we get more out of CPU anyways? What am I missing here?
    - michaelt 2y ago
      It has been the norm for several decades to have hardware features that go unused. The realities of mass manufacturing and supply chains and whatnot mean it's cheaper to get a laptop with a webcam I don't use, a fingerprint reader I don't use, and an SD card reader I don't use. It's cheaper to get a CPU with integrated graphics I don't use, a trusted execution environment I don't use, remote management features I don't use. It's cheaper to get a discrete GPU with RGB LEDs I don't use, directx support I don't use, four outputs when I only need one. It's cheaper to get a motherboard with integrated wifi than one without.
- tedunangst 2y ago
  No, no, no, you just don't get it. The only thing Dell will sell me is a laptop 324mm wide, which is totally appalling, but if they offered me a laptop that's 320mm wide, I'd immediately buy it. In my line of work, which is totally serious business, every millimeter counts.
_zoltan_ 2y ago
News flash: you're in the niche of the niche. People don't care about die size. I'd be willing to bet that the amount of money they are missing out on is miniscule and is by far offset by people's money who care about other stuff. Like you know, performance and battery life, just to stick to your examples.
- mattnewton 2y ago
  That’s exactly what the poster is arguing- they are being sarcastic.
  - shermantanktop 2y ago
    It whooshed over my head too. That’s the danger of sarcasm…it’s a cooperative form of humor but the other party might not get it.
waveBidder 2y ago
your satire is off base enough that people don't understand it's satire.
- heavyset_go 2y ago
  The Poe's Law means it's working.
- 0xDEAFBEAD 2y ago
  Says a lot about HN that so many believed he was genuine.
fijiaarone 2y ago
Yeah, I know what you mean. I hate lugging around a big CPU core.
ezst 2y ago
I'm fine with the mockery, I genuinely hadn't realized that "wanting to pay for what one needs" was such a hot and controversial take.
- ginko 2y ago
  The extra cost of the area spent on npu cores is pretty hard to quantify. I guess removing it would allow for higher yields and number of chips per wafer but then you’d need to set up tooling for two separate runs (one with npu and one without) Add to that that most of the cost is actually the design of the chip and it’s clear why manufacturers just always add the extra features. Maybe they could sell a chip with the NPU permanently disabled but I guess that wouldn’t be what you want either? Fwiw there should be no power downside to having an unused unit. It’ll just not be powered.
  - ezst 2y ago
    The argument boils down to "since it's there, better to keep it because making a version without it would defeat economies of scale and not save much, if at all", and that's a sensible take… under the assumption that there's a general demand for NPUs, which I contest. In practice, everyone is paying a premium for NPUs that only a minority desires, and only a fraction of that minority essentially does "something" with it. This thread really helps to show that the use-cases are few, non-essential, and that the general application landscape hasn't adopted NPUs and has very little incentive to do so (because of the alien programming model, because of hardware compat across vendors, because of the ecosystem being a moving target with little stability in sight, and because of the high-effort/low-reward in general). I do want to be wrong, of course. Tech generally is exciting because it offers new tools to crack old problems, opening new venues and opportunities in the process. Here it looks like we have a solution in search for a problem that was set by marketing departments.
    - Miraste 2y ago
      Modern SoCs already have all kinds of features with use-cases that are few and non-essential. Granted they don't take as much space as NPUS, but manufacturers are betting that if NPUs are available, software will evolve to use them regularly. If it doesn't, they'll probably go away in a few generations. But at a minimum, Microsoft and Apple seem highly committed to using them.

mardifoufs 2y ago

NPUs were a thing (and a very common one in mobile CPUs too) way before the LLM craze.

kalleboo 2y ago

> most people have no use for that Apple originally added their NPUs before the current LLM wave to support things like indexing your photo library so that objects and people are searchable. These features are still very popular. I don't think these NPUs are fast enough for GenAI anyway.

wmf 2y ago
MS Copilot and "Apple Intelligence" are running a small language model and image generation on the NPU so that should count as "GenAI".
- kalleboo 2y ago
  It's still in beta so we'll see how things go but I saw someone testing what Apple Intelligence ran on-device vs sent off to the "private secure cloud" and even stuff like text summaries were being sent to the cloud.
grugagag 2y ago
I wish I could turn that off on my phone.

jcgrillo 2y ago

I just got an iphone and the whole photos thing is absolutely garbage. All I wanted to do was look through my damn photos and find one I took recently but it started playing some random music and organized them in no discernible order.. like it wasn't the reverse time sorted.. Idk what kind of fucked up "creative process" came up with that bullshit but I sure wish they'd unfuck it stat. The camera is real good though.

james_marks 2y ago
There’s an album called “Recents” that’s chronological and scrolled to the end. “Recent” seems to mean everything; I’ve got 6k+ photos, I think since the last fresh install, which is many devices ago. Sounds like the view you’re looking for and will stick as the default once you find it, but you do have to bat away some BS at first.
coldpie 2y ago
There is a chronological view tucked in there somewhere, but they really do hide it behind the other crap. Once you manage to get into it, it usually stays that way, but sometimes it kicks me back out to the random nonsense view and I have to take a few minutes to find chronological again.
nioj 2y ago
I think the music part is related to the setting called "Show Featured Content" in the Photos app settings.
- jcgrillo 2y ago
  Yeah that's how I ended up making it stop but if this is what these NPUs are being used for just.. why?

barfingclouds 2y ago

Agreed. I use perplexity and chat gpt daily, but if I understand correctly that’s all done off device which is fine with me. I don’t need to generate weird images, or have my phone rewrite emails for me, or summarize stuff.