4 ms·

I remember reading about HAMR a few years ago Why does memory and other hardware tech always seem to magically come to fruition? At least compared to software

by lmpdev 3y ago

I remember reading about HAMR a few years ago Why does memory and other hardware tech always seem to magically come to fruition? At least compared to software Is it because they're more conservative in their announcements? More certainty in a path forward? More $$$?

celeritascelery 3y ago
It seems like it was doubtful this would come to fruition, at least in 2013. There are many promising technologies that that never become economically viable. But necessity is the mother of invention.
wmf 3y ago
I'm not sure what you mean by magically. For every tech that makes it to production there are one or two others that don't and people just forget about them. For example, storage roadmaps used to show bit-patterned media. Phase-change memory would have been nice to have too.
- magicalhippo 3y ago
  I recall reading about holographic storage[1] back in the 90s, was supposed to become a big thing "real soon". [1]: https://en.wikipedia.org/wiki/Holographic_data_storage https://en.wikipedia.org/wiki/Holographic_data_storage
  - Twirrim 3y ago
    There's whitepapers and articles every few years on Holographic storage, it's one of those perpetually just around the corner things. See also memristors, that HP in particular have been saying is coming in the next couple of years, fundamentally changing the entire computing industry, for decades.
  - OldGuyInTheClub 3y ago
    It's not dead yet[1] but, yes, it's never quite caught on the way so many of us thought it would back in the 1980s and 90s. The 1987 movie "Innerspace" even referred to "photon echo memory chips." Friends of mine worked for years at one of the companies mentioned in the Wiki page but came up just a little short. FWIW, gallium arsenide was also supposed to take over from silicon for CPUs once speeds went above 25-MHz (or so). A senior colleague of mine said, "There are a lot of smart people whose kids's college tuition depend on making silicon a little better every year." And so, it came to pass. GaAs is definitely important but we've got multi-GHz silicon CPUs now. [1] https://www.microsoft.com/en-us/research/project/hsd/ https://www.microsoft.com/en-us/research/project/hsd/ "Project HSD: Holographic Storage Device for the Cloud"
- Clamchop 3y ago
  If we extend that to medical, the graveyard is bananas big.
pyuser583 3y ago
It’s the new Moores law. Instead of processing power doubling, it’s storage.
- wmf 3y ago
  Kryder's law is the storage equivalent of Moore's law. It's also coming to an end.
scheme271 3y ago
Might be a selective bias effect? A lot of hardware stuff has crashed and burned. Most famously fusion has been 20 years away for 60+ years, and there's always new rechargable battery tech that's going to replace lithium ion in a few years. For computer hardware, mram, feram, and optane/crosspoint memory has all been pretty disappointing.
- KirillPanov 3y ago
  My favorite is "bubble memory". Everybody was convinced we were going to store all our precious data on these itty-bitty little bubbles. It was all over the trade publications, very hype, such bling, much buzz: https://en.wikipedia.org/wiki/Bubble_memory https://en.wikipedia.org/wiki/Bubble_memory And then it just basically turned out to be a flop.
  - bioemerl 3y ago
    Looks like it actually was kind of successful for a while though. It just got out competed by other methods.
gibolt 3y ago
Lots of things don't work until they do, and then iteration can be rapid. In addition, solutions for the same problem tend to have drastically different limitations and bottlenecks. Any new breakthrough can move the boundaries forward in leaps.
omeysalvi 3y ago
Most software innovations seem to be made to make the developer's life easier instead of creating more performant software. I'm not complaining but user focused software innovations like LLMs do seem to come very few and far between compared to hardware.
- wahahah 3y ago
  I wouldn't consider LLMs a software innovation.
  - karlshea 3y ago
    I feel like if you're gonna drop a take like that you should have at least one more sentence of justification to follow it up with.
    - hkt 3y ago
      I kind of admire the gumption, personally. Like writing them off as fancy markov bots.
    - Joeri 3y ago
      I’m not sure I agree, but I’ll do my best to argue the position. The neural net software algorithms have been around for decades. What made LLM’s feasible are the hardware advances to achieve unprecedented scale, just barely providing the ability (at great cost) to train today’s LLM models. Transformer architecture might be called a software innovation, but RWKV Raven gets similar performance to transformers and is built on decades-old RNN technology. So it is the hardware that was far more instrumental than the software in achieving LLM’s. Counter to that argument: had google not done neural net research for google translate and proved the transformer approach scaled and performed well in their “attention is all you need” paper, people wouldn’t have spent the money to train foundation LLM models and we would not be having this discussion, so the software really mattered more than the hardware. In reality I think it’s a little bit of both.
      - 3y ago
dheera 3y ago
I wouldn't be surprised if they have all this tech developed for 256TB hard drives already but they're deliberately announcing it slowly over the years to buffer their time to do more research, lest they announce 256TB too fast and shareholders are like "why don't we have 1PB already damnit" a quarter later and short the stock to oblivion
- lmpdev 3y ago
  This happens all the time in industries close to the "asymptote of perfection" for whatever product or service they're producing. Think of products like ICEs, industrial processes or mature/standardised commodities At what point does an influencial government say "it's unlikely there is substantial innovation to be uncovered for ___ product" and begin the process of deprivatisation for good of the public?
physicsguy 3y ago
I worked in this area of magnetics, and it’s been in research for at least 15 years. Companies had made devices by about 10 years ago but the storage capacities were low so they weren’t that useful for consumer applications at that point. The hard problem with HAMR is dissipating the heat.
fulafel 3y ago
In storage there are standards and abstractions that commodify the product. There are lots of vendors and lots of projects at said vendors that try to deliver improved parts and they can count on customer demand & competitive advantage if they succeed. Most software (and hardware) is not like this.

by lmpdev 3y ago

celeritascelery 3y ago

It seems like it was doubtful this would come to fruition, at least in 2013. There are many promising technologies that that never become economically viable. But necessity is the mother of invention.

wmf 3y ago

I'm not sure what you mean by magically. For every tech that makes it to production there are one or two others that don't and people just forget about them. For example, storage roadmaps used to show bit-patterned media. Phase-change memory would have been nice to have too.

magicalhippo 3y ago
I recall reading about holographic storage[1] back in the 90s, was supposed to become a big thing "real soon". [1]: https://en.wikipedia.org/wiki/Holographic_data_storage https://en.wikipedia.org/wiki/Holographic_data_storage
- Twirrim 3y ago
  There's whitepapers and articles every few years on Holographic storage, it's one of those perpetually just around the corner things. See also memristors, that HP in particular have been saying is coming in the next couple of years, fundamentally changing the entire computing industry, for decades.
- OldGuyInTheClub 3y ago
  It's not dead yet[1] but, yes, it's never quite caught on the way so many of us thought it would back in the 1980s and 90s. The 1987 movie "Innerspace" even referred to "photon echo memory chips." Friends of mine worked for years at one of the companies mentioned in the Wiki page but came up just a little short. FWIW, gallium arsenide was also supposed to take over from silicon for CPUs once speeds went above 25-MHz (or so). A senior colleague of mine said, "There are a lot of smart people whose kids's college tuition depend on making silicon a little better every year." And so, it came to pass. GaAs is definitely important but we've got multi-GHz silicon CPUs now. [1] https://www.microsoft.com/en-us/research/project/hsd/ https://www.microsoft.com/en-us/research/project/hsd/ "Project HSD: Holographic Storage Device for the Cloud"
Clamchop 3y ago
If we extend that to medical, the graveyard is bananas big.

pyuser583 3y ago

It’s the new Moores law. Instead of processing power doubling, it’s storage.

wmf 3y ago
Kryder's law is the storage equivalent of Moore's law. It's also coming to an end.

scheme271 3y ago

Might be a selective bias effect? A lot of hardware stuff has crashed and burned. Most famously fusion has been 20 years away for 60+ years, and there's always new rechargable battery tech that's going to replace lithium ion in a few years. For computer hardware, mram, feram, and optane/crosspoint memory has all been pretty disappointing.

KirillPanov 3y ago
My favorite is "bubble memory". Everybody was convinced we were going to store all our precious data on these itty-bitty little bubbles. It was all over the trade publications, very hype, such bling, much buzz: https://en.wikipedia.org/wiki/Bubble_memory https://en.wikipedia.org/wiki/Bubble_memory And then it just basically turned out to be a flop.
- bioemerl 3y ago
  Looks like it actually was kind of successful for a while though. It just got out competed by other methods.

gibolt 3y ago

Lots of things don't work until they do, and then iteration can be rapid. In addition, solutions for the same problem tend to have drastically different limitations and bottlenecks. Any new breakthrough can move the boundaries forward in leaps.

omeysalvi 3y ago

Most software innovations seem to be made to make the developer's life easier instead of creating more performant software. I'm not complaining but user focused software innovations like LLMs do seem to come very few and far between compared to hardware.

wahahah 3y ago
I wouldn't consider LLMs a software innovation.
- karlshea 3y ago
  I feel like if you're gonna drop a take like that you should have at least one more sentence of justification to follow it up with.
  - hkt 3y ago
    I kind of admire the gumption, personally. Like writing them off as fancy markov bots.
  - Joeri 3y ago
    I’m not sure I agree, but I’ll do my best to argue the position. The neural net software algorithms have been around for decades. What made LLM’s feasible are the hardware advances to achieve unprecedented scale, just barely providing the ability (at great cost) to train today’s LLM models. Transformer architecture might be called a software innovation, but RWKV Raven gets similar performance to transformers and is built on decades-old RNN technology. So it is the hardware that was far more instrumental than the software in achieving LLM’s. Counter to that argument: had google not done neural net research for google translate and proved the transformer approach scaled and performed well in their “attention is all you need” paper, people wouldn’t have spent the money to train foundation LLM models and we would not be having this discussion, so the software really mattered more than the hardware. In reality I think it’s a little bit of both.
    - 3y ago

dheera 3y ago

I wouldn't be surprised if they have all this tech developed for 256TB hard drives already but they're deliberately announcing it slowly over the years to buffer their time to do more research, lest they announce 256TB too fast and shareholders are like "why don't we have 1PB already damnit" a quarter later and short the stock to oblivion

lmpdev 3y ago
This happens all the time in industries close to the "asymptote of perfection" for whatever product or service they're producing. Think of products like ICEs, industrial processes or mature/standardised commodities At what point does an influencial government say "it's unlikely there is substantial innovation to be uncovered for ___ product" and begin the process of deprivatisation for good of the public?

physicsguy 3y ago

I worked in this area of magnetics, and it’s been in research for at least 15 years. Companies had made devices by about 10 years ago but the storage capacities were low so they weren’t that useful for consumer applications at that point. The hard problem with HAMR is dissipating the heat.

fulafel 3y ago

In storage there are standards and abstractions that commodify the product. There are lots of vendors and lots of projects at said vendors that try to deliver improved parts and they can count on customer demand & competitive advantage if they succeed. Most software (and hardware) is not like this.