4 ms·

Google already did this. Google X, Google Brain, Deepmind... The invention of the transformer architecture that underpins modern AI, dozens of core papers in t

Google already did this. Google X, Google Brain, Deepmind... The invention of the transformer architecture that underpins modern AI, dozens of core papers in the field. What they haven't revolutionized is materials science. Memristor technology for example, is still grossly underdeveloped.

deleted 2y ago
[deleted]
nine_k 2y ago
Both memristors and 3D XPoint aka Optane are in a patent limbo. The current patent holders tried them, and found the current crop too hard and unprofitable, but further research and commercialization would require either buying the key patents or waiting for them to expire.
- pklausler 2y ago
  Commercialization, yes, but not necessarily further research.
  - jahewson 2y ago
    Hard to get research funded when it can’t be commercialised sufficiently to merit the risk.
  - nine_k 2y ago
    Fundamental research is likely unhindered. But any research of incremental improvements in an attempt to make the tech commercially viable is stymied. And I would love to see some accessible and available non-volatile RAM measured in gigabytes. It means computers that never get switched "of, no more than a paper book can be switched off. It would take a radically different approach to operating syssystems to reap the benefits though.
- LarsDu88 2y ago
  That one's on Intel. They marketed Optane poorly and created a subpar product for the market they were tackling. The whole point of memristors is to make something better than DRAM and SRAM. Perhaps even blow up the dichotomy between storage and memory altogether (as memristor MRAM can store information even when powered down, unlike DRAM and SRAM). Let here we are in 2024 with no viable memristor products on the market.
- Balgair 2y ago
  Holy crap! Someone else knows about memristors! Oh man, coming neuroscience I had no idea why these weren't being more researched. Thanks for the info there. Just as an FYI to y'all, a memristor is essentially how a synapse works. In that it's an electrical element that counts the change in charges across the leads over time (dCharge/dFlux or dq/dPhi). And all for very little power and it survives power cycles. Like, we talk about AI these days, but memristor incorporating hardware would really get us there. You'd just, you know, need to invent the solid state memristor first, and then make it commercially viable, and then redesign all of computer architecture from the transistor on up. Easy-peasy.
lemoncucumber 2y ago
Unfortunately that's the Google of a decade or two ago. Today's Google is very focused on cost-cutting and I can't imagine them investing in new future-focused projects like those which are not under pressure to turn a profit any time soon.
- ASalazarMX 2y ago
  It's the circle of life, companies eventually get captured by accountants and lose their initial spark. Even companies like Microsoft had an era of exciting innovation.
  - potsandpans 2y ago
    I might argue that Microsoft had two distinct eras of innovation
    - deleted 2y ago
      [deleted]
    - fuzztester 2y ago
      Yes. Less and even less.
      - ASalazarMX 2y ago
        You joke, but upgrading from Windows For Workgroups to Windows 95 felt like entering the future. Its hardware discovery was years ahead of Linux. It was an amazing innovation, which IMO peaked with Windows 2000, and then commercial/political goals started dominating over technical goals.
        fuzztester 2y ago
        interesting. yes it was just a joke, wordplay, really. I barely used windows for workgroups. but I did use windows from XP or 95 onwards. actually I used both. and yeah I remember a bit about the hardware discovery. it was called plug and play, iirc. I remember reading how badly windows Vista turned out. and then there was 8, which I didn't check out much, but remember reading it was somewhat of a flop, because they tried to make everything mobile oriented, even on desktops. haven't used windows much since, except for 7, which was okay for me.
blueboo 2y ago
Demis on designing DeepMind: "When we set up DeepMind, I took inspiration for our research culture from many innovative organizations, including Bell Labs and the Apollo program, but also creative cultures like Pixar." https://www.theatlantic.com/sponsored/google-2023/unlocking-lifes-building-blocks-demis-hassabis/3867/ https://www.theatlantic.com/sponsored/google-2023/unlocking-... As for materials science, there's signs of life in graph nets https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/ https://deepmind.google/discover/blog/millions-of-new-materi.... Meanwhile maybe AlphaFold indicates a precedent towards deep learning systems that advance chemistry. I don't claim these are revolutions, but extrapolating into the future, they might be seen as the catalysts. Maybe.
- zacmps 2y ago
  Personally I think chemistry is going to trail a long way behind biology for a while (in terms of ML solutions). The data, supporting libraries, and funding doesn't seem to be on the same level.

deleted 2y ago

[deleted]

nine_k 2y ago

Both memristors and 3D XPoint aka Optane are in a patent limbo. The current patent holders tried them, and found the current crop too hard and unprofitable, but further research and commercialization would require either buying the key patents or waiting for them to expire.

pklausler 2y ago
Commercialization, yes, but not necessarily further research.
- jahewson 2y ago
  Hard to get research funded when it can’t be commercialised sufficiently to merit the risk.
- nine_k 2y ago
  Fundamental research is likely unhindered. But any research of incremental improvements in an attempt to make the tech commercially viable is stymied. And I would love to see some accessible and available non-volatile RAM measured in gigabytes. It means computers that never get switched "of, no more than a paper book can be switched off. It would take a radically different approach to operating syssystems to reap the benefits though.
LarsDu88 2y ago
That one's on Intel. They marketed Optane poorly and created a subpar product for the market they were tackling. The whole point of memristors is to make something better than DRAM and SRAM. Perhaps even blow up the dichotomy between storage and memory altogether (as memristor MRAM can store information even when powered down, unlike DRAM and SRAM). Let here we are in 2024 with no viable memristor products on the market.
Balgair 2y ago
Holy crap! Someone else knows about memristors! Oh man, coming neuroscience I had no idea why these weren't being more researched. Thanks for the info there. Just as an FYI to y'all, a memristor is essentially how a synapse works. In that it's an electrical element that counts the change in charges across the leads over time (dCharge/dFlux or dq/dPhi). And all for very little power and it survives power cycles. Like, we talk about AI these days, but memristor incorporating hardware would really get us there. You'd just, you know, need to invent the solid state memristor first, and then make it commercially viable, and then redesign all of computer architecture from the transistor on up. Easy-peasy.

lemoncucumber 2y ago

Unfortunately that's the Google of a decade or two ago. Today's Google is very focused on cost-cutting and I can't imagine them investing in new future-focused projects like those which are not under pressure to turn a profit any time soon.

ASalazarMX 2y ago
It's the circle of life, companies eventually get captured by accountants and lose their initial spark. Even companies like Microsoft had an era of exciting innovation.
- potsandpans 2y ago
  I might argue that Microsoft had two distinct eras of innovation
  - deleted 2y ago
    [deleted]
  - fuzztester 2y ago
    Yes. Less and even less.
    - ASalazarMX 2y ago
      You joke, but upgrading from Windows For Workgroups to Windows 95 felt like entering the future. Its hardware discovery was years ahead of Linux. It was an amazing innovation, which IMO peaked with Windows 2000, and then commercial/political goals started dominating over technical goals.
      - fuzztester 2y ago
        interesting. yes it was just a joke, wordplay, really. I barely used windows for workgroups. but I did use windows from XP or 95 onwards. actually I used both. and yeah I remember a bit about the hardware discovery. it was called plug and play, iirc. I remember reading how badly windows Vista turned out. and then there was 8, which I didn't check out much, but remember reading it was somewhat of a flop, because they tried to make everything mobile oriented, even on desktops. haven't used windows much since, except for 7, which was okay for me.

blueboo 2y ago

Demis on designing DeepMind: "When we set up DeepMind, I took inspiration for our research culture from many innovative organizations, including Bell Labs and the Apollo program, but also creative cultures like Pixar." https://www.theatlantic.com/sponsored/google-2023/unlocking-lifes-building-blocks-demis-hassabis/3867/ https://www.theatlantic.com/sponsored/google-2023/unlocking-... As for materials science, there's signs of life in graph nets https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/ https://deepmind.google/discover/blog/millions-of-new-materi.... Meanwhile maybe AlphaFold indicates a precedent towards deep learning systems that advance chemistry. I don't claim these are revolutions, but extrapolating into the future, they might be seen as the catalysts. Maybe.

zacmps 2y ago
Personally I think chemistry is going to trail a long way behind biology for a while (in terms of ML solutions). The data, supporting libraries, and funding doesn't seem to be on the same level.