4 ms·

I'm not in this field as others. so I'd really appreciate a comment from some experienced HN folks. I wonder what's the end goal. The ML/AI world seems to be

• 7moritz7 3y ago
  Historically speaking there is a good chance the transformers approach will be replaced by a new approach or a substantially altered transformers approach in the next five years. Efficiency will increase both on the software and the hardware front simultaneously, there is a lot to be optimized with the same parameter count. What looks state of the art will probably look to people in 20 years how the 1885 Mercedes Benz car looks like to us in terms of efficiency, right now it's a bit of a bruteforce approach in general
• throwawayadvsec 3y ago
  there are already dozens of use cases that would become (more) profitable/interesting if we had chips optimized for LLMs they were asic for BTC at a way earlier stage +they can always pivot in a year if the market changes too much
  • DoingIsLearning 3y ago
    > they can always pivot in a year I think you are downplaying a bit the timescale of going from ideation to tape-out in ASIC design.
• aperrien 3y ago
  Perhaps some sort of memristor crossbar array that could process large array calculations efficiently? Something like that could be pretty universal, until the tech moves to spiking neural networks. And even then the knowledge you gain from advancing on that path would likely transfer over.