4 ms·

I don't understand at all what the monetary value of this algorithm should be. The authors are positioning themselves as a company and not merely academics :

I don't understand at all what the monetary value of this algorithm should be. The authors are positioning themselves as a company and not merely academics : A Sepp Hochreiter's video from 6 months ago hyping xLSTM : https://youtu.be/hwIt7ezy6t8?feature=shared&t=561 https://youtu.be/hwIt7ezy6t8?feature=shared&t=561 in which he state his intent to raise €300M to make a european alternative to openai's GPT for niche domains thanks to this new method that will allow to train for cheaper and better. He recently received (2023) € 35,000 in prize money at the 5th annual German AI Award. https://www.jku.at/en/festival-university/media/detail/news/deutscher-ki-innovationspreis-fuer-jku-forscher-sepp-hochreiter/ https://www.jku.at/en/festival-university/media/detail/news/... Or is it just an academic tactic to get more funding ? To extract more work from PhD students by making them think they are going to strike it big ? How are they intending to build a moat if they publish their papers ? Will this technology be encumbered by patents/license ?

AIsore 2y ago
If you are asking that question, I guess you must have wondered about this for years, right, in fact nearly a decade? I mean why would Google have bought DeepMind with them publishing in peer reviewed journals for years after? Same for Meta (formerly facebook)? I think there is a well trodden path being followed here ... and I am surprised by your surprise.
- GistNoesis 2y ago
  Acquisitions, like for DeepMind is usually a way to hire talent. It can make sense when the technology is new and getting a few year of lead time on what is going to be a growing market may make some financial sense. In this specific xLSTM case, the industry has matured, they are just one among many (Mamba, S3Ms, transformers-variants... ), they have already been sitting on it for at least 6 months, I don't see what their play is. An other case study that's probably interesting, are the authors of the Adam Paper, https://arxiv.org/abs/1412.6980 https://arxiv.org/abs/1412.6980 , (Awarded "2020: The Adam optimization paper is the world's #1 most cited scientific paper of the past five years"). Probably a few (10?,100?) billions worth value created. You can find the authors bios http://dpkingma.com/ http://dpkingma.com/ https://jimmylba.github.io/ https://jimmylba.github.io/ I think there is a huge problem with the capture and sharing of value in the whole deep-learning industry. Academia's naivety plays a role in it, Generational Shift technologies are badly rewarded. Incremental Shift technologies aren't rewarded at all. Powerful technologies into many hands with low rewards for their creators while the value they generate keeps going to the same pockets. That's a recipe for disaster. Will be a fun thing to come back in a few years to see how it had unfold.
  - AIsore 2y ago
    There is a lot to unpack. But let's start with your first point. If the acquisition of DeepMind was just a talent acquisition, why continue to let them publish? Your second point: how did you get the impression that this market is "mature"? And, going back to the first point, which market do you actually mean to have matured? Regarding value creation/capturing/sharing, academic naivety, this industry is no different to any other, nor has basic economics changed. Deep Learning is an amazingly powerful new technology that has the potential to change the world. But how you make products/services out of it which we all value, pay for and thus provide the basis of employment is the usual risk/reward cycle ANY business has to subject itself to. More believe in the technology = more investors willing to fund businesses that have negative free cash-flow for longer. Yes, the competitive landscape seems stacked against new entrants, but that is no different to when today's teach behemoths started. And yes, as with any industry, monopolies are not great and, according to Kara Swisher, maybe tech at large, today, is an unhealthy monopoly.
NOCompromisER 2y ago
Will this technology be encumbered by patents/license ? I guess it is most likely already patented (or very close) and you will need a license. xLTSM is not open source

AIsore 2y ago

If you are asking that question, I guess you must have wondered about this for years, right, in fact nearly a decade? I mean why would Google have bought DeepMind with them publishing in peer reviewed journals for years after? Same for Meta (formerly facebook)? I think there is a well trodden path being followed here ... and I am surprised by your surprise.

GistNoesis 2y ago
Acquisitions, like for DeepMind is usually a way to hire talent. It can make sense when the technology is new and getting a few year of lead time on what is going to be a growing market may make some financial sense. In this specific xLSTM case, the industry has matured, they are just one among many (Mamba, S3Ms, transformers-variants... ), they have already been sitting on it for at least 6 months, I don't see what their play is. An other case study that's probably interesting, are the authors of the Adam Paper, https://arxiv.org/abs/1412.6980 https://arxiv.org/abs/1412.6980 , (Awarded "2020: The Adam optimization paper is the world's #1 most cited scientific paper of the past five years"). Probably a few (10?,100?) billions worth value created. You can find the authors bios http://dpkingma.com/ http://dpkingma.com/ https://jimmylba.github.io/ https://jimmylba.github.io/ I think there is a huge problem with the capture and sharing of value in the whole deep-learning industry. Academia's naivety plays a role in it, Generational Shift technologies are badly rewarded. Incremental Shift technologies aren't rewarded at all. Powerful technologies into many hands with low rewards for their creators while the value they generate keeps going to the same pockets. That's a recipe for disaster. Will be a fun thing to come back in a few years to see how it had unfold.
- AIsore 2y ago
  There is a lot to unpack. But let's start with your first point. If the acquisition of DeepMind was just a talent acquisition, why continue to let them publish? Your second point: how did you get the impression that this market is "mature"? And, going back to the first point, which market do you actually mean to have matured? Regarding value creation/capturing/sharing, academic naivety, this industry is no different to any other, nor has basic economics changed. Deep Learning is an amazingly powerful new technology that has the potential to change the world. But how you make products/services out of it which we all value, pay for and thus provide the basis of employment is the usual risk/reward cycle ANY business has to subject itself to. More believe in the technology = more investors willing to fund businesses that have negative free cash-flow for longer. Yes, the competitive landscape seems stacked against new entrants, but that is no different to when today's teach behemoths started. And yes, as with any industry, monopolies are not great and, according to Kara Swisher, maybe tech at large, today, is an unhealthy monopoly.

NOCompromisER 2y ago

Will this technology be encumbered by patents/license ? I guess it is most likely already patented (or very close) and you will need a license. xLTSM is not open source