3 ms·

> In a couple years we will have multiple new announcements per hour Models are research output. If 10 new models are being announced every day in a couple yea

> In a couple years we will have multiple new announcements per hour Models are research output. If 10 new models are being announced every day in a couple years, it would mean that generative AI research has failed to stabilize and produce a stable, reliable component ready for product engineering. And if that's where we are in a couple years, that's almost certainly a sign that the hype was misplaced and that money is chasing after itself trying to recoup sunk costs. That's a failure scenario for this technology, not what an AI-optimist (you otherwise seem to be one) should be anticipating.

int_19h 3y ago
That doesn't follow at all. It just means that there are still low-hanging fruits to pursue for better (smarter, faster, larger context etc) new models, but it doesn't say anything about the stability and usefulness of existing models.
nickpsecurity 3y ago
That’s not true. Both good science and market-driven engineering favor continued iterations on existing ideas looking for improvements or alternatives. We’re often exploring a giant space of solutions. Unlike many fields, the A.I. people are publicly posting many of their steps in this journey, their iterations, for review. While it brings lots of fluff, such openness dramatically increases innovation rate compared to fields where you only see results once or twice a year. Both people using cloud API’s and FOSS developers are steadily increasing effectiveness in both experimentation and product development. So, it’s working.
pama 3y ago
I referee for a lot of the top machine learning conferences and yes I am very optimistic about AI and its impact on humanity. The amount of exciting new papers in machine learning and AI was definitely on an exponential rise for a decade since about 2012 or so, and the total production has kept increasing even during the last couple of years when the submissions in some top annual conferences exceeded 10k. Not every paper results in a useable model but a higher fraction of papers come with code and pretrained weights over time. Many of these papers will never be read by many more than the reviewers and the group who wrote them and a couple friends, but it does not speak necessarily to the quality of the work itself or the potential impact it could have on every possible future if we found better ways to separate the useful information. As the exponential increase in total compute becomes more widely accessible there are exponentially more applications that are of broader interest and will have even bigger impact than nowadays. I don’t think that the model of reviewing 10s or 100s of thousands of papers in conferences, or playing the popularity contest on social media is going to be productive so we need better methods for advancing the useful ideas more quickly. (Case in point: the mamba state space model by Gu and Dao was rejected from a conference this winter, but it happened to be advertised enough at a keynote presentation by Chris Re with a packed audience at neurIPS23, so the model was picked up by a lot of people who used it and submitted applications that used it to the ICML conference already.) I also don’t think that some of the biggest companies have enough manpower, motivation and interest in going alone, though of course they can easily stay ahead of the game in specialized areas with their own resources.

int_19h 3y ago

That doesn't follow at all. It just means that there are still low-hanging fruits to pursue for better (smarter, faster, larger context etc) new models, but it doesn't say anything about the stability and usefulness of existing models.

nickpsecurity 3y ago

That’s not true. Both good science and market-driven engineering favor continued iterations on existing ideas looking for improvements or alternatives. We’re often exploring a giant space of solutions. Unlike many fields, the A.I. people are publicly posting many of their steps in this journey, their iterations, for review. While it brings lots of fluff, such openness dramatically increases innovation rate compared to fields where you only see results once or twice a year. Both people using cloud API’s and FOSS developers are steadily increasing effectiveness in both experimentation and product development. So, it’s working.

pama 3y ago

I referee for a lot of the top machine learning conferences and yes I am very optimistic about AI and its impact on humanity. The amount of exciting new papers in machine learning and AI was definitely on an exponential rise for a decade since about 2012 or so, and the total production has kept increasing even during the last couple of years when the submissions in some top annual conferences exceeded 10k. Not every paper results in a useable model but a higher fraction of papers come with code and pretrained weights over time. Many of these papers will never be read by many more than the reviewers and the group who wrote them and a couple friends, but it does not speak necessarily to the quality of the work itself or the potential impact it could have on every possible future if we found better ways to separate the useful information. As the exponential increase in total compute becomes more widely accessible there are exponentially more applications that are of broader interest and will have even bigger impact than nowadays. I don’t think that the model of reviewing 10s or 100s of thousands of papers in conferences, or playing the popularity contest on social media is going to be productive so we need better methods for advancing the useful ideas more quickly. (Case in point: the mamba state space model by Gu and Dao was rejected from a conference this winter, but it happened to be advertised enough at a keynote presentation by Chris Re with a packed audience at neurIPS23, so the model was picked up by a lot of people who used it and submitted applications that used it to the ICML conference already.) I also don’t think that some of the biggest companies have enough manpower, motivation and interest in going alone, though of course they can easily stay ahead of the game in specialized areas with their own resources.