19 ms·

Is there actual overconfidence? I think that if just in 2020 we did a survey about whether the capabilities of (for example) the image generation systems discu

Is there actual overconfidence? I think that if just in 2020 we did a survey about whether the capabilities of (for example) the image generation systems discussed in other today's threads would materialize at 2022, then more than half of deep learning technologists would have said that it's not going to happen so fast, demonstrating underconfidence instead of overconfidence. For another example, when people in NLP put up tasks like the GLUE dataset, a reasonable expectation was that it would be a meaningful benchmark for some time, but it turned out that the previously too hard tasks became too easy just a year or so later - again, demonstrating underconfidence instead of overconfidence. IMHO the overconfidence appears when people listen to PR of various companies overpromising products, not to the actual technologists doing research on the technology.

frgtpsswrdlame 4y ago
Maybe the problem is trying to understand this in terms of confidence and not in terms of accuracy. In fact, we might say that some are underconfident, some are overconfident but almost all are inaccurate in predicting the form and/or details of progress. In that case we don't have to care about the technologists who thought the recent wave of image generation would take 10+ years AND we don't have to care about the people who think evil AI will bury us under paperclips in the next 30.
logisticseh 4y ago
I agree. There are two different sets: 1. Actual deep learning researchers, actively doing research. I rarely hear nonsense about impending AGI or mass technological unemployment from these folks. They might express tepid concern about the ethics of various applications of deep learning and tepidly point out that some technological disruption in employment is possible, but in both cases with an emphasis on tepid. And who, if anything, tend to under-estimate progress. (Self-driving is still very far off and also much further along than I thought it would be 9 years ago. I turned down jobs at self-driving cos because I didn't believe they would make enough progress to even have an ADAS product, but I was clearly wrong and if I could do it over I'd work on self-driving from 2013 onwards even though L5 is still a long way out.) 2. Deep learning fan-boys, for lack of a better term. The rationalist community in particular has a sub-community of tech-adjacent folks who aren't publishing in major conferences every cycle or running research labs but do talk a lot about AGI/UBI. IMO it's not that dissimilar from climate science or even in the extreme the existence of aliens. Scientists with a lot of real expertise will sort of tepidly talk you through the full complexity. And then some "true believers" who aren't actually expert will sort of run to the extremes of anti-natalism or aliens among us. If that makes sense.
- porknubbins 4y ago
  I guess John Carmack falls somewhere in between but as a complete layperson when he said he gave AGI by 2030 a 50 percent chance, that at least indicated to me that some really smart people think there is a chance.
  - logisticseh 4y ago
    1. He predicts "signs of life" by 2030, which is a (probably intentionally) vague statement. 2. He raised $20MM for an AI startup, which is fine and well but also makes him not entirely disincentivized from hype. 3. I wouldn't characterize him as someone in the trenches of deep learning. More of a meta point: technical depth in more than a few things is impossible in a human lifespan, and just a bit harder once you become a "somebody" since a portion of your life becomes consumed by the fact that you're a "somebody". You end up doing things like raising VC money and starting companies with bold ambitions. Its own time sink. I had this realization when I had a conversation with Lamport about a niche topic in distributed systems and he expressed a position that was just wrong. It was a minor point that didn't really matter much at all, but he was pretty confident in a conjecture I knew was wrong. To be clear, the fact that no one can be an expert on everything -- even everything within a subfield of CS -- doesn't detract from the fact that geniuses exist. Someone can "forget more than you know" and also not know something that you know. Life is just sadly very short.
    - porknubbins 4y ago
      Thats true, he seems to have flopped on the rockets startup too, although you might argue AI is still much closer to his wheelhouse than aerospace so it make more sense. Before hearing that I had some vague idea that the most AI experts timeline for AI was significantly longer and some never.
- Mordisquitos 4y ago
  > IMO it's not that dissimilar from climate science or even in the extreme the existence of aliens. Those may be similar to the contrasting attitudes regarding AI, but the analogy I first thought of for what you describe is the topic of cryptocurrency. 1. On the one hand, most people who have deep knowledge and experience of software development and databases, or professional experience of financial markets and banking, tend to be extremely sceptical, critical, or outright dismissive of the whole idea of cryptocurrency. 2. On the other, people who are technology "enthusiasts" and have some limited or self-taught programming skills, or those who have some moderate knowledge of the basics of finance and investments (often motivated by personal ambition), are much more likely to be cryptocurrency fan-boys.
mountainriver 4y ago
Agree, deep learning seems to be exceeding all most expectations over the last decade. Only time it doesn’t is when some vc funded startup is making insane claims. GPT-3 and stable diffusion are much more capable than most predicted
- AstralStorm 4y ago
  Much more capable at what?
legulere 4y ago
I think the big issue is that we have bad intuition about what works well with machine learning and what doesn't, leading to both under- and overconfidence at the same time. When I was at university I came across machine learning in medical imaging. The job of radiologists seemed obvious to be a classic machine learning problem that should be easily solved by the upcoming possibilities. In the end other stuff materialized.
- gpm 4y ago
  Do you happen to know what stopped the radiologist stuff from happening? I know I read press releases ages ago saying "hey look, we can do this machine learning", but I haven't really heard anything about it since.
  - sdenton4 4y ago
    My impression was that medical data access for training and eval was the really big thing holding it back... You wind up with deep learning folks obsessing over datasets with tens of examples.
  - labcomputer 4y ago
    Part of it is medical regulation. For example, only a human doctor can make a "diagnosis" or write a prescription. Even just making "treatment recommendations" (e.g., a more-effective treatment or one with less side effects than the current one), which would then be reviewed and approved by an appropriately-licensed physician, is a problem. Not having worked in the radiology space, my naive hunch is that it has to be framed as a "screening tool" which "helps the physicians find cancers." (Which can be justified to regulators because ML tends to have higher recall than humans on these tasks, albeit lower precision). In theory, ML helps doctors spot cancers they wouldn't ordinarily see. In practice, it probably means that the physicians still technically "review" all images, but only give a cursory glance to those not flagged by the system.
    - naijaboiler 4y ago
      I am a doctor and an AI practioner. A radiologists job is far far more than pathology detector. No AI will replace the image recongition aspect of their job in my lifetime. And that's the most automatable aspect of their job. The human and legal aspects of their job is a whole different beast. At best, technologies will allow radiologists to better or more efficient
JohnHaugeland 4y ago
remember that time that lesswrong proved agi would be present by 2014? you're talking about the people that came up with roko's basilisk, which is religion / d&d tier overconfident
- mindcrime 4y ago
  WTF does LessWrong have to do with anything? Is anybody associated with LessWrong an actual researcher, as opposed to just an armchair philosopher? And even if some token number are, they hardly represent any kind of field-wide consensus.
  - JohnHaugeland 4y ago
    > WTF does LessWrong have to do with anything? It's an example of the previous discussion: people talking about AI in the ridiculously infinitive, in threads. Nothing there says "practicing researchers in papers." I even agree with what appears to be your evaluation of LessWrong folks, but the thing being asked didn't gate the discussion to people who /should/ /be/ in the discussion, and AI's discussion is about as weighed down by outsider fakers as politics or earth-shapeology is I mean, a discussion of the discussion of vaccines has to include the cranks, doesn't it? As much as they shouldn't be there, they very much are, and the impact they're having is very real In case there's any inclarity, LessWrong are the people that got famous for writing incorrect parables about logic principles in the actual tone of Harry Potter fan fiction, an act I can't even make fun of because it's funnier than any of my jokes about it. I had not intended the HPMoR authors as a north star reference. I had actually indicated them to remind us that the vast majority of the AI discussion isn't by AI people. In the meantime, even if you gate it to real AI people, it'd be pretty surprising if you can't name any overconfidence. How many different years has Kurzweil assigned the proven birth of AI to? What did Cyc promise? When did Tesla self-driving first miss its mark? Do you remember the promises they used to make about Siri? Am I the only person who remembers that Cue:CAT was sold as an AI device, as was Teddy Ruxpin? Who here remembers IBM Watson? Remember when AI was going to solve credit card fraud? Remember when AI was going to defeat child pornography? It's not like it was rare. There was a toothbrush called the Ara that was going to use AI to solve cavities. AIs can see your heart attack five years in advance! AIs can predict psychosis from how you write. AIs can pick out criminals by their faces! AI can tell if you're gay. And then like, you try it in the field, and chihuahua vs blueberry muffin is still a struggle. Tanks and time of day, you know? AI has more reliably lampooned itself with public promises and failures than any other industry I can think of offhand except maybe Hollywood, and I live on the planet with "fusion is just 10 years away" as a slogan Also I'm in AI, and I'm responsible for a few of those failures. Go team! Let's promise the moon next, then deliver them a Lego Challenger Shuttle playset a year late. Don't get me wrong - Stable Diffusion is literal, actual magic. I'm scared for emad, that Lord Oberon is going to beat down his door one night and take it back, and possibly turn him into a tree. He's violated the "the humans turned that star into a donut and we don't actually know how" rule. https://imgur.com/gallery/wpZ4w https://imgur.com/gallery/wpZ4w But also, AI was going to have given us self-driving cars before Tesla was founded, and I think people have just forgotten that I mean we haven't even lived up to the 1990s AT&T commercials yet. It's really weird to be this far in the future and this far in the past at the same time. It's almost 2023 and I'm still yelling "no" into the phone four times in a row to get the menu to understand a single syllable, and barely half of that is my mobile carrier's fault . > Is anybody associated with LessWrong an actual researcher I have no idea. I don't read that stuff. I just like making fun of Roko's Basilisk. If I was going to bet money, I'd put 100% on no, but also, in a community that size, y've gotta figure they got at least a couple over the years? I'd bet fewer than a quarter as many as you'd expect from just a random sampling of the populace. I mean. Also let's remember there have been more than a few questionable researchers over the years. I'd probably get flagged if I Bogdanoved a list here, but, look, if you're at a quack convention, a fringe "researcher" is gonna show up for some credibility trading, right? And also Gwern just said he was there since 2014 in a different thread, and he's the very much the real deal, so I guess the answer is "at least one apparently." Anyway, if you can't laugh at Roko's Basilisk, buddy, I feel for you. That's eternal comedy gold. There are 1980s cartoon villains who broke the fourth wall and time travelled to the modern day just to make fun of it. You can write it on two pounds of paper and get the gold exchanged for money at most banks. Gary Larson cannot be reached for comment. If you don't know what it means, pretty please look it up; I promise you that it's worth your time. I didn't say D&D tier for no reason. They genuinely large-group panicked that a scary AI from the future of an alternate universe was going to come back in time to punish them to death for not dumping all their money into speculatively inventing it. Like I'm not even kidding, that's really what it is. I was definitely not invoking Harry Potter and the Basilisk of Rationality as a bastion of credibility. Pinky swear. I'm not even spoiling it; that's just the beginning. I didn't get to the funny part. Please, set up some warm candle light, some soft music, a couple of your favorite beverages, and a bucket. You're in for a wild ride. (Two points, Quidditch themed presumably, to anyone who chooses to interpret Basilisk in the Carolingian sense.) Always remember: the discussion about Foo is not limited to Foo professionals
gwern 4y ago
> Is there actual overconfidence? There isn't. Arvind is just making this up. (Anytime a post leads with many controversial major claims and then handwaves away the entire discussion with "We won’t waste your time arguing with these claims.", while providing almost zero references for anything at all, you should smell something rotten. No no, go right ahead, feel free to 'waste our time', you're not charging us by the word. By all means, do discuss recent DL work on time-series forecasting and why DL is unable to predict, the industrial/organizational psychology lit on predicting job performance, prediction of recividism, why you think scaling is now the dominant paradigm both in terms of researcher headcount and budgets, etc - we're all ears!) And we know this in part because surveys regularly ask about relevant questions like AGI timelines (median is still ~2050, which implies most believe scaling won't work) or show that AGI & scaling critics are still in the majority, despite posturing as an oppressed minority; most recently: "What Do NLP Researchers Believe? Results Of The NLP Community Metasurvey", Michael et al 2022 https://arxiv.org/abs/2208.12852 https://arxiv.org/abs/2208.12852 If you believe in scaling, you are still in a small minority of researchers pursuing an unpopular and widely-criticized paradigm. (That it is still producing so many incredible results and appearing so dominant despite being so disliked and small is, IMO, to its credit and one of the best arguments for why new researchers should go into scaling - it is still underrated.)
- scarmig 4y ago
  Why is the scaling hypothesis so disliked? I guess "we just need MOAR LAYERS" comes off as a bit naive and implies a lot of extant research programs are dead ends, so there are elements of researchers not wanting to appear to be AI techbros and of mutual backscratching. But it's proven to be surprisingly productive for the last few years, so I'd have expected more people to jump on if only out of hype.
  - sinenomine 4y ago
    > Why is the scaling hypothesis so disliked? This is a difficult question. I have several hypotheses. When one works with highly productive/prestigious/famous people, one can't help but notice[1] how they instinctively avoid subproblems in their niche which don't let them leverage their intelligence/energy/fame to a sufficient degree and/or don't give their personal brand enough credit. This dynamic is at play in academia and in tech corporations as well (though arguably, tech is still more amenable to large-scale engineering). These people live by (public perception of) credit and have very fine sense for it. Scaling could be (during the recent history, at least, maybe increasingly not so in the near future as the obvious breakthrough results became widely known) an inconvenient lever for applying one's academic presence simply because there is little prestige behind it, and (at least as understood by academics) it's merely engineering work, which doesn't require true genius. Naturally, academics are led to creation of sophisticated theories, or even whole fields (even better, because it's hard to measure the output in this case), preemptively predicting and deflecting as much criticism as possible - and view such efforts and their conceptual output as a mark of true intelligence. Engineering large-scale systems executing gradient-descent doesn't really look valuable from such point of view, because it tells little about the brilliance of the chief investigator. 1. http://yosefk.com/blog/10x-more-selective.html http://yosefk.com/blog/10x-more-selective.html
mindcrime 4y ago
Is there actual overconfidence? No. This is pretty much made-up bullshit. OK, there may be "overconfidence" among spectators and people on the fringes, commenting from the peanut gallery. But among people actually doing real meaningful research? No, this whole story is a big nothing-burger.

frgtpsswrdlame 4y ago

Maybe the problem is trying to understand this in terms of confidence and not in terms of accuracy. In fact, we might say that some are underconfident, some are overconfident but almost all are inaccurate in predicting the form and/or details of progress. In that case we don't have to care about the technologists who thought the recent wave of image generation would take 10+ years AND we don't have to care about the people who think evil AI will bury us under paperclips in the next 30.

logisticseh 4y ago

I agree. There are two different sets: 1. Actual deep learning researchers, actively doing research. I rarely hear nonsense about impending AGI or mass technological unemployment from these folks. They might express tepid concern about the ethics of various applications of deep learning and tepidly point out that some technological disruption in employment is possible, but in both cases with an emphasis on tepid. And who, if anything, tend to under-estimate progress. (Self-driving is still very far off and also much further along than I thought it would be 9 years ago. I turned down jobs at self-driving cos because I didn't believe they would make enough progress to even have an ADAS product, but I was clearly wrong and if I could do it over I'd work on self-driving from 2013 onwards even though L5 is still a long way out.) 2. Deep learning fan-boys, for lack of a better term. The rationalist community in particular has a sub-community of tech-adjacent folks who aren't publishing in major conferences every cycle or running research labs but do talk a lot about AGI/UBI. IMO it's not that dissimilar from climate science or even in the extreme the existence of aliens. Scientists with a lot of real expertise will sort of tepidly talk you through the full complexity. And then some "true believers" who aren't actually expert will sort of run to the extremes of anti-natalism or aliens among us. If that makes sense.

porknubbins 4y ago
I guess John Carmack falls somewhere in between but as a complete layperson when he said he gave AGI by 2030 a 50 percent chance, that at least indicated to me that some really smart people think there is a chance.
- logisticseh 4y ago
  1. He predicts "signs of life" by 2030, which is a (probably intentionally) vague statement. 2. He raised $20MM for an AI startup, which is fine and well but also makes him not entirely disincentivized from hype. 3. I wouldn't characterize him as someone in the trenches of deep learning. More of a meta point: technical depth in more than a few things is impossible in a human lifespan, and just a bit harder once you become a "somebody" since a portion of your life becomes consumed by the fact that you're a "somebody". You end up doing things like raising VC money and starting companies with bold ambitions. Its own time sink. I had this realization when I had a conversation with Lamport about a niche topic in distributed systems and he expressed a position that was just wrong. It was a minor point that didn't really matter much at all, but he was pretty confident in a conjecture I knew was wrong. To be clear, the fact that no one can be an expert on everything -- even everything within a subfield of CS -- doesn't detract from the fact that geniuses exist. Someone can "forget more than you know" and also not know something that you know. Life is just sadly very short.
  - porknubbins 4y ago
    Thats true, he seems to have flopped on the rockets startup too, although you might argue AI is still much closer to his wheelhouse than aerospace so it make more sense. Before hearing that I had some vague idea that the most AI experts timeline for AI was significantly longer and some never.
Mordisquitos 4y ago
> IMO it's not that dissimilar from climate science or even in the extreme the existence of aliens. Those may be similar to the contrasting attitudes regarding AI, but the analogy I first thought of for what you describe is the topic of cryptocurrency. 1. On the one hand, most people who have deep knowledge and experience of software development and databases, or professional experience of financial markets and banking, tend to be extremely sceptical, critical, or outright dismissive of the whole idea of cryptocurrency. 2. On the other, people who are technology "enthusiasts" and have some limited or self-taught programming skills, or those who have some moderate knowledge of the basics of finance and investments (often motivated by personal ambition), are much more likely to be cryptocurrency fan-boys.

mountainriver 4y ago

Agree, deep learning seems to be exceeding all most expectations over the last decade. Only time it doesn’t is when some vc funded startup is making insane claims. GPT-3 and stable diffusion are much more capable than most predicted

AstralStorm 4y ago
Much more capable at what?

legulere 4y ago

I think the big issue is that we have bad intuition about what works well with machine learning and what doesn't, leading to both under- and overconfidence at the same time. When I was at university I came across machine learning in medical imaging. The job of radiologists seemed obvious to be a classic machine learning problem that should be easily solved by the upcoming possibilities. In the end other stuff materialized.

gpm 4y ago
Do you happen to know what stopped the radiologist stuff from happening? I know I read press releases ages ago saying "hey look, we can do this machine learning", but I haven't really heard anything about it since.
- sdenton4 4y ago
  My impression was that medical data access for training and eval was the really big thing holding it back... You wind up with deep learning folks obsessing over datasets with tens of examples.
- labcomputer 4y ago
  Part of it is medical regulation. For example, only a human doctor can make a "diagnosis" or write a prescription. Even just making "treatment recommendations" (e.g., a more-effective treatment or one with less side effects than the current one), which would then be reviewed and approved by an appropriately-licensed physician, is a problem. Not having worked in the radiology space, my naive hunch is that it has to be framed as a "screening tool" which "helps the physicians find cancers." (Which can be justified to regulators because ML tends to have higher recall than humans on these tasks, albeit lower precision). In theory, ML helps doctors spot cancers they wouldn't ordinarily see. In practice, it probably means that the physicians still technically "review" all images, but only give a cursory glance to those not flagged by the system.
  - naijaboiler 4y ago
    I am a doctor and an AI practioner. A radiologists job is far far more than pathology detector. No AI will replace the image recongition aspect of their job in my lifetime. And that's the most automatable aspect of their job. The human and legal aspects of their job is a whole different beast. At best, technologies will allow radiologists to better or more efficient

JohnHaugeland 4y ago

remember that time that lesswrong proved agi would be present by 2014? you're talking about the people that came up with roko's basilisk, which is religion / d&d tier overconfident

mindcrime 4y ago
WTF does LessWrong have to do with anything? Is anybody associated with LessWrong an actual researcher, as opposed to just an armchair philosopher? And even if some token number are, they hardly represent any kind of field-wide consensus.
- JohnHaugeland 4y ago
  > WTF does LessWrong have to do with anything? It's an example of the previous discussion: people talking about AI in the ridiculously infinitive, in threads. Nothing there says "practicing researchers in papers." I even agree with what appears to be your evaluation of LessWrong folks, but the thing being asked didn't gate the discussion to people who /should/ /be/ in the discussion, and AI's discussion is about as weighed down by outsider fakers as politics or earth-shapeology is I mean, a discussion of the discussion of vaccines has to include the cranks, doesn't it? As much as they shouldn't be there, they very much are, and the impact they're having is very real In case there's any inclarity, LessWrong are the people that got famous for writing incorrect parables about logic principles in the actual tone of Harry Potter fan fiction, an act I can't even make fun of because it's funnier than any of my jokes about it. I had not intended the HPMoR authors as a north star reference. I had actually indicated them to remind us that the vast majority of the AI discussion isn't by AI people. In the meantime, even if you gate it to real AI people, it'd be pretty surprising if you can't name any overconfidence. How many different years has Kurzweil assigned the proven birth of AI to? What did Cyc promise? When did Tesla self-driving first miss its mark? Do you remember the promises they used to make about Siri? Am I the only person who remembers that Cue:CAT was sold as an AI device, as was Teddy Ruxpin? Who here remembers IBM Watson? Remember when AI was going to solve credit card fraud? Remember when AI was going to defeat child pornography? It's not like it was rare. There was a toothbrush called the Ara that was going to use AI to solve cavities. AIs can see your heart attack five years in advance! AIs can predict psychosis from how you write. AIs can pick out criminals by their faces! AI can tell if you're gay. And then like, you try it in the field, and chihuahua vs blueberry muffin is still a struggle. Tanks and time of day, you know? AI has more reliably lampooned itself with public promises and failures than any other industry I can think of offhand except maybe Hollywood, and I live on the planet with "fusion is just 10 years away" as a slogan Also I'm in AI, and I'm responsible for a few of those failures. Go team! Let's promise the moon next, then deliver them a Lego Challenger Shuttle playset a year late. Don't get me wrong - Stable Diffusion is literal, actual magic. I'm scared for emad, that Lord Oberon is going to beat down his door one night and take it back, and possibly turn him into a tree. He's violated the "the humans turned that star into a donut and we don't actually know how" rule. https://imgur.com/gallery/wpZ4w https://imgur.com/gallery/wpZ4w But also, AI was going to have given us self-driving cars before Tesla was founded, and I think people have just forgotten that I mean we haven't even lived up to the 1990s AT&T commercials yet. It's really weird to be this far in the future and this far in the past at the same time. It's almost 2023 and I'm still yelling "no" into the phone four times in a row to get the menu to understand a single syllable, and barely half of that is my mobile carrier's fault . > Is anybody associated with LessWrong an actual researcher I have no idea. I don't read that stuff. I just like making fun of Roko's Basilisk. If I was going to bet money, I'd put 100% on no, but also, in a community that size, y've gotta figure they got at least a couple over the years? I'd bet fewer than a quarter as many as you'd expect from just a random sampling of the populace. I mean. Also let's remember there have been more than a few questionable researchers over the years. I'd probably get flagged if I Bogdanoved a list here, but, look, if you're at a quack convention, a fringe "researcher" is gonna show up for some credibility trading, right? And also Gwern just said he was there since 2014 in a different thread, and he's the very much the real deal, so I guess the answer is "at least one apparently." Anyway, if you can't laugh at Roko's Basilisk, buddy, I feel for you. That's eternal comedy gold. There are 1980s cartoon villains who broke the fourth wall and time travelled to the modern day just to make fun of it. You can write it on two pounds of paper and get the gold exchanged for money at most banks. Gary Larson cannot be reached for comment. If you don't know what it means, pretty please look it up; I promise you that it's worth your time. I didn't say D&D tier for no reason. They genuinely large-group panicked that a scary AI from the future of an alternate universe was going to come back in time to punish them to death for not dumping all their money into speculatively inventing it. Like I'm not even kidding, that's really what it is. I was definitely not invoking Harry Potter and the Basilisk of Rationality as a bastion of credibility. Pinky swear. I'm not even spoiling it; that's just the beginning. I didn't get to the funny part. Please, set up some warm candle light, some soft music, a couple of your favorite beverages, and a bucket. You're in for a wild ride. (Two points, Quidditch themed presumably, to anyone who chooses to interpret Basilisk in the Carolingian sense.) Always remember: the discussion about Foo is not limited to Foo professionals

gwern 4y ago

> Is there actual overconfidence? There isn't. Arvind is just making this up. (Anytime a post leads with many controversial major claims and then handwaves away the entire discussion with "We won’t waste your time arguing with these claims.", while providing almost zero references for anything at all, you should smell something rotten. No no, go right ahead, feel free to 'waste our time', you're not charging us by the word. By all means, do discuss recent DL work on time-series forecasting and why DL is unable to predict, the industrial/organizational psychology lit on predicting job performance, prediction of recividism, why you think scaling is now the dominant paradigm both in terms of researcher headcount and budgets, etc - we're all ears!) And we know this in part because surveys regularly ask about relevant questions like AGI timelines (median is still ~2050, which implies most believe scaling won't work) or show that AGI & scaling critics are still in the majority, despite posturing as an oppressed minority; most recently: "What Do NLP Researchers Believe? Results Of The NLP Community Metasurvey", Michael et al 2022 https://arxiv.org/abs/2208.12852 https://arxiv.org/abs/2208.12852 If you believe in scaling, you are still in a small minority of researchers pursuing an unpopular and widely-criticized paradigm. (That it is still producing so many incredible results and appearing so dominant despite being so disliked and small is, IMO, to its credit and one of the best arguments for why new researchers should go into scaling - it is still underrated.)

scarmig 4y ago
Why is the scaling hypothesis so disliked? I guess "we just need MOAR LAYERS" comes off as a bit naive and implies a lot of extant research programs are dead ends, so there are elements of researchers not wanting to appear to be AI techbros and of mutual backscratching. But it's proven to be surprisingly productive for the last few years, so I'd have expected more people to jump on if only out of hype.
- sinenomine 4y ago
  > Why is the scaling hypothesis so disliked? This is a difficult question. I have several hypotheses. When one works with highly productive/prestigious/famous people, one can't help but notice[1] how they instinctively avoid subproblems in their niche which don't let them leverage their intelligence/energy/fame to a sufficient degree and/or don't give their personal brand enough credit. This dynamic is at play in academia and in tech corporations as well (though arguably, tech is still more amenable to large-scale engineering). These people live by (public perception of) credit and have very fine sense for it. Scaling could be (during the recent history, at least, maybe increasingly not so in the near future as the obvious breakthrough results became widely known) an inconvenient lever for applying one's academic presence simply because there is little prestige behind it, and (at least as understood by academics) it's merely engineering work, which doesn't require true genius. Naturally, academics are led to creation of sophisticated theories, or even whole fields (even better, because it's hard to measure the output in this case), preemptively predicting and deflecting as much criticism as possible - and view such efforts and their conceptual output as a mark of true intelligence. Engineering large-scale systems executing gradient-descent doesn't really look valuable from such point of view, because it tells little about the brilliance of the chief investigator. 1. http://yosefk.com/blog/10x-more-selective.html http://yosefk.com/blog/10x-more-selective.html

mindcrime 4y ago

Is there actual overconfidence? No. This is pretty much made-up bullshit. OK, there may be "overconfidence" among spectators and people on the fringes, commenting from the peanut gallery. But among people actually doing real meaningful research? No, this whole story is a big nothing-burger.