8 ms·

Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spiraled out of control’

110 points by CharlesW 3y ago

8thcross 3y ago
such a title!
version_five 3y ago
Not that I disagree, but as I've said before, it's easy to be a theoretical contrarian, as in claim something is BS but not stake anything on it or profit from it (other than book deals). We're in a housing bubble, at the peak of a multi-century economic mega-cycle, we're overdue for earthquakes, tsunamis, asteroids, the caldera under Yellowstone, flu pandemics, etc etc. It's super easy to say all that and demand people should listen, what's hard is committing to any actionable prediction.
- lordkirchner 3y ago
  All I ever wanted was a way to short NFTs
  - lokar 3y ago
    Never short a ponzi, even if you win your counterparty won’t pay
- piloto_ciego 3y ago
  Being a pessimist is academically safe and intellectually lazy. If you’re wrong in your predictions nobody cares because it’s better than if you’re right. On the other hand, if you’re an optimist and you are wrong you look like an idiot and the social penalty is higher. So people continue to predict doomsday and most of the time it doesn’t happen.
  - md_ 3y ago
    > So people continue to predict doomsday and most of the time it doesn’t happen. Well, the same is true for any prediction of anything unlikely. People continue to predict world-changing innovations, and most of the time they are wrong. That's the reality of any low-probability/high-impact prediction, in the positive or negative direction. As a specific case of that: most startups fail, so predicting a startup will fail is safe. I'm pretty prone to exactly that prediction. But I like this quote from Erik Davis (https://www.google.ch/books/edition/High_Weirdness/Rcq2DwAAQBAJ https://www.google.ch/books/edition/High_Weirdness/Rcq2DwAAQ...): "In the court of the mind, skepticism makes a great grand vizier, but a lousy lord."
JieJie 3y ago
It's not really about "ai hype", it's about funding. "In the last few months, there has been this increasing so-called rift between the AI ethics and AI safety communities. There is a lot of talk about how this is an academic rift that needs to be resolved, how these communities are basically aiming for the same purpose. I think the thing that annoys me most about the discourse around this is that people don’t recognize this as a power struggle. It is not really about intellectual merit of these ideas. Of course, there are lots of bad intellectual and academic claims that have been made on both sides. But that isn’t what this is really about. It’s about who gets funding, which concerns are prioritized. So looking at it as if it is like a clash of individuals or a clash of personalities just really undersells the whole thing, makes it sound like people are out there bickering, whereas in fact, it’s about something much deeper."
- deleted 3y ago
  [deleted]
ajcp 3y ago
All snark aside it must be really tough to make a name for yourself by calling something snake oil only to have a flavor of it become massively successful...and then have to double down because you already had a book deal to fulfill.
- deleted 3y ago
  [deleted]
ftxbro 3y ago
They were wrong and now they have to pivot, from "AI isn't effective" to "it's bad that AI is so effective."
- deleted 3y ago
  [deleted]
Someone1234 3y ago
The key way I try to distinguish "Snake Oil" is: Is anyone actually using this routinely, and are the problems it solves self-evident to new non-zealot users after a demo? Take blockchain for example; people spent years and millions of dollars looking for additional usages for the technology beside coins/coin-contracts; and aside from the forementioned usages it didn't really gain routine adoption. That was why the non-coin-blockchain always struct me as a "hype train" or "snake oil" because it was a solution in search of a problem. NFTs are even worse since they haven't found any purpose to exist yet (money laundering?). Contrast that with the "GPT" offerings (ChatGPT/Bing/Copilot/Bard/etc). Multiple of my colleagues are actively using them routinely every workday and when you demo it to another person, they understand it, and they too start utilizing it in their workflow. Heck, my mom discovered it herself on Bing's homepage and was telling me I should check it out. That's the opposite of "snake oil" or a hype train, it is arguably a competitive threat to search engines. PS - Small disclaimer: "AI" is a nebulous term. I'm talking specifically about LLMs. Other types of "AI" have already been here for a long time making our lives better (e.g. computer vision).
- lIIllIIllIIllII 3y ago
  I'm routinely using LLMs in place of search engines. Would you rather type in a cocktail and instantly get a recipe? or do you want to open google, scroll past 50 ads, click through a bunch of blogspam that gives you the history of cocktails, the prohibition, the chemical formula for ethanol and their life story, followed by a recipe that was itself generated by a substandard AI model anyway edit: I'm prepared for the enshittification of GenAI, but given competitive edge, many people will pay top $ for faster/higher parameter inference, I'm hoping ad support is minimal on the super deluxe god tier AI subscription which I'll have to pile all of my spare cash into.
  - danmur 3y ago
    If you pay for search you get some of those benefits already
    - chargingmarmot 3y ago
j45 3y ago
Does Princeton not have any meaningful generative AI work or research going on?
- calf 3y ago
  I went to Princeton, I can assure you Narayanan is only one professor among the many brilliant STEM scientists there.
clircle 3y ago
Analysis: true, six months ago
TuringNYC 3y ago
Everyone should read this Reddit comment: https://www.reddit.com/r/blender/comments/121lhfq/i_lost_everything_that_made_me_love_my_job/ https://www.reddit.com/r/blender/comments/121lhfq/i_lost_eve... This is happening, and the reddit thread is just a single instance of a wide phenomenon. I wonder how one can square "snakeoil" theories against reality like this.
- flangola7 3y ago
  My company has openly bragged they predicted they would need to hire over 200 employees this year, and AI systems meant they only had to hire 45.
- numpad0 3y ago
  Maybe the reality isn't evenly distributed. I'm supposedly at the working end of that job and can't immediately recall seeing the result of that transition.
- slfnflctd 3y ago
  > just a single instance of a wide phenomenon Another instance is the 'hey pi' chatbot. For some people it's a superior experience to BetterHelp, or even a lot of so-called professional therapists out there. The lack of actual human presence is a bit deflating for me, but I still got a good session out of it once. It feels like it has the potential to be at least better than nothing for some of the lonely people out there. The option for different voices is a nice touch that adds to the illusion. There are so many possibilities. I'm looking forward to seeing what happens with everything from NPCs in video games to robotics (among other things, to actually explain what they're doing & why, and converse). Not to mention the applications in education and health care. Anyone who thinks this is a flash in the pan has not observed enough of what's going on.
- satvikpendem 3y ago
  Most jobs in human history were not labors of passion, it is only in very recent history that it is so. We should not expect it to be the case that people will love their job, as not all loves are employable, as OP in the thread is finding out. While OP might dislike it, there are many others who will fill their space if OP leaves that job or industry, those that actually enjoy what they do. Contrast that to the people on https://old.reddit.com/r/StableDiffusion https://old.reddit.com/r/StableDiffusion, where people willingly experiment with multiple approaches to media generation. At least some of those people will be interested in 3D modeling and will willingly take OP's job, because they genuinely enjoy it and OP now does not. In the end, adapt, or others who do will take over. That's been the historical precedent for millennia, and it will not stop now.
atleastoptimal 3y ago
My hypothesis is that a lot of high-status intellectuals are reflexively dismissing AI out of fear that it threatens their status, as AI poses to turn intellectual labor into a cheap commodity rather than the premium bespoke product they offer it as via their professorships, speaking fees and book sales. Most criticism of AI has all the hallmarks of coping mechanisms. They seem to shift quickly between declaring it an ineffective parlor trick, and then say it's so good and effective that it poses a risk to the human spirit. There are a lot of legitimate criticisms I've seen, especially of OpenAI's potential regulation capture and misuse of models, but calling it all snake oil is hilariously naïve and seems more like a fear of change.
- deleted 3y ago
  [deleted]
- kromem 3y ago
  Really? You think people who write for a living might be biased towards confirming that the thing already replacing many people who write for a living is a fad that's going to blow over soon? My favorite part with crap predicting doom is the sheer unawareness of the rate of change of the rate of change. If you had asked me 18 months ago how long after GPT-3 until something existed with the capabilities of GPT-4, I'd probably have guessed about 5 years. If you asked me 5 years ago how long until an AI could explain why a joke was funny, I'd have maybe guessed at least a decade or two, if it was even possible. If you asked me 10 years ago if I'd see AI replacing artists or copywriters in my lifetime, I'd have guessed maybe when I was in a retirement home (I'm still a fair ways away from that). No one thought what exists today was even possible within our lifetimes a decade back. I'm reminded of the Louis C.K. routine about everything being amazing and no one is happy. Unthinkable AI has already become so normalized that people are predicting its doom based on shortcomings that are less than two years old because AI doing fucking IMPOSSIBLE things (or so everyone thought) is only that old. The rate is outrageous, and while I do think there's currently a significant setback with obsolete alignment approaches being carried forward to models that probably need new techniques, that's going to be a temporary step back in parallel to significant strides forward in the underlying technology from hardware to model design to improved knowledge in how to squeeze the most water from the rock. I just hope when all these folks turn out to have been dead wrong that we don't collectively forget. Futurists should live and die by their record, but too many have goldfish memory and continue to listen to false futurists well after they've shown their own snake oil hand.
phillipcarter 3y ago
The authors' substack is great. They're sort of like investigative journalists who pick a topic that's got a lot of hype and then dissect it to (often very quickly) find that the hyped up topic was just hot air. Some examples: The claims of "liberal bias" being just a completely bunk study: https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bia... No evidence of GPT-4 getting worse over time: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim... The analyses are measured and well written. I always enjoy when a new one comes out.
- bookaway 3y ago
  >No evidence of GPT-4 getting worse over time I read the the article you linked to, and either you're putting words in their mouth and/or making claims that go way beyond the conclusions presented in the article. The authors choose to distinguish between "capability degradation" and "drastic behavior change." Now, almost no one I've witnessed complaining about past and present results of ChatGPT care one bit about making such a distinction. They often simply complain that their previously good result producing prompts are now spewing garbage. The authors explicitly admit that "the kind of fine tuning that LLMs regularly undergo can have unintended effects, including drastic behavior changes on some tasks." i.e. Fine tuning may in fact be causing the worse results. Furthermore, the authors merely make an additional point that such a drastic behavior change shouldn't be assumed to be a "capability degradation." Meaning that, theoretically, if you toy around with your past prompt enough you should be able to eventually find an alternative version of the original prompt that outputs the past result you were looking for. Again, none of the complainers that I've come across really give a shit about this theoretical conclusion. They only care about past prompts now giving different results. In the article, the authors freely admit that this "drastic behavior change" could be objectively occurring due to fine-tuning. To further prevent readers from making sweeping generalizations from their comments, the author's explicitly state that OpenAI makes it pretty damn difficult to do reproducible research on the LLMs in question, so you should not come to any hasty conclusions regardless ("As we have written before, this underscores how hard it is to do reproducible research that uses these APIs, or to build reliable products on top of them.") Well, "building reliable products/processes on top of ChatGPT" is all the complainers care about anyway, so we're back to square one. Far from disproving the claim that ChatGPT is becoming more unreliable and producing worse results for the complainers use cases, the authors are in fact saying that their grievances may be justified for the only practical criteria the naysayers actually give a shit about.
NoZebra120vClip 3y ago
I have stopped using the "snake oil" epithet, because I found out an interesting background on that. Oils derived from various snakes in the desert were found to be effective treatments and remedies for various things by the Native American population, and therefore widely used by certain tribes. Once the White Man caught wind of this, he capitalized on its reputation, and a booming business of scams bloomed anywhere there were gullible buyers. Of course, the snake oil was rarely authentic or effective for its advertised uses. Therefore, snake oil was given a very bad and rather undeserved reputation for the rest of history, and medical science continues to flounder.
- skhr0680 3y ago
  “Snake oil” is a reference to Clark Stanley, whose snake oil was infamously found to contain no snake oil whatsoever.
- ggm 3y ago
  C/F "goanna oil" Australia. Still made. Nowadays, eucalyptus oil, pine oil, peppermint oil, camphor, menthol and turpentine. https://ourstory.moretonbay.qld.gov.au/nodes/view/38026 https://ourstory.moretonbay.qld.gov.au/nodes/view/38026
- deleted 3y ago
  [deleted]
  - deleted 3y ago
    [deleted]
ggm 3y ago
Great to read this back-to-back with the takedown on the GAN image generation offering also on HN. The point of most critiques/polemics on AI Snake oil isn't on effectiveness of AI in context. It's about mis-application, belief its AGI, belief the answers are right, and belief it has no downsides. It's often mis-applied, It's absolutely not, and not even on the road to AGI, the answers are not always right, and it has massive social downsides, employment included. It has upsides, sure. It's not the job of snake oil warners to catalog the good, they're pointing out the abysmally bad, in the current AI hype-wave.
- m00x 3y ago
  Crazy thing is that intelligence doesn't mean you get the right answer. Humans get the answer right fewer times than AI. It's absolutely not AGI, but it's not "not the road to AGI". Every step forward could be the road to AGI.
  - ggm 3y ago
    Could is doing a lot of heavy lifting here. People have been hypothesising about the nature of mind and intelligence for a long time. There is no compelling connectionist, moar-will-work, it-can-learn-to-think model, which infers the nature of intelligence and how that relates to GAN or GPT or any of what people are doing now. What people are doing now is amazing. I love looking at pictures of the pope in puffer jackets. I love "this person does not exist" It could be on the road to AGI, but then, I could win lotto. and I do enter lotto, even knowing the odds. The thing is I don't plan as if I WILL win lotto and a lot of "this AI could be the road to AGI" is about planning as if it will, to secure a slice of the future. It could? yea. But, it isn't.
    - visarga 3y ago
      The future will probably have amazing computers and models, but what we already have has the capacity to profoundly change the world. I don't think you are focusing on what matters most. It's really the language itself that contains the intelligence, not the AI models. The models are just vessels for absorbing all that knowledge encoded in text. Just like human brains - we can have different neural wiring but learn the same things through education. So the huge datasets these AI systems are trained on are key. That's how models like GPT-4 gain such language understanding. The architecture matters less than all that linguistic data it ingests. And language has been evolving for millennia, long before AI. It replicates through culture, speech, writing. Now with AI it has a whole new medium. Fascinating question - how will AI affect language evolution going forward? As models produce more human-like text, which can further train better models, it's like a Lamarckian evolution. Acquired linguistic intelligence gets absorbed by AI then improved and propagated back into the corpus. So while AI tech will keep advancing, it's language evolution that's most profound. By creating this new way for language to evolve, AI could really reshape cultural evolution. Since language enables intelligence, it'll have big impacts on where AI is headed next.
xen2xen1 3y ago
My washing machine says "Optimized by AI". My clothes washing machine. Hell yes it's gotten out of control.
- cainxinth 3y ago
  When TVs switched from standard def, suddenly all sorts of products from makeup to paint to sunglasses were advertising themselves as “HD.” After the iMac became a surprise hit for Apple, every piece of tech junk imaginable was immediately renamed to add a lowercase “i.” Blockchain had a similar, albeit briefer, hypecycle. Marketers love to jump on the latest trend by co-opting buzzwords.
  - jjk166 3y ago
    I liked how in the late 90s everything imaginable was updated to be the Generic Item 2000 to indicate it was the latest and greatest, and then once Y2K came and went they had to one up it and make everything a Generic Item 3000. Gonna confuse the hell out of some future archaeologists.
  - swagasaurus-rex 3y ago
    Many of the appliances from the 90s came with a “fuzzy logic” label. Nobody knew what it meant then, but it was still used to hype up products
    - copperx 3y ago
      I remember seeing the fuzzy logic label on japanese rice makers.
  - Izkata 3y ago
    There's quite a few results in google for "iBlockchain", plus some "Blockchain and AI" articles. Unfortunately I don't see any that mixes all three, or any of them with "HD".

110 points by CharlesW 3y ago

such a title!

Not that I disagree, but as I've said before, it's easy to be a theoretical contrarian, as in claim something is BS but not stake anything on it or profit from it (other than book deals). We're in a housing bubble, at the peak of a multi-century economic mega-cycle, we're overdue for earthquakes, tsunamis, asteroids, the caldera under Yellowstone, flu pandemics, etc etc. It's super easy to say all that and demand people should listen, what's hard is committing to any actionable prediction.

lordkirchner 3y ago
All I ever wanted was a way to short NFTs
- lokar 3y ago
  Never short a ponzi, even if you win your counterparty won’t pay
piloto_ciego 3y ago
Being a pessimist is academically safe and intellectually lazy. If you’re wrong in your predictions nobody cares because it’s better than if you’re right. On the other hand, if you’re an optimist and you are wrong you look like an idiot and the social penalty is higher. So people continue to predict doomsday and most of the time it doesn’t happen.
- md_ 3y ago
  > So people continue to predict doomsday and most of the time it doesn’t happen. Well, the same is true for any prediction of anything unlikely. People continue to predict world-changing innovations, and most of the time they are wrong. That's the reality of any low-probability/high-impact prediction, in the positive or negative direction. As a specific case of that: most startups fail, so predicting a startup will fail is safe. I'm pretty prone to exactly that prediction. But I like this quote from Erik Davis (https://www.google.ch/books/edition/High_Weirdness/Rcq2DwAAQBAJ https://www.google.ch/books/edition/High_Weirdness/Rcq2DwAAQ...): "In the court of the mind, skepticism makes a great grand vizier, but a lousy lord."

JieJie 3y ago

It's not really about "ai hype", it's about funding. "In the last few months, there has been this increasing so-called rift between the AI ethics and AI safety communities. There is a lot of talk about how this is an academic rift that needs to be resolved, how these communities are basically aiming for the same purpose. I think the thing that annoys me most about the discourse around this is that people don’t recognize this as a power struggle. It is not really about intellectual merit of these ideas. Of course, there are lots of bad intellectual and academic claims that have been made on both sides. But that isn’t what this is really about. It’s about who gets funding, which concerns are prioritized. So looking at it as if it is like a clash of individuals or a clash of personalities just really undersells the whole thing, makes it sound like people are out there bickering, whereas in fact, it’s about something much deeper."

deleted 3y ago
[deleted]

ajcp 3y ago

All snark aside it must be really tough to make a name for yourself by calling something snake oil only to have a flavor of it become massively successful...and then have to double down because you already had a book deal to fulfill.

deleted 3y ago
[deleted]

ftxbro 3y ago

They were wrong and now they have to pivot, from "AI isn't effective" to "it's bad that AI is so effective."

deleted 3y ago
[deleted]

Someone1234 3y ago

The key way I try to distinguish "Snake Oil" is: Is anyone actually using this routinely, and are the problems it solves self-evident to new non-zealot users after a demo? Take blockchain for example; people spent years and millions of dollars looking for additional usages for the technology beside coins/coin-contracts; and aside from the forementioned usages it didn't really gain routine adoption. That was why the non-coin-blockchain always struct me as a "hype train" or "snake oil" because it was a solution in search of a problem. NFTs are even worse since they haven't found any purpose to exist yet (money laundering?). Contrast that with the "GPT" offerings (ChatGPT/Bing/Copilot/Bard/etc). Multiple of my colleagues are actively using them routinely every workday and when you demo it to another person, they understand it, and they too start utilizing it in their workflow. Heck, my mom discovered it herself on Bing's homepage and was telling me I should check it out. That's the opposite of "snake oil" or a hype train, it is arguably a competitive threat to search engines. PS - Small disclaimer: "AI" is a nebulous term. I'm talking specifically about LLMs. Other types of "AI" have already been here for a long time making our lives better (e.g. computer vision).

lIIllIIllIIllII 3y ago
I'm routinely using LLMs in place of search engines. Would you rather type in a cocktail and instantly get a recipe? or do you want to open google, scroll past 50 ads, click through a bunch of blogspam that gives you the history of cocktails, the prohibition, the chemical formula for ethanol and their life story, followed by a recipe that was itself generated by a substandard AI model anyway edit: I'm prepared for the enshittification of GenAI, but given competitive edge, many people will pay top $ for faster/higher parameter inference, I'm hoping ad support is minimal on the super deluxe god tier AI subscription which I'll have to pile all of my spare cash into.
- danmur 3y ago
  If you pay for search you get some of those benefits already
  - chargingmarmot 3y ago

j45 3y ago

Does Princeton not have any meaningful generative AI work or research going on?

calf 3y ago
I went to Princeton, I can assure you Narayanan is only one professor among the many brilliant STEM scientists there.

clircle 3y ago

Analysis: true, six months ago

TuringNYC 3y ago

Everyone should read this Reddit comment: https://www.reddit.com/r/blender/comments/121lhfq/i_lost_everything_that_made_me_love_my_job/ https://www.reddit.com/r/blender/comments/121lhfq/i_lost_eve... This is happening, and the reddit thread is just a single instance of a wide phenomenon. I wonder how one can square "snakeoil" theories against reality like this.

flangola7 3y ago
My company has openly bragged they predicted they would need to hire over 200 employees this year, and AI systems meant they only had to hire 45.
numpad0 3y ago
Maybe the reality isn't evenly distributed. I'm supposedly at the working end of that job and can't immediately recall seeing the result of that transition.
slfnflctd 3y ago
> just a single instance of a wide phenomenon Another instance is the 'hey pi' chatbot. For some people it's a superior experience to BetterHelp, or even a lot of so-called professional therapists out there. The lack of actual human presence is a bit deflating for me, but I still got a good session out of it once. It feels like it has the potential to be at least better than nothing for some of the lonely people out there. The option for different voices is a nice touch that adds to the illusion. There are so many possibilities. I'm looking forward to seeing what happens with everything from NPCs in video games to robotics (among other things, to actually explain what they're doing & why, and converse). Not to mention the applications in education and health care. Anyone who thinks this is a flash in the pan has not observed enough of what's going on.
satvikpendem 3y ago
Most jobs in human history were not labors of passion, it is only in very recent history that it is so. We should not expect it to be the case that people will love their job, as not all loves are employable, as OP in the thread is finding out. While OP might dislike it, there are many others who will fill their space if OP leaves that job or industry, those that actually enjoy what they do. Contrast that to the people on https://old.reddit.com/r/StableDiffusion https://old.reddit.com/r/StableDiffusion, where people willingly experiment with multiple approaches to media generation. At least some of those people will be interested in 3D modeling and will willingly take OP's job, because they genuinely enjoy it and OP now does not. In the end, adapt, or others who do will take over. That's been the historical precedent for millennia, and it will not stop now.

atleastoptimal 3y ago

My hypothesis is that a lot of high-status intellectuals are reflexively dismissing AI out of fear that it threatens their status, as AI poses to turn intellectual labor into a cheap commodity rather than the premium bespoke product they offer it as via their professorships, speaking fees and book sales. Most criticism of AI has all the hallmarks of coping mechanisms. They seem to shift quickly between declaring it an ineffective parlor trick, and then say it's so good and effective that it poses a risk to the human spirit. There are a lot of legitimate criticisms I've seen, especially of OpenAI's potential regulation capture and misuse of models, but calling it all snake oil is hilariously naïve and seems more like a fear of change.

deleted 3y ago
[deleted]
kromem 3y ago
Really? You think people who write for a living might be biased towards confirming that the thing already replacing many people who write for a living is a fad that's going to blow over soon? My favorite part with crap predicting doom is the sheer unawareness of the rate of change of the rate of change. If you had asked me 18 months ago how long after GPT-3 until something existed with the capabilities of GPT-4, I'd probably have guessed about 5 years. If you asked me 5 years ago how long until an AI could explain why a joke was funny, I'd have maybe guessed at least a decade or two, if it was even possible. If you asked me 10 years ago if I'd see AI replacing artists or copywriters in my lifetime, I'd have guessed maybe when I was in a retirement home (I'm still a fair ways away from that). No one thought what exists today was even possible within our lifetimes a decade back. I'm reminded of the Louis C.K. routine about everything being amazing and no one is happy. Unthinkable AI has already become so normalized that people are predicting its doom based on shortcomings that are less than two years old because AI doing fucking IMPOSSIBLE things (or so everyone thought) is only that old. The rate is outrageous, and while I do think there's currently a significant setback with obsolete alignment approaches being carried forward to models that probably need new techniques, that's going to be a temporary step back in parallel to significant strides forward in the underlying technology from hardware to model design to improved knowledge in how to squeeze the most water from the rock. I just hope when all these folks turn out to have been dead wrong that we don't collectively forget. Futurists should live and die by their record, but too many have goldfish memory and continue to listen to false futurists well after they've shown their own snake oil hand.

phillipcarter 3y ago

The authors' substack is great. They're sort of like investigative journalists who pick a topic that's got a lot of hype and then dissect it to (often very quickly) find that the hyped up topic was just hot air. Some examples: The claims of "liberal bias" being just a completely bunk study: https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bia... No evidence of GPT-4 getting worse over time: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim... The analyses are measured and well written. I always enjoy when a new one comes out.

bookaway 3y ago
>No evidence of GPT-4 getting worse over time I read the the article you linked to, and either you're putting words in their mouth and/or making claims that go way beyond the conclusions presented in the article. The authors choose to distinguish between "capability degradation" and "drastic behavior change." Now, almost no one I've witnessed complaining about past and present results of ChatGPT care one bit about making such a distinction. They often simply complain that their previously good result producing prompts are now spewing garbage. The authors explicitly admit that "the kind of fine tuning that LLMs regularly undergo can have unintended effects, including drastic behavior changes on some tasks." i.e. Fine tuning may in fact be causing the worse results. Furthermore, the authors merely make an additional point that such a drastic behavior change shouldn't be assumed to be a "capability degradation." Meaning that, theoretically, if you toy around with your past prompt enough you should be able to eventually find an alternative version of the original prompt that outputs the past result you were looking for. Again, none of the complainers that I've come across really give a shit about this theoretical conclusion. They only care about past prompts now giving different results. In the article, the authors freely admit that this "drastic behavior change" could be objectively occurring due to fine-tuning. To further prevent readers from making sweeping generalizations from their comments, the author's explicitly state that OpenAI makes it pretty damn difficult to do reproducible research on the LLMs in question, so you should not come to any hasty conclusions regardless ("As we have written before, this underscores how hard it is to do reproducible research that uses these APIs, or to build reliable products on top of them.") Well, "building reliable products/processes on top of ChatGPT" is all the complainers care about anyway, so we're back to square one. Far from disproving the claim that ChatGPT is becoming more unreliable and producing worse results for the complainers use cases, the authors are in fact saying that their grievances may be justified for the only practical criteria the naysayers actually give a shit about.

NoZebra120vClip 3y ago

I have stopped using the "snake oil" epithet, because I found out an interesting background on that. Oils derived from various snakes in the desert were found to be effective treatments and remedies for various things by the Native American population, and therefore widely used by certain tribes. Once the White Man caught wind of this, he capitalized on its reputation, and a booming business of scams bloomed anywhere there were gullible buyers. Of course, the snake oil was rarely authentic or effective for its advertised uses. Therefore, snake oil was given a very bad and rather undeserved reputation for the rest of history, and medical science continues to flounder.

skhr0680 3y ago
“Snake oil” is a reference to Clark Stanley, whose snake oil was infamously found to contain no snake oil whatsoever.
ggm 3y ago
C/F "goanna oil" Australia. Still made. Nowadays, eucalyptus oil, pine oil, peppermint oil, camphor, menthol and turpentine. https://ourstory.moretonbay.qld.gov.au/nodes/view/38026 https://ourstory.moretonbay.qld.gov.au/nodes/view/38026
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]

ggm 3y ago

Great to read this back-to-back with the takedown on the GAN image generation offering also on HN. The point of most critiques/polemics on AI Snake oil isn't on effectiveness of AI in context. It's about mis-application, belief its AGI, belief the answers are right, and belief it has no downsides. It's often mis-applied, It's absolutely not, and not even on the road to AGI, the answers are not always right, and it has massive social downsides, employment included. It has upsides, sure. It's not the job of snake oil warners to catalog the good, they're pointing out the abysmally bad, in the current AI hype-wave.

m00x 3y ago
Crazy thing is that intelligence doesn't mean you get the right answer. Humans get the answer right fewer times than AI. It's absolutely not AGI, but it's not "not the road to AGI". Every step forward could be the road to AGI.
- ggm 3y ago
  Could is doing a lot of heavy lifting here. People have been hypothesising about the nature of mind and intelligence for a long time. There is no compelling connectionist, moar-will-work, it-can-learn-to-think model, which infers the nature of intelligence and how that relates to GAN or GPT or any of what people are doing now. What people are doing now is amazing. I love looking at pictures of the pope in puffer jackets. I love "this person does not exist" It could be on the road to AGI, but then, I could win lotto. and I do enter lotto, even knowing the odds. The thing is I don't plan as if I WILL win lotto and a lot of "this AI could be the road to AGI" is about planning as if it will, to secure a slice of the future. It could? yea. But, it isn't.
  - visarga 3y ago
    The future will probably have amazing computers and models, but what we already have has the capacity to profoundly change the world. I don't think you are focusing on what matters most. It's really the language itself that contains the intelligence, not the AI models. The models are just vessels for absorbing all that knowledge encoded in text. Just like human brains - we can have different neural wiring but learn the same things through education. So the huge datasets these AI systems are trained on are key. That's how models like GPT-4 gain such language understanding. The architecture matters less than all that linguistic data it ingests. And language has been evolving for millennia, long before AI. It replicates through culture, speech, writing. Now with AI it has a whole new medium. Fascinating question - how will AI affect language evolution going forward? As models produce more human-like text, which can further train better models, it's like a Lamarckian evolution. Acquired linguistic intelligence gets absorbed by AI then improved and propagated back into the corpus. So while AI tech will keep advancing, it's language evolution that's most profound. By creating this new way for language to evolve, AI could really reshape cultural evolution. Since language enables intelligence, it'll have big impacts on where AI is headed next.

xen2xen1 3y ago

My washing machine says "Optimized by AI". My clothes washing machine. Hell yes it's gotten out of control.

cainxinth 3y ago
When TVs switched from standard def, suddenly all sorts of products from makeup to paint to sunglasses were advertising themselves as “HD.” After the iMac became a surprise hit for Apple, every piece of tech junk imaginable was immediately renamed to add a lowercase “i.” Blockchain had a similar, albeit briefer, hypecycle. Marketers love to jump on the latest trend by co-opting buzzwords.
- jjk166 3y ago
  I liked how in the late 90s everything imaginable was updated to be the Generic Item 2000 to indicate it was the latest and greatest, and then once Y2K came and went they had to one up it and make everything a Generic Item 3000. Gonna confuse the hell out of some future archaeologists.
- swagasaurus-rex 3y ago
  Many of the appliances from the 90s came with a “fuzzy logic” label. Nobody knew what it meant then, but it was still used to hype up products
  - copperx 3y ago
    I remember seeing the fuzzy logic label on japanese rice makers.
- Izkata 3y ago
  There's quite a few results in google for "iBlockchain", plus some "Blockchain and AI" articles. Unfortunately I don't see any that mixes all three, or any of them with "HD".