4 ms·

> If you wanted it to act like a real Egyptologist I don't care about LLMs. I'm pretending to be a gullable customer, not being myself. Companies and individu

> If you wanted it to act like a real Egyptologist I don't care about LLMs. I'm pretending to be a gullable customer, not being myself. Companies and individuals are buying LLMs expecting them to be real developers, and real writers, and real risk analysts... but they'll get average dumb-as-they-come internet commenter. It's fraud. It doesn't matter if you explain to me the obvious thing that I already know (they suck). The marketing is telling everyone that they're amazing PhD level geniuses. I just demonstrated that they resemble more an average internet idiot than a specialist. If I were a customer from academia, and you were an AI company, you just lost a client. You're trying to justify a failure in the product. Also, if I try to report an issue online, I won't be able to. A hoarde of hyped "enthusiasts" will flood me trying to convince me that the issue simply does not exist. I will tell everyone not to buy it, because the whole experience sucks.

ben_w 1y ago
Ahem, you previously: Remember, the claim I challenged is that LLMs know culture and can handle talking about them. You need to focus. Anyway: > The marketing is telling everyone that they're amazing PhD level geniuses. I just demonstrated that they resemble more an average internet idiot than a specialist. First, you didn't. Average internet idiot doesn't know jack about either Western New Age ancient aliens culture or actual ancient Egypt, let alone being able to write an essay on both. Second: You seem to be wildly overestimating what "PhD-level" implies. Academics do a lot of post-docs before they can turn a doctorate into a professorship. The SOTA models are what PhD level looks like: freshly minted from university without much experience. Rather than what you suggest, the academic response to "PhD level" is not to be impressed by marketing then disapointed with results, because an academic saying "wow, a whole PhD!" would be sarcasm in many cases: a PhD just step 1 of that career path. Similarly, medical doctors have not been impressed just by LLMs passing the medical exam, and lawyers not impressed by passing the Bar exam. Because that's the entry requirement for the career. Funnily enough, three letters after the name does not make someone infallible, it's the start of a long, long journey. Academia, medics, lawyers, coders, hearing about PhD level means we're expecting juniors and getting them too.
- alganet 1y ago
  I stand by my claim. I pretended to be less knowledgeable than I currently am about the egyptologists vs. ancient aliens public debate. Then I reported my results, together with the opinion of specialists from trusted sources (what actual egyptologists say). There is _plenty_ of debate on the internet about this. It is a popular subject, approached by many average internet idiots in many ways. Anyone reading this right now can confirm this by performing a search. You're trying to blur the lines between what an actual PhD is and what the perceived notion of what a PhD is. This is an error. My original comment regarding PhDs was placed in a marketing context. It is the same as the "9 in 10 specialists recomment Colgate" trick. In that analogy, you're trying to explain to me how dentists get their degree, instead of acknowledging that I was talking about the deceptive marketing campaign. You also failed to generalize the example outside of the egyptology realm. I can come up with other examples in other areas I consider myself above-average-but-not-actual-researcher. Attempts to demoralize me in those subjects won't make the LLM better, this is not a classical idiot internet debate: you winning doesn't make me lose. On the contrary: your use of diversion and misdirection actually support my case. You need to rely on cheap rethoric tactics to succeed, I don't. This video came out right after I posted my original challenge, and it explains some of the concepts I'm hopelessly trying to convey to you: https://www.youtube.com/watch?v=5mco9zAamRk https://www.youtube.com/watch?v=5mco9zAamRk It is a friendly cartoon-like simplification of how AIs are evaluated. It is actually friendly to AI enthusiasts, I recommend you to watch it and rethink the conversation from its perspective.
  - TeMPOraL 1y ago
    The video seems to be an animated retelling of a 2021 blog post, so I'll just link to the post itself: https://www.lesswrong.com/posts/PZtsoaoSLpKjjbMqM/the-case-for-aligning-narrowly-superhuman-models https://www.lesswrong.com/posts/PZtsoaoSLpKjjbMqM/the-case-f... There's some extra commentary from the reviewers of earlier drafts here: https://www.lesswrong.com/posts/AyfDnnAdjG7HHeD3d/miri-comments-on-cotra-s-case-for-aligning-narrowly https://www.lesswrong.com/posts/AyfDnnAdjG7HHeD3d/miri-comme... I've skimmed both of these - this is some substantial and pretty insightful reading (though 2021 was ages ago - especially now that AI safety stopped being a purely theoretical field). However, as of now, I can't really see the connections between the points being discussed there, and anything you tried to explain or communicate. Could you spell out the connection for us please?
TeMPOraL 1y ago
'ben_w addressed other points, but it would be amiss not to comment on this too: > The marketing is telling everyone that they're amazing PhD level geniuses. No it is not. LLM vendors are, and have always been, open about the limits of the models, and I'm yet to see a major provider claiming their models are geniuses, PhD-level or otherwise. Nothing of the sort is happening - on the contrary, the vendors are avoiding making such claims or positioning their offerings this way. No, this perspective doesn't come from LLM marketing. It comes from people who ignore both the official information from the vendors and the experience of LLM users, who are oblivious to what's common knowledge and instead let their imagination run wild, perhaps fueled by bullshit they heard from other clueless people, or more likely, from third parties on the Internet that say all kinds of outlandish things to get more eyes looking at the ads they run. > Companies and individuals are buying LLMs expecting them to be real developers, and real writers, and real risk analysts... but they'll get average dumb-as-they-come internet commenter. Curiously, this is wrong in two opposite directions at once. Yes, many companies and individuals have overinflated expectations, but that's frankly because they're idiots. There's no first-party fraud going on here; if you get fooled by hype from some random LinkedIn "thought leaders", that's on you; sue them for making a fool of you, and don't make yourself a bigger one by blaming LLM vendors for your own poor information diet. At the same time, LLMs actually are at the level of real developers, real writers and real risk analysts; downplaying capabilities of current LLMs doesn't make you less wrong than overestimating them.
- alganet 1y ago
  Me: > > Companies and individuals are buying LLMs expecting them to be real developers You: > Yes, many companies and individuals have overinflated expectations > LLMs actually are at the level of real developers Ok then.

ben_w 1y ago

Ahem, you previously: Remember, the claim I challenged is that LLMs know culture and can handle talking about them. You need to focus. Anyway: > The marketing is telling everyone that they're amazing PhD level geniuses. I just demonstrated that they resemble more an average internet idiot than a specialist. First, you didn't. Average internet idiot doesn't know jack about either Western New Age ancient aliens culture or actual ancient Egypt, let alone being able to write an essay on both. Second: You seem to be wildly overestimating what "PhD-level" implies. Academics do a lot of post-docs before they can turn a doctorate into a professorship. The SOTA models are what PhD level looks like: freshly minted from university without much experience. Rather than what you suggest, the academic response to "PhD level" is not to be impressed by marketing then disapointed with results, because an academic saying "wow, a whole PhD!" would be sarcasm in many cases: a PhD just step 1 of that career path. Similarly, medical doctors have not been impressed just by LLMs passing the medical exam, and lawyers not impressed by passing the Bar exam. Because that's the entry requirement for the career. Funnily enough, three letters after the name does not make someone infallible, it's the start of a long, long journey. Academia, medics, lawyers, coders, hearing about PhD level means we're expecting juniors and getting them too.

alganet 1y ago
I stand by my claim. I pretended to be less knowledgeable than I currently am about the egyptologists vs. ancient aliens public debate. Then I reported my results, together with the opinion of specialists from trusted sources (what actual egyptologists say). There is _plenty_ of debate on the internet about this. It is a popular subject, approached by many average internet idiots in many ways. Anyone reading this right now can confirm this by performing a search. You're trying to blur the lines between what an actual PhD is and what the perceived notion of what a PhD is. This is an error. My original comment regarding PhDs was placed in a marketing context. It is the same as the "9 in 10 specialists recomment Colgate" trick. In that analogy, you're trying to explain to me how dentists get their degree, instead of acknowledging that I was talking about the deceptive marketing campaign. You also failed to generalize the example outside of the egyptology realm. I can come up with other examples in other areas I consider myself above-average-but-not-actual-researcher. Attempts to demoralize me in those subjects won't make the LLM better, this is not a classical idiot internet debate: you winning doesn't make me lose. On the contrary: your use of diversion and misdirection actually support my case. You need to rely on cheap rethoric tactics to succeed, I don't. This video came out right after I posted my original challenge, and it explains some of the concepts I'm hopelessly trying to convey to you: https://www.youtube.com/watch?v=5mco9zAamRk https://www.youtube.com/watch?v=5mco9zAamRk It is a friendly cartoon-like simplification of how AIs are evaluated. It is actually friendly to AI enthusiasts, I recommend you to watch it and rethink the conversation from its perspective.
- TeMPOraL 1y ago
  The video seems to be an animated retelling of a 2021 blog post, so I'll just link to the post itself: https://www.lesswrong.com/posts/PZtsoaoSLpKjjbMqM/the-case-for-aligning-narrowly-superhuman-models https://www.lesswrong.com/posts/PZtsoaoSLpKjjbMqM/the-case-f... There's some extra commentary from the reviewers of earlier drafts here: https://www.lesswrong.com/posts/AyfDnnAdjG7HHeD3d/miri-comments-on-cotra-s-case-for-aligning-narrowly https://www.lesswrong.com/posts/AyfDnnAdjG7HHeD3d/miri-comme... I've skimmed both of these - this is some substantial and pretty insightful reading (though 2021 was ages ago - especially now that AI safety stopped being a purely theoretical field). However, as of now, I can't really see the connections between the points being discussed there, and anything you tried to explain or communicate. Could you spell out the connection for us please?

TeMPOraL 1y ago

'ben_w addressed other points, but it would be amiss not to comment on this too: > The marketing is telling everyone that they're amazing PhD level geniuses. No it is not. LLM vendors are, and have always been, open about the limits of the models, and I'm yet to see a major provider claiming their models are geniuses, PhD-level or otherwise. Nothing of the sort is happening - on the contrary, the vendors are avoiding making such claims or positioning their offerings this way. No, this perspective doesn't come from LLM marketing. It comes from people who ignore both the official information from the vendors and the experience of LLM users, who are oblivious to what's common knowledge and instead let their imagination run wild, perhaps fueled by bullshit they heard from other clueless people, or more likely, from third parties on the Internet that say all kinds of outlandish things to get more eyes looking at the ads they run. > Companies and individuals are buying LLMs expecting them to be real developers, and real writers, and real risk analysts... but they'll get average dumb-as-they-come internet commenter. Curiously, this is wrong in two opposite directions at once. Yes, many companies and individuals have overinflated expectations, but that's frankly because they're idiots. There's no first-party fraud going on here; if you get fooled by hype from some random LinkedIn "thought leaders", that's on you; sue them for making a fool of you, and don't make yourself a bigger one by blaming LLM vendors for your own poor information diet. At the same time, LLMs actually are at the level of real developers, real writers and real risk analysts; downplaying capabilities of current LLMs doesn't make you less wrong than overestimating them.

alganet 1y ago
Me: > > Companies and individuals are buying LLMs expecting them to be real developers You: > Yes, many companies and individuals have overinflated expectations > LLMs actually are at the level of real developers Ok then.