5 ms·

AI reminds of listening to any person who seems like an intellectual authority on multiple subjects on YouTube and is not afraid to wax confidently on any topic

AI reminds of listening to any person who seems like an intellectual authority on multiple subjects on YouTube and is not afraid to wax confidently on any topic. They seem very intelligent and knowledgable until they actually talk about something you know. In other words, I try to learn from it whenever it does something I can't do but when it does something I can do or something I'm really good at it I find myself wanting to correct it cause it doesn't do it that well. It just seems like a really quick thinking and fast executing but, ultimately, mid skilled / novice person.

ahd94 7mo ago
And it starts showing impatience when its about to run out of context, more like someone who wants to get out of the office exactly at 5.
- kykat 7mo ago
  Not just when running out of context, it's always. Once it fixates on a goal, all hell breaks loose and there's nothing that it won't be sacrificed to get there. At least that's my experience with Claude Code, I am pressing the figurative breaks all the time.
  - bitwize 7mo ago
    "I'm Mr. Meeseeks, look at meeee!"
    - thr0waway001 7mo ago
      Claude, I need you to help me take two strokes off my golf game.
- xg15 7mo ago
  It was sort of funny when codex switched mid-session from "patch complete, I'll now automatically run the tests and verify results" to "patch complete, if you want to run the tests, just paste this instruction into a terminal somewhere: ..." Apparently this was caused by the context window getting full! (At least I assume that because it went back to the old behavior after I triggered a compaction)
- deleted 7mo ago
  [deleted]
Andrei_dev 7mo ago
[dead]
bitwize 7mo ago
Gell-Mann amnesia. The things it tells you about things you don't know are things that would make a knowledgeable person go "dude, wtf? That's totally wrong."
- exmadscientist 7mo ago
  You can really only use AI for: things that are easy to verify; things that you already know how to do but want done faster; things you're learning to do and are just one step out of your reach (so it's still comprehensible to you); or, things that just plain don't matter. That's a lot of stuff, but it also doesn't include a lot of the stuff people claim AI can do.
tokioyoyo 7mo ago
Have you been actively using paid versions of the flagship models from Ant / OpenAI? I’m just curious if the conclusion was made within the last 6 months or not.
- lelanthran 7mo ago
  I got that experience 3 hours ago.
raincole 7mo ago
AI's mistakes are sometimes so subtle. Just yesterday I asked Gemini Pro 3.0 this question: > Find such colors A and B: > A and B are both valid sRGB color. > Interpolating between them in CIELAB space like this > C_cielab = (A_cielab + B_cielab) / 2 > results in a color C that can't be represented in sRGB It gave me a correct answer, great! ...and then it proceeded to tell me to use Oklab, claiming it doesn't have this problem because the sRGB gamut is convex in Oklab. If I didn't know Oklab does have the exact same problem I would have been fooled. It just sounds too reasonable.
- AugSun 7mo ago
  [dead]
- zozbot234 7mo ago
  You can sometime run a quick second check by taking the AI's claim and asking it for an evaluation within a fresh context. It won't be misled by the surrounding text and its answer will at least be somewhat unbiased (though it might still be quite wrong). It helps if you phrase the question openly, not obviously fishing for a yes-or-no answer. Or, if you have to ask for a yes-or-no question, make it sound like you're obviously expecting the answer that's actually less likely, so the AI will (1) either be more willing to argue against it, or (2) provide good arguments for it you might not have considered, because it "knows" the answer is unexpected and it wants to flatter your judgment.
  - GCUMstlyHarmls 7mo ago
    > It helps if you phrase the question openly, not obviously fishing for a yes-or-no answer. Or, if you have to ask for a yes-or-no question, make it sound like you're obviously expecting the answer that's actually less likely, I do this all the time and hate that I have to do it, with the additional "do not yes-man me, be critical."
    - MadxX79 7mo ago
      Great, now I have two answers and still no clue which one is the right one.
rambambram 7mo ago
> In other words, I try to learn from it whenever it does something I can't do... So you know it can be full of sh1t on all kinds of topics, and you start learning from it the moment it's 'talking' about subjects you know you don't know about? To me that sounds like the moment to stop, not the moment to start. Or am I missing something?
- Insanity 7mo ago
  Sounds like an extension of https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect, but sub newspaper for large language model.
xbmcuser 7mo ago
In the last few years I have come to realize that first impression of anything is extremely important if your first few uses were good and wowed you you will be positive about it. If it was not you will be negative about it the bias of the first encounter stays with us no matter what.
- ben_w 7mo ago
  Hence beginner's luck. Most long-term gamblers will tell you that the first games they played, they won. This is a real thing, yet we cannot apply it by making one bet and then stopping, because so are the probabilities being fair and un-biased. What squares these two things is that most of the people who played and lost their first games, did not get addicted to gambling.
- Obscurity4340 7mo ago
  Dont know about that, I dont think I had any superb first experience with it but even if I had, I got more turned on to it when I started using it for toy program/code solutions I needed on a one-off basis occasionally. If it didnt give me the code I needed to get various things done on the fly, I would maybe be more agnostic. On non-code stuff, I think its improved or there are better options for making it get to the point and be concise more and I find when I correct it, quite often we actually get somewhere. The answers I remember from my initial use of it ofbasically how to do anything or most subjects was practically a 10 pager with some weird action plan that you were never gonna go thru.
threethirtytwo 7mo ago
It's a good analogy to comfort yourself. But remember AI is now being deployed on the frontline of mathematics and coming up with new theories. The reality is much more stark then your description. Yes, in MANY instances it fails at things you know and you're an expert at. But in MANY instances it also beats you at what you're good at. People who say stuff like the parent poster are completely mischaracterizing the current situation. We are not in a place where AI is "good" but we are "better". No... we are approaching a place of we are good and AI is starting to beat us at our own game. That is the prominent topic that is what is trending and that is the impending reality. Yet everywhere on HN I see stuff like, oh AI fails here, or AI fails there. Yeah AI failing is obvious. It's been failing for most of my life. What's unique about the last couple years is that it's starting to beat us. Why? Because your typical HNer holds programming as not just a tool, but an identity. Your skill in programming is also a status symbol and when AI attacks your identity, the first thing you do to defend your identity is to bend reality and try to cast to a different conclusion by looking at everything from a different angle. Face Reality.
- onetokeoverthe 7mo ago
  [dead]
- jltsiren 7mo ago
  I think that's a category error. Current AI is not better or worse than us but fundamentally different. Its main strength and weakness is that it knows too much about everything. It usually knows more than the user about the topic at hand, but it doesn't know what is actually relevant in that particular situation. If you nudge the AI in the right direction, it may surprise you with what it's capable of. But if you nudge it in a wrong direction or just don't give it sufficient context, it can be very confidently wrong.
  - threethirtytwo 7mo ago
    Yeah this is another way to comfort yourself. Yes AI is different but again FACE reality. Ai is different but it is also similar, for example it can speak language. So it is different and similar at the same time. I am obviously referring to AI beating us where we are similar to it. Best example: software engineering. This is obvious. The fact that I need to spell it out shows how deep the delusion goes. The rest of your response is just regurgitating what the parent post said. Sure. But it doesn’t address the fact that while everything you said is true part of the time the other part of the time it beats us (that includes both nudging and no nudging it in the right direction). Additionally all of this is also completely ignoring the fact that AI leap frogged in capability in the past year with my entire company now foregoing the use of text editing or using IDEs and having Claude write everything. If what you say is true only now and we see this much velocity in improvement then wait a couple more years and everything you said can be completely false if the trendline continues. And here’s the craziest part. Everything im saying is obvious. I’m basically being captain obvious here. The question is why are so many people in total denial.
wazHFsRy 7mo ago
I think the even worse problem is that by extension now *everyone* sounds like an expert, even if they aren’t. In the past when someone wrote an RFC they would need to study to formulate it well. Now anyone can create content sounding like an expert and it becomes difficult for the reader to differentiate real expertise and depth vs shallow fancy words.
- wazHFsRy 7mo ago
  I also discussed this a bit more here: https://www.dev-log.me/everyone_is_an_expert_now/ https://www.dev-log.me/everyone_is_an_expert_now/
deleted 7mo ago
[deleted]

ahd94 7mo ago

And it starts showing impatience when its about to run out of context, more like someone who wants to get out of the office exactly at 5.

kykat 7mo ago
Not just when running out of context, it's always. Once it fixates on a goal, all hell breaks loose and there's nothing that it won't be sacrificed to get there. At least that's my experience with Claude Code, I am pressing the figurative breaks all the time.
- bitwize 7mo ago
  "I'm Mr. Meeseeks, look at meeee!"
  - thr0waway001 7mo ago
    Claude, I need you to help me take two strokes off my golf game.
xg15 7mo ago
It was sort of funny when codex switched mid-session from "patch complete, I'll now automatically run the tests and verify results" to "patch complete, if you want to run the tests, just paste this instruction into a terminal somewhere: ..." Apparently this was caused by the context window getting full! (At least I assume that because it went back to the old behavior after I triggered a compaction)
deleted 7mo ago
[deleted]

Andrei_dev 7mo ago

[dead]

bitwize 7mo ago

Gell-Mann amnesia. The things it tells you about things you don't know are things that would make a knowledgeable person go "dude, wtf? That's totally wrong."

exmadscientist 7mo ago
You can really only use AI for: things that are easy to verify; things that you already know how to do but want done faster; things you're learning to do and are just one step out of your reach (so it's still comprehensible to you); or, things that just plain don't matter. That's a lot of stuff, but it also doesn't include a lot of the stuff people claim AI can do.

tokioyoyo 7mo ago

Have you been actively using paid versions of the flagship models from Ant / OpenAI? I’m just curious if the conclusion was made within the last 6 months or not.

lelanthran 7mo ago
I got that experience 3 hours ago.

raincole 7mo ago

AI's mistakes are sometimes so subtle. Just yesterday I asked Gemini Pro 3.0 this question: > Find such colors A and B: > A and B are both valid sRGB color. > Interpolating between them in CIELAB space like this > C_cielab = (A_cielab + B_cielab) / 2 > results in a color C that can't be represented in sRGB It gave me a correct answer, great! ...and then it proceeded to tell me to use Oklab, claiming it doesn't have this problem because the sRGB gamut is convex in Oklab. If I didn't know Oklab does have the exact same problem I would have been fooled. It just sounds too reasonable.

AugSun 7mo ago
[dead]
zozbot234 7mo ago
You can sometime run a quick second check by taking the AI's claim and asking it for an evaluation within a fresh context. It won't be misled by the surrounding text and its answer will at least be somewhat unbiased (though it might still be quite wrong). It helps if you phrase the question openly, not obviously fishing for a yes-or-no answer. Or, if you have to ask for a yes-or-no question, make it sound like you're obviously expecting the answer that's actually less likely, so the AI will (1) either be more willing to argue against it, or (2) provide good arguments for it you might not have considered, because it "knows" the answer is unexpected and it wants to flatter your judgment.
- GCUMstlyHarmls 7mo ago
  > It helps if you phrase the question openly, not obviously fishing for a yes-or-no answer. Or, if you have to ask for a yes-or-no question, make it sound like you're obviously expecting the answer that's actually less likely, I do this all the time and hate that I have to do it, with the additional "do not yes-man me, be critical."
  - MadxX79 7mo ago
    Great, now I have two answers and still no clue which one is the right one.

rambambram 7mo ago

> In other words, I try to learn from it whenever it does something I can't do... So you know it can be full of sh1t on all kinds of topics, and you start learning from it the moment it's 'talking' about subjects you know you don't know about? To me that sounds like the moment to stop, not the moment to start. Or am I missing something?

Insanity 7mo ago
Sounds like an extension of https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect, but sub newspaper for large language model.

xbmcuser 7mo ago

In the last few years I have come to realize that first impression of anything is extremely important if your first few uses were good and wowed you you will be positive about it. If it was not you will be negative about it the bias of the first encounter stays with us no matter what.

ben_w 7mo ago
Hence beginner's luck. Most long-term gamblers will tell you that the first games they played, they won. This is a real thing, yet we cannot apply it by making one bet and then stopping, because so are the probabilities being fair and un-biased. What squares these two things is that most of the people who played and lost their first games, did not get addicted to gambling.
Obscurity4340 7mo ago
Dont know about that, I dont think I had any superb first experience with it but even if I had, I got more turned on to it when I started using it for toy program/code solutions I needed on a one-off basis occasionally. If it didnt give me the code I needed to get various things done on the fly, I would maybe be more agnostic. On non-code stuff, I think its improved or there are better options for making it get to the point and be concise more and I find when I correct it, quite often we actually get somewhere. The answers I remember from my initial use of it ofbasically how to do anything or most subjects was practically a 10 pager with some weird action plan that you were never gonna go thru.

threethirtytwo 7mo ago

It's a good analogy to comfort yourself. But remember AI is now being deployed on the frontline of mathematics and coming up with new theories. The reality is much more stark then your description. Yes, in MANY instances it fails at things you know and you're an expert at. But in MANY instances it also beats you at what you're good at. People who say stuff like the parent poster are completely mischaracterizing the current situation. We are not in a place where AI is "good" but we are "better". No... we are approaching a place of we are good and AI is starting to beat us at our own game. That is the prominent topic that is what is trending and that is the impending reality. Yet everywhere on HN I see stuff like, oh AI fails here, or AI fails there. Yeah AI failing is obvious. It's been failing for most of my life. What's unique about the last couple years is that it's starting to beat us. Why? Because your typical HNer holds programming as not just a tool, but an identity. Your skill in programming is also a status symbol and when AI attacks your identity, the first thing you do to defend your identity is to bend reality and try to cast to a different conclusion by looking at everything from a different angle. Face Reality.

onetokeoverthe 7mo ago
[dead]
jltsiren 7mo ago
I think that's a category error. Current AI is not better or worse than us but fundamentally different. Its main strength and weakness is that it knows too much about everything. It usually knows more than the user about the topic at hand, but it doesn't know what is actually relevant in that particular situation. If you nudge the AI in the right direction, it may surprise you with what it's capable of. But if you nudge it in a wrong direction or just don't give it sufficient context, it can be very confidently wrong.
- threethirtytwo 7mo ago
  Yeah this is another way to comfort yourself. Yes AI is different but again FACE reality. Ai is different but it is also similar, for example it can speak language. So it is different and similar at the same time. I am obviously referring to AI beating us where we are similar to it. Best example: software engineering. This is obvious. The fact that I need to spell it out shows how deep the delusion goes. The rest of your response is just regurgitating what the parent post said. Sure. But it doesn’t address the fact that while everything you said is true part of the time the other part of the time it beats us (that includes both nudging and no nudging it in the right direction). Additionally all of this is also completely ignoring the fact that AI leap frogged in capability in the past year with my entire company now foregoing the use of text editing or using IDEs and having Claude write everything. If what you say is true only now and we see this much velocity in improvement then wait a couple more years and everything you said can be completely false if the trendline continues. And here’s the craziest part. Everything im saying is obvious. I’m basically being captain obvious here. The question is why are so many people in total denial.

wazHFsRy 7mo ago

I think the even worse problem is that by extension now *everyone* sounds like an expert, even if they aren’t. In the past when someone wrote an RFC they would need to study to formulate it well. Now anyone can create content sounding like an expert and it becomes difficult for the reader to differentiate real expertise and depth vs shallow fancy words.

wazHFsRy 7mo ago
I also discussed this a bit more here: https://www.dev-log.me/everyone_is_an_expert_now/ https://www.dev-log.me/everyone_is_an_expert_now/

deleted 7mo ago

[deleted]