4 ms·

When I see these debates it's always the other way around - one person speaks colloquially about an LLM's behavior, and then somebody else jumps on them for sup

When I see these debates it's always the other way around - one person speaks colloquially about an LLM's behavior, and then somebody else jumps on them for supposedly believing the model is conscious, just because the speaker said "the model thinks.." or "the model knows.." or whatever. To be honest the impression I've gotten is that some people are just very interested in talking about not anthropomorphizing AI, and less interested in talking about AI behaviors, so they see conversations about the latter as a chance to talk about the former.

latexr 1y ago
Respectfully, that is a reflection of the places you hang out in (like HN) and not the reality of the population. Outside the technical world it gets much worse. There are people who killed themselves because of LLMs, people who are in love with them, people who genuinely believe they have “awakened” their own private ChatGPT instance into AGI and are eschewing the real humans in their lives.
- fenomas 1y ago
  Naturally I'm aware of those things, but I don't think TFA or GGP were commenting on them so I wasn't either.
- Xss3 1y ago
  The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. It's going to take a lot to get him out of that mindset and frankly I'm dreading trying to compare and contrast imperfect human behaviour and friendships with a sycophantic AI.
  - bonoboTP 1y ago
    It's surprisingly common on reddit that people talk about "my chatgpt", and they don't always seem like the type who are "in a relationship" with the bot or unlocking the secrets of the cosmos with it, but still they write "my chatgpt" and "your chatgpt". I guess the custom prompt and the available context does customize the model for them in some sense, but I suspect they likely have a wrong mental model of how this customization works. I guess they imagine it as their own little model being stored on file at OpenAI and as they interact with it, it's being shaped by it, and each time they connect, their model is retrieved from the cloud storage and they connect to it or something.
  - lelanthran 1y ago
    > The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. The therapist thing might be correct, though. You can send a well-adjusted person to three renowned therapists and get three different reasons for why they need to continue sessions. No therapist ever says "Congratulations, you're perfectly normal. Now go away and come back when you have a real problem." Statistically it is vanishingly unlikely that every person who ever visited a therapist is in need of a second (more more) visit. The main problem with therapy is a lack of objectivity[1]. When people talk about what their sessions resulted in, it's always "My problem is that I'm too perfect". I've known actual bullies whose therapist apparently told them that they are too submissive and need to be more assertive. The secondary problem is that all diagnosis is based on self-reported metrics of the subject. All improvement is equally based on self-reported metrics. This is no different from prayer. You don't have a medical practice there; you've got an Imam and a sophisticated but still medically-insured way to plead with thunderstorms[2]. I fail to see how an LLM (or even the Rogerian a-x doctor in Emacs) will do worse on average. After all, if you're at a therapist and you're doing most of the talking, how would an LLM perform worse than the therapist? ---------------- [1] If I'm at a therapist, and they're asking me to do most of the talking, I would damn well feel that I am not getting my moneys worth. I'd be there primarily to learn (and practice a little) whatever tools they can teach me to handle my $PROBLEM. I don't want someone to vent at, I want to learn coping mechanisms and mitigation strategies. [2] This is not an obscure reference.
    - medvezhenok 1y ago
      Yup, this problem is why I think all therapists should ideally know behavioral genetics and evolutionary psychology (there is at least a plausibly objective measure there which is dissonance between the ancestral environment in which the brain developed and the modern day environment. And at least some amount of psychological problems can be explained by it). I am a fan of the « Beat Your Genes » podcast, and while some of the prescriptions can be a bit heavy handed, most feel intuitively right. It’s approaching human problems as intelligent mammal problems, as opposed to something in a category of its own.
positron26 1y ago
Most certainly the conversation is extremely political. There are not simply different points of view. There are competitive, gladiatorial opinions ready to ambush anyone not wearing the right colors. It's a situation where the technical conversation is drowning. I suppose this war will be fought until people are out of energy, and if reason has no place, it is reasonable to let others tire themselves out reiterating statements that are not designed to bring anyone closer to the truth.
- bonoboTP 1y ago
  If this tech is going to be half as impactful as its proponents predict, then I'd say it's still under-politicized. Of course the politics around it doesn't have to be knee-jerk mudslinging, but it's no surprise that politics enters the picture when the tech can significantly transform society.
  - positron26 1y ago
    Go politicize it on Reddit, preferably on a political sub and not a tech sub. On this forum, I would like to expect a lot more intelligent conversation.
scarface_74 1y ago
Wait until a conversation about “serverless” comes up and someone says there is no such thing because there are servers somewhere as if everyone - especially on HN -doesn’t already know that.
- Tijdreiziger 1y ago
  Why would everyone know that? Not everyone has experience in sysops, especially not beginners. E.g. when I first started learning webdev, I didn’t think about ‘servers’. I just knew that if I uploaded my HTML/PHP files to my shared web host, then they appeared online. It was only much later that I realized that shared webhosting is ‘just’ an abstraction over Linux/Apache (after all, I first had to learn about those topics).
  - scarface_74 1y ago
    I am saying that most people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation. I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”. It’s about as tiring as people on HN who know just a little about LLMs thinking they are sounding smart when they say they are just advanced autocomplete. Both responses are just as unproductive
    - Tijdreiziger 1y ago
      > I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”. Meh, I just knew that the browser would display HTML if I wrote it, and that uploading the HTML files made them available on my domain. I didn’t really think about where the files went, specifically. Try asking an average high school kid how cloud storage works. I doubt you’ll get any further than ‘I make files on my Google Docs and then they are saved there’. This is one step short of ‘well, the files must be on some system in some data center’. I really disagree that “people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation.” On the contrary, it’s an invitation to beginning coders to think about what the ‘serverless’ abstraction actually means.
Wowfunhappy 1y ago
As I write this, Claude Code is currently opening and closing various media files on my computer. Sometimes it plays the file for a few seconds before closing it, sometimes it starts playback and then seeks to a different position, sometimes it fast forwards or rewinds, etc. I asked Claude to write a E-AC3 audio component so I can play videos with E-AC3 audio in the old version of QuickTime I really like using. Claude's decoder includes the ability to write debug output to a log file, so Claude is studying how QuickTime and the component interact, and it's controlling QuickTime via Applescript. Sometimes QuickTime crashes, because this ancient API has its roots in the classic Mac OS days and is not exactly good. Claude reads the crash logs on its own—it knows where they are—and continues on its way. I'm just sitting back and trying to do other things while Claude works, although it's a little distracting that something else is using my computer at the same time. I really don't want to anthropomorphize these programs, but it's just so hard when it's acting so much like a person...
- godelski 1y ago
  Would it help you to know that trial and error is a common tactic by machines? Yes, humans do it too, but that doesn't mean the process isn't mechanical. In fact, in computing we might call this a "brute force" approach. You don't have to cover the entire search space to brute force something, and it certainly doesn't mean you can't have optimization strategies and need to grid search (e.g. you can use Bayesian methods, multi-armed bandit approaches, or a whole world of things). I would call "fuck around and find out" a rather simple approach. It is why we use it! It is why lots of animals use it. Even very dumb animals use it. Though, we do notice more intelligent animals use more efficient optimization methods. All of this is technically hypothesis testing. Even a naive grid search. But that is still in the class of "fuck around and find out" or "brute force", right? I should also mention two important things. 1) as a human we are biased to anthropomorphize. We see faces in clouds. We tell stories of mighty beings controlling the world in an effort to explain why things happen. This is anthropomorphization of the universe itself! 2) We design LLMs (and many other large ML systems) to optimize towards human preference. This reinforces an anthropomorphized interpretation. The reason for doing this (2) is based on a naive assumption[0]: If it looks like a duck, swims like a duck, and quacks like a duck, then it *probably* is a duck. But the duck test doesn't rule out a highly sophisticated animatronic. It's a good rule of thumb, but wouldn't it also be incredibly naive to assume that it *is* a duck? Isn't the duck test itself entirely dependent on our own personal familiarity with ducks? I think this is important to remember and can help combat our own propensity for creating biases. [0] It is not a bad strategy to build in that direction. When faced with many possible ways to go, this is a very reasonable approach. The naive part is if you assume that it will take you all the way to making a duck. It is also a perilous approach because you are explicitly making it harder for you to evaluate. It is, in the fullest sense of the phrase, "metric hacking."

latexr 1y ago

Respectfully, that is a reflection of the places you hang out in (like HN) and not the reality of the population. Outside the technical world it gets much worse. There are people who killed themselves because of LLMs, people who are in love with them, people who genuinely believe they have “awakened” their own private ChatGPT instance into AGI and are eschewing the real humans in their lives.

fenomas 1y ago
Naturally I'm aware of those things, but I don't think TFA or GGP were commenting on them so I wasn't either.
Xss3 1y ago
The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. It's going to take a lot to get him out of that mindset and frankly I'm dreading trying to compare and contrast imperfect human behaviour and friendships with a sycophantic AI.
- bonoboTP 1y ago
  It's surprisingly common on reddit that people talk about "my chatgpt", and they don't always seem like the type who are "in a relationship" with the bot or unlocking the secrets of the cosmos with it, but still they write "my chatgpt" and "your chatgpt". I guess the custom prompt and the available context does customize the model for them in some sense, but I suspect they likely have a wrong mental model of how this customization works. I guess they imagine it as their own little model being stored on file at OpenAI and as they interact with it, it's being shaped by it, and each time they connect, their model is retrieved from the cloud storage and they connect to it or something.
- lelanthran 1y ago
  > The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. The therapist thing might be correct, though. You can send a well-adjusted person to three renowned therapists and get three different reasons for why they need to continue sessions. No therapist ever says "Congratulations, you're perfectly normal. Now go away and come back when you have a real problem." Statistically it is vanishingly unlikely that every person who ever visited a therapist is in need of a second (more more) visit. The main problem with therapy is a lack of objectivity[1]. When people talk about what their sessions resulted in, it's always "My problem is that I'm too perfect". I've known actual bullies whose therapist apparently told them that they are too submissive and need to be more assertive. The secondary problem is that all diagnosis is based on self-reported metrics of the subject. All improvement is equally based on self-reported metrics. This is no different from prayer. You don't have a medical practice there; you've got an Imam and a sophisticated but still medically-insured way to plead with thunderstorms[2]. I fail to see how an LLM (or even the Rogerian a-x doctor in Emacs) will do worse on average. After all, if you're at a therapist and you're doing most of the talking, how would an LLM perform worse than the therapist? ---------------- [1] If I'm at a therapist, and they're asking me to do most of the talking, I would damn well feel that I am not getting my moneys worth. I'd be there primarily to learn (and practice a little) whatever tools they can teach me to handle my $PROBLEM. I don't want someone to vent at, I want to learn coping mechanisms and mitigation strategies. [2] This is not an obscure reference.
  - medvezhenok 1y ago
    Yup, this problem is why I think all therapists should ideally know behavioral genetics and evolutionary psychology (there is at least a plausibly objective measure there which is dissonance between the ancestral environment in which the brain developed and the modern day environment. And at least some amount of psychological problems can be explained by it). I am a fan of the « Beat Your Genes » podcast, and while some of the prescriptions can be a bit heavy handed, most feel intuitively right. It’s approaching human problems as intelligent mammal problems, as opposed to something in a category of its own.

positron26 1y ago

Most certainly the conversation is extremely political. There are not simply different points of view. There are competitive, gladiatorial opinions ready to ambush anyone not wearing the right colors. It's a situation where the technical conversation is drowning. I suppose this war will be fought until people are out of energy, and if reason has no place, it is reasonable to let others tire themselves out reiterating statements that are not designed to bring anyone closer to the truth.

bonoboTP 1y ago
If this tech is going to be half as impactful as its proponents predict, then I'd say it's still under-politicized. Of course the politics around it doesn't have to be knee-jerk mudslinging, but it's no surprise that politics enters the picture when the tech can significantly transform society.
- positron26 1y ago
  Go politicize it on Reddit, preferably on a political sub and not a tech sub. On this forum, I would like to expect a lot more intelligent conversation.

scarface_74 1y ago

Wait until a conversation about “serverless” comes up and someone says there is no such thing because there are servers somewhere as if everyone - especially on HN -doesn’t already know that.

Tijdreiziger 1y ago
Why would everyone know that? Not everyone has experience in sysops, especially not beginners. E.g. when I first started learning webdev, I didn’t think about ‘servers’. I just knew that if I uploaded my HTML/PHP files to my shared web host, then they appeared online. It was only much later that I realized that shared webhosting is ‘just’ an abstraction over Linux/Apache (after all, I first had to learn about those topics).
- scarface_74 1y ago
  I am saying that most people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation. I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”. It’s about as tiring as people on HN who know just a little about LLMs thinking they are sounding smart when they say they are just advanced autocomplete. Both responses are just as unproductive
  - Tijdreiziger 1y ago
    > I’m sure you knew that your code was running on computers somewhere even when you first started and wasn’t running in a literal “cloud”. Meh, I just knew that the browser would display HTML if I wrote it, and that uploading the HTML files made them available on my domain. I didn’t really think about where the files went, specifically. Try asking an average high school kid how cloud storage works. I doubt you’ll get any further than ‘I make files on my Google Docs and then they are saved there’. This is one step short of ‘well, the files must be on some system in some data center’. I really disagree that “people who come on HN and say “there is no such thing as serverless and there are servers somewhere” think they are sounding smart when they are adding nothing to the conversation.” On the contrary, it’s an invitation to beginning coders to think about what the ‘serverless’ abstraction actually means.

Wowfunhappy 1y ago

As I write this, Claude Code is currently opening and closing various media files on my computer. Sometimes it plays the file for a few seconds before closing it, sometimes it starts playback and then seeks to a different position, sometimes it fast forwards or rewinds, etc. I asked Claude to write a E-AC3 audio component so I can play videos with E-AC3 audio in the old version of QuickTime I really like using. Claude's decoder includes the ability to write debug output to a log file, so Claude is studying how QuickTime and the component interact, and it's controlling QuickTime via Applescript. Sometimes QuickTime crashes, because this ancient API has its roots in the classic Mac OS days and is not exactly good. Claude reads the crash logs on its own—it knows where they are—and continues on its way. I'm just sitting back and trying to do other things while Claude works, although it's a little distracting that something else is using my computer at the same time. I really don't want to anthropomorphize these programs, but it's just so hard when it's acting so much like a person...

godelski 1y ago
Would it help you to know that trial and error is a common tactic by machines? Yes, humans do it too, but that doesn't mean the process isn't mechanical. In fact, in computing we might call this a "brute force" approach. You don't have to cover the entire search space to brute force something, and it certainly doesn't mean you can't have optimization strategies and need to grid search (e.g. you can use Bayesian methods, multi-armed bandit approaches, or a whole world of things). I would call "fuck around and find out" a rather simple approach. It is why we use it! It is why lots of animals use it. Even very dumb animals use it. Though, we do notice more intelligent animals use more efficient optimization methods. All of this is technically hypothesis testing. Even a naive grid search. But that is still in the class of "fuck around and find out" or "brute force", right? I should also mention two important things. 1) as a human we are biased to anthropomorphize. We see faces in clouds. We tell stories of mighty beings controlling the world in an effort to explain why things happen. This is anthropomorphization of the universe itself! 2) We design LLMs (and many other large ML systems) to optimize towards human preference. This reinforces an anthropomorphized interpretation. The reason for doing this (2) is based on a naive assumption[0]: If it looks like a duck, swims like a duck, and quacks like a duck, then it *probably* is a duck. But the duck test doesn't rule out a highly sophisticated animatronic. It's a good rule of thumb, but wouldn't it also be incredibly naive to assume that it *is* a duck? Isn't the duck test itself entirely dependent on our own personal familiarity with ducks? I think this is important to remember and can help combat our own propensity for creating biases. [0] It is not a bad strategy to build in that direction. When faced with many possible ways to go, this is a very reasonable approach. The naive part is if you assume that it will take you all the way to making a duck. It is also a perilous approach because you are explicitly making it harder for you to evaluate. It is, in the fullest sense of the phrase, "metric hacking."