7 ms·

> ...a lot of the safeguards and policy we have to manage humans own unreliability may serve us well in managing the unreliability of AI systems too. It seems

> ...a lot of the safeguards and policy we have to manage humans own unreliability may serve us well in managing the unreliability of AI systems too. It seems like an incredibly bad outcome if we accept "AI" that's fundamentally flawed in a way similar to if not worse than humans and try to work around it rather than relegating it to unimportant tasks while we work towards a standard of intelligence we'd otherwise expect from a computer. LLMs certainly appear to be the closest to real AI that we've gotten so far. But I think a lot of that is due to the human bias that language is a sign of intelligence and our measuring stick is unsuited to evaluate software specifically designed to mimic the human ability to string words together. We now have the unreliability of human language processes without most of the benefits that comes from actual human level intelligence. Managing that unreliability with systems designed for humans bakes in all the downsides without further pursuing the potential upsides from legitimate computer intelligence.

smohare 2y ago
[dead]
itchyjunk 2y ago
What is your measure of intelligence?
- rainsford 2y ago
  I honestly don't have a great one, which is less worrying than it might otherwise be since I'm not sure anyone else does either. But in a human context, I think intelligence requires some degree of creativity, self-motivation, and improvement through feedback. Put a bunch of humans on an island with various objects and the means for survival and they're going to do...something. Over enough time they're likely to do a lot of unpredictable somethings and turn coconuts into rocket ships or whatever. Put a bunch of LLMs on an equivalent island with equivalent ability to work with their environment and they're going to do precisely nothing at all. On the computer side of things, I think at a minimum I'd want intelligence capable of taking advantage of the fact that it's a deterministic machine capable of unerringly performing various operations with perfect accuracy absent a stray cosmic ray or programming bug. Star Trek's Data struggled with human emotions and things like that, but at least he typically got the warp core calculations correct. Accepting LLMs with the accuracy of a particularly lazy intern feels like it misses the point of computers entirely.
- lo_zamoyski 2y ago
  I think using the word “intelligence” when speaking of computers, beyond a kind of figure of speech, is anthropomorphizing, and it is a common pseudoscientific habit that must go. What is most characteristic about human intelligence is the ability to abstract from particular, concrete instances of things we experience. This allows us to form general concepts which are the foundation of reason. Analysis requires concepts (as concepts are what are analyzed), inference requires concepts (as we determine logical relations between them). We could say that computers might simulate intelligent behavior in some way or other, but this is observer relative not an objective property of the machine, and it is a category mistake to call computers intelligent in any way that is coherent and not the result of projecting qualities onto things that do not possess them. What makes all of this even more mystifying is that, first, the very founding papers of computer science speak of effective methods, which is by definition about methods that are completely mechanical and formal, and this stripped of the substantive conceptual content it can be applied to. Historically, this practically meant instructions given to human computers who merely completed them without any comprehension of what they were participating in. Second, computers are formal models, not physical machines. Physical machines simulate the computer formalism, but are not identical with the formalism. And as Kripke and Searle showed, there is no way in which you can say that a computer is objectively calculating anything! When we use a computer to add two numbers, you cannot say that the computer is objectively adding two numbers. It isn’t. The addition is merely an interpretation of a totally mechanistic and formal process that has been designed to be interpretable in such ways. It is analogous to reading a book. A book does not objectively contains words. It contains shaped blots of pigment on sheets of cellulose that have been assigned a conventional meaning in a culture and language. In other words, you being the words, the concepts, to the book. You bring the grammar. The book itself doesn’t have them. So we must stop confusing figurative language with literal language. AI, LLMs, whatever can be very useful, but it isn’t even wrong to call them intelligent in any literal sense.
  - Nevermark 2y ago
    > I think using the word “intelligence” when speaking of computers, beyond a kind of figure of Intelligence is what we call problem solving when the class of "problem" that a being or artifact is solving is extremely complex, involves many or near uncountable combinations of constraints, and is impossible to really characterize well. Other than examples, of data points, and some way for the person or artifact to extract something general and useful from them. Like human languages and sensibly weaving together knowledge on virtually every topic known to humans, whether any humans have put those topics together before or not. Human beings have widely ranging abilities in different kinds of thinking, despite our common design. Machines, deep learning architectures, underpinnings are software. There are endless things to try, and they are going to have a very wide set of intelligence profiles. I am staggered how quickly people downplay the abilities of these models. We literally don't know the principles they have learned (post training) for doing the kinds of processing they do. The magic of gradient algorithms. They are far from "perfect", but at what they do there is no human that can hold a candle to them. They might not be creative, but I am, and their versatility in discussing combinations of topics I am fluent in, and am not, is incredibly helpful. And unattainable from human intelligence. Unless I had a few thousand researchers, craftsman, etc. all on a Zoom call 24/7. Which might not work out so well anyway. I get that they have their glaring weaknesses. So do I! So does everyone I have ever had the pleasure to meet. If anyone can write a symbolic or numerical program to do what LLM's are doing now - without training, just code - even on some very small scale, I have yet to hear of it. I.e. someone who can demonstrate they understand the style of versatile pattern logic they learn to do. (I am very familiar with deep learning models and training algorithms and strategies. But they learn patterns suited to the data they are trained on, implicit in the data that we don't see. Knowing the very general algorithms that train them doesn't shed light on the particular pattern logic they learn for any particular problem.)
- cudgy 2y ago
  The ability to create novel solutions without a priori knowledge.
  - re-thc 2y ago
    > The ability to create novel solutions without a priori knowledge. If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.
    - collingreen 2y ago
      It takes novel solutions to walk down the street, interact with folks, dodge random incoming obstacles, respond to comments, and a bazillion other things almost everyone does all the time. I probably agree that most people aren't engaged very often and even when they are they suck at being awesome but that really isn't the bar being mentioned here.
      - re-thc 2y ago
        > It takes novel solutions to Right > but that really isn't the bar being mentioned here. Yes, the bar was "novel solutions WITHOUT priori knowledge" So you've changed the definition. Please re-read what I disagree with and it's not just the novel part i.e. if I read it all from the Internet and copied it to be successful then that fails this definition.
      - stavros 2y ago
        LLMs can interact with folks and respond to comments, which are the things on that list you should judge someone without a physical presence on.
    - bawolff 2y ago
      > If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. I think most people agree with this statement.
- throw4847285 2y ago
  If I was smarter, I could probably come up with a Kantian definition. Something about our capacity to model subjective representations as a coherent experience of the world within a unified space-time. Unfortunately, it's been a long time since I tried to read A Critique of Pure Reason, and I never understood it very well anyway. Even though my professor was one of the top Kant scholars, he admitted that reading Kant is a huge slog. So I'll leave it to Skeeter to explain. https://www.youtube.com/watch?v=W9zCI4SI6v8 https://www.youtube.com/watch?v=W9zCI4SI6v8
sigpwned 2y ago
I don’t disagree. But I also wonder if there even is an objective “right” answer in a lot of cases. If the goal is for computers to replace humans in a task, then the computer can only get the right answer for that task if humans agree what the right answer is. Outside of STEM, where AI is already having a meaningful impact (at least in my opinion), I’m not sure humans actually agree that there is a right answer in many cases, let alone what the right answer is. From that perspective, correctness is in the eye of the beholder (or the metric), and “correct” AI is somewhere between poorly defined and a contradiction. Also, I think it’s apparent that the world won’t wait for correct AI, whatever that even is, whether or not it even can exist, before it adopts AI. It sure looks like some employers are hurtling towards replacing (or, at least, reducing) human headcount with AI that performs below average at best, and expecting whoever’s left standing to clean up the mess. This will free up a lot of talent, both the people who are cut and the people who aren’t willing to clean up the resulting mess, for other shops that take a more human-based approach to staffing. I’m looking forward to seeing which side wins. I don’t expect it to be cut-and-dry. But I do expect it to be interesting.
- tharkun__ 2y ago
  Does "knowing what today is" count as "Outside STEM"? Coz my interactions with LLMs are certainly way worse than most people. Just tried it: tell me the current date please Today's date is October 3, 2023. Sorry ChatGPT, that's just wrong and your confidence in the answer is not helpful at all. It's also funny how different versions of GPT I've been interacting with always seem to return some date in October 2023, but they don't all agree on the exact day. If someone knows why, please do tell! Most real actual human people would either know the date, check their phone or their watch or be like "Oh, that's a good question lol!". But somehow GPTs always be the 1% of people that will lie to know the answer to whatever question you ask them. You know, the kind that evening talk shows will ask ask. Questions like "how do do chickens lay eggs" and you get all sorts of totally completely b0nkers but entirely "confidently told" answers. And of course they only show the ones that give the b0nkers con-man answers. Or the obviously funnily stupid people. Of course absent access to a "get the current date" function it makes sense why an LLM would behave like it does. But it also means: not AGI, sorry.
dartos 2y ago
Amen. People’s unawareness of their own personification bias with LLMs is wild.
pbreit 2y ago
I would say people are much, much worse. Compare that to the weight we place on "experts" many of whom are hopelessly compromised or dragged by mountains of baggage.
Rhapso 2y ago
Lower quality is fine economically as long as it has a good enough reduction in cost to match
- michaelteter 2y ago
  No thank you. You've just explained "race to the bottom". We've had enough of this race, and it has left us with so many poor services and products.
  - FirmwareBurner 2y ago
    The race to the bottom happens regardless whether you like it or not. Saying "no thank you" doesn't stop it. If only things in life were that easy.
    - afiori 2y ago
      Races to the bottom are incentive driven as anything else
      - FirmwareBurner 2y ago
        Sure because the incentive is always quick and easy money. Apple nearly went bankrupt in the late 90s early 00s by avoiding the race to the bottom of the PC industry till they pivoted to music players. Look at the auto makers today. Unless you can convince customers why they should pay a premium for your commodity products, you will be wiped out by your competitors who do not refuse the race to the bottom.
eternityforest 2y ago
Perhaps that kind of thing could help us finally move on from the "stupid should hurt" mindset to a real safety culture, where we value fault tolerance. We like to pretend humans can reliably execute basic tasks like telling left from right or counting to ten, or reading a four digit number, and we assume that anyone who fails at these tasks is "not even trying" But people do make these kinds of mistakes all the time, and some of them lead to patients having the wrong leg amputated. A lot of people seem to see fault tolerance as cheating or relying on crutches, it's almost like they actively want mistakes to result in major problems. If we make it so that AI failing to count the Rs doesn't kill anyone, that same attitude might help us build our equipment so that connecting the red wire to R2 instead of R3 results in a self test warning instead of a funeral announcement. Obviously I'm all for improving the underlying AI tech itself ("Maintain Competence" is a rule in crew resource management), but I'm not a super big fan of unnecessary single points of failure.

smohare 2y ago

[dead]

itchyjunk 2y ago

What is your measure of intelligence?

rainsford 2y ago
I honestly don't have a great one, which is less worrying than it might otherwise be since I'm not sure anyone else does either. But in a human context, I think intelligence requires some degree of creativity, self-motivation, and improvement through feedback. Put a bunch of humans on an island with various objects and the means for survival and they're going to do...something. Over enough time they're likely to do a lot of unpredictable somethings and turn coconuts into rocket ships or whatever. Put a bunch of LLMs on an equivalent island with equivalent ability to work with their environment and they're going to do precisely nothing at all. On the computer side of things, I think at a minimum I'd want intelligence capable of taking advantage of the fact that it's a deterministic machine capable of unerringly performing various operations with perfect accuracy absent a stray cosmic ray or programming bug. Star Trek's Data struggled with human emotions and things like that, but at least he typically got the warp core calculations correct. Accepting LLMs with the accuracy of a particularly lazy intern feels like it misses the point of computers entirely.
lo_zamoyski 2y ago
I think using the word “intelligence” when speaking of computers, beyond a kind of figure of speech, is anthropomorphizing, and it is a common pseudoscientific habit that must go. What is most characteristic about human intelligence is the ability to abstract from particular, concrete instances of things we experience. This allows us to form general concepts which are the foundation of reason. Analysis requires concepts (as concepts are what are analyzed), inference requires concepts (as we determine logical relations between them). We could say that computers might simulate intelligent behavior in some way or other, but this is observer relative not an objective property of the machine, and it is a category mistake to call computers intelligent in any way that is coherent and not the result of projecting qualities onto things that do not possess them. What makes all of this even more mystifying is that, first, the very founding papers of computer science speak of effective methods, which is by definition about methods that are completely mechanical and formal, and this stripped of the substantive conceptual content it can be applied to. Historically, this practically meant instructions given to human computers who merely completed them without any comprehension of what they were participating in. Second, computers are formal models, not physical machines. Physical machines simulate the computer formalism, but are not identical with the formalism. And as Kripke and Searle showed, there is no way in which you can say that a computer is objectively calculating anything! When we use a computer to add two numbers, you cannot say that the computer is objectively adding two numbers. It isn’t. The addition is merely an interpretation of a totally mechanistic and formal process that has been designed to be interpretable in such ways. It is analogous to reading a book. A book does not objectively contains words. It contains shaped blots of pigment on sheets of cellulose that have been assigned a conventional meaning in a culture and language. In other words, you being the words, the concepts, to the book. You bring the grammar. The book itself doesn’t have them. So we must stop confusing figurative language with literal language. AI, LLMs, whatever can be very useful, but it isn’t even wrong to call them intelligent in any literal sense.
- Nevermark 2y ago
  > I think using the word “intelligence” when speaking of computers, beyond a kind of figure of Intelligence is what we call problem solving when the class of "problem" that a being or artifact is solving is extremely complex, involves many or near uncountable combinations of constraints, and is impossible to really characterize well. Other than examples, of data points, and some way for the person or artifact to extract something general and useful from them. Like human languages and sensibly weaving together knowledge on virtually every topic known to humans, whether any humans have put those topics together before or not. Human beings have widely ranging abilities in different kinds of thinking, despite our common design. Machines, deep learning architectures, underpinnings are software. There are endless things to try, and they are going to have a very wide set of intelligence profiles. I am staggered how quickly people downplay the abilities of these models. We literally don't know the principles they have learned (post training) for doing the kinds of processing they do. The magic of gradient algorithms. They are far from "perfect", but at what they do there is no human that can hold a candle to them. They might not be creative, but I am, and their versatility in discussing combinations of topics I am fluent in, and am not, is incredibly helpful. And unattainable from human intelligence. Unless I had a few thousand researchers, craftsman, etc. all on a Zoom call 24/7. Which might not work out so well anyway. I get that they have their glaring weaknesses. So do I! So does everyone I have ever had the pleasure to meet. If anyone can write a symbolic or numerical program to do what LLM's are doing now - without training, just code - even on some very small scale, I have yet to hear of it. I.e. someone who can demonstrate they understand the style of versatile pattern logic they learn to do. (I am very familiar with deep learning models and training algorithms and strategies. But they learn patterns suited to the data they are trained on, implicit in the data that we don't see. Knowing the very general algorithms that train them doesn't shed light on the particular pattern logic they learn for any particular problem.)
cudgy 2y ago
The ability to create novel solutions without a priori knowledge.
- re-thc 2y ago
  > The ability to create novel solutions without a priori knowledge. If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.
  - collingreen 2y ago
    It takes novel solutions to walk down the street, interact with folks, dodge random incoming obstacles, respond to comments, and a bazillion other things almost everyone does all the time. I probably agree that most people aren't engaged very often and even when they are they suck at being awesome but that really isn't the bar being mentioned here.
    - re-thc 2y ago
      > It takes novel solutions to Right > but that really isn't the bar being mentioned here. Yes, the bar was "novel solutions WITHOUT priori knowledge" So you've changed the definition. Please re-read what I disagree with and it's not just the novel part i.e. if I read it all from the Internet and copied it to be successful then that fails this definition.
    - stavros 2y ago
      LLMs can interact with folks and respond to comments, which are the things on that list you should judge someone without a physical presence on.
  - bawolff 2y ago
    > If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. I think most people agree with this statement.
throw4847285 2y ago
If I was smarter, I could probably come up with a Kantian definition. Something about our capacity to model subjective representations as a coherent experience of the world within a unified space-time. Unfortunately, it's been a long time since I tried to read A Critique of Pure Reason, and I never understood it very well anyway. Even though my professor was one of the top Kant scholars, he admitted that reading Kant is a huge slog. So I'll leave it to Skeeter to explain. https://www.youtube.com/watch?v=W9zCI4SI6v8 https://www.youtube.com/watch?v=W9zCI4SI6v8

sigpwned 2y ago

I don’t disagree. But I also wonder if there even is an objective “right” answer in a lot of cases. If the goal is for computers to replace humans in a task, then the computer can only get the right answer for that task if humans agree what the right answer is. Outside of STEM, where AI is already having a meaningful impact (at least in my opinion), I’m not sure humans actually agree that there is a right answer in many cases, let alone what the right answer is. From that perspective, correctness is in the eye of the beholder (or the metric), and “correct” AI is somewhere between poorly defined and a contradiction. Also, I think it’s apparent that the world won’t wait for correct AI, whatever that even is, whether or not it even can exist, before it adopts AI. It sure looks like some employers are hurtling towards replacing (or, at least, reducing) human headcount with AI that performs below average at best, and expecting whoever’s left standing to clean up the mess. This will free up a lot of talent, both the people who are cut and the people who aren’t willing to clean up the resulting mess, for other shops that take a more human-based approach to staffing. I’m looking forward to seeing which side wins. I don’t expect it to be cut-and-dry. But I do expect it to be interesting.

tharkun__ 2y ago
Does "knowing what today is" count as "Outside STEM"? Coz my interactions with LLMs are certainly way worse than most people. Just tried it: tell me the current date please Today's date is October 3, 2023. Sorry ChatGPT, that's just wrong and your confidence in the answer is not helpful at all. It's also funny how different versions of GPT I've been interacting with always seem to return some date in October 2023, but they don't all agree on the exact day. If someone knows why, please do tell! Most real actual human people would either know the date, check their phone or their watch or be like "Oh, that's a good question lol!". But somehow GPTs always be the 1% of people that will lie to know the answer to whatever question you ask them. You know, the kind that evening talk shows will ask ask. Questions like "how do do chickens lay eggs" and you get all sorts of totally completely b0nkers but entirely "confidently told" answers. And of course they only show the ones that give the b0nkers con-man answers. Or the obviously funnily stupid people. Of course absent access to a "get the current date" function it makes sense why an LLM would behave like it does. But it also means: not AGI, sorry.

dartos 2y ago

Amen. People’s unawareness of their own personification bias with LLMs is wild.

pbreit 2y ago

I would say people are much, much worse. Compare that to the weight we place on "experts" many of whom are hopelessly compromised or dragged by mountains of baggage.

Rhapso 2y ago

Lower quality is fine economically as long as it has a good enough reduction in cost to match

michaelteter 2y ago
No thank you. You've just explained "race to the bottom". We've had enough of this race, and it has left us with so many poor services and products.
- FirmwareBurner 2y ago
  The race to the bottom happens regardless whether you like it or not. Saying "no thank you" doesn't stop it. If only things in life were that easy.
  - afiori 2y ago
    Races to the bottom are incentive driven as anything else
    - FirmwareBurner 2y ago
      Sure because the incentive is always quick and easy money. Apple nearly went bankrupt in the late 90s early 00s by avoiding the race to the bottom of the PC industry till they pivoted to music players. Look at the auto makers today. Unless you can convince customers why they should pay a premium for your commodity products, you will be wiped out by your competitors who do not refuse the race to the bottom.

eternityforest 2y ago

Perhaps that kind of thing could help us finally move on from the "stupid should hurt" mindset to a real safety culture, where we value fault tolerance. We like to pretend humans can reliably execute basic tasks like telling left from right or counting to ten, or reading a four digit number, and we assume that anyone who fails at these tasks is "not even trying" But people do make these kinds of mistakes all the time, and some of them lead to patients having the wrong leg amputated. A lot of people seem to see fault tolerance as cheating or relying on crutches, it's almost like they actively want mistakes to result in major problems. If we make it so that AI failing to count the Rs doesn't kill anyone, that same attitude might help us build our equipment so that connecting the red wire to R2 instead of R3 results in a self test warning instead of a funeral announcement. Obviously I'm all for improving the underlying AI tech itself ("Maintain Competence" is a rule in crew resource management), but I'm not a super big fan of unnecessary single points of failure.