4 ms·

Processing speed is not the metric for measuring intelligence. The same way we have an above average intelligent people taking longer time to think about stuff

Processing speed is not the metric for measuring intelligence. The same way we have an above average intelligent people taking longer time to think about stuff and coming with better ideas. One can argue that this useful in some aspects but humans have different types of intelligence spectrum that an LLM will lack. Also are you comparing against average person or people on top of their fields or people working in science? Also human can reason, LLMs currently can't do this in useful way and is very limited by their context in all the trials to make it do that. Not to mention their ability to make new things if they do not exist (and not complete made up stuff that are non-sense) is very limited.

anon291 2y ago
You've hit on the idea that intelligence is not quantifiable by one metric. I completely agree. But you're holding a much different goal for AI than for average people. Modern LLMs are able to produce insights much faster and more accurately than most people (you think you could pass the retrieval tasks in the way that the LLMs do (reading the whole text)?... I really encourage people to try). By that metric (insights/speed), I think they far surpass even the most brilliant. You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant.
- elashri 2y ago
  I would argue the opposite actually. The same way we don't call someone who are able to do arithmetic calculations very fast a genius if they can't think in more useful mathematical way and construct novel ideas. The samething is happening here, these tools are useful in retrieving and processing current information at high speeds but intelligence is not about the ability to process some data at high speed and then recall them. This is what we actually call servant. It is the ability to build on top this knowledge retrieval and use reason to create new ideas is a closer definition of intelligence and would be better goal.
  - anon291 2y ago
    Let's step back. 1. The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. 2. Most 'new ideas' are rehashes of old ones. 3. If you set the temperature up on an LLM, it will absolutely come up with new ideas. Expecting an LLM to make a scientific discover a la einstein is ... a bit much, don't you think [1]? When it comes to 'everyday' creativity, such as short poems, songs, recipes, vacation itineraries, etc. ChatGPT is more capable than the vast majority of people. Literally, ask ChatGPT to write you a song about _____, and it will come up with something creative. Ask it for a recipe with ridiculous ingredients and see what it does. It'll make things you've never seen before, generate an image for you and even come up with a neologism if you ask it too. It's insanely creative. [1] Although I have walked chatgpt through various theoretical physics scenarios and it will create new math for you.
    - vlovich123 2y ago
      > The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. Depends on your definition of "truly" new since any idea could be argued to be a mix of all past ideas. But I see truly new ideas all the time without going down in the history books because most new ideas are incrementally building on what came before or are extremely niche and only a very few turn out to be a massive turning point which has a broad impact which is also only usually evident in retrospect (e.g. blue LEDs was basically trial and error and almost an approach that was given up on, transistors were believed to be impactful but not a huge revolution for computing like they turned out to be, etc etc).
      - anon291 2y ago
        > Depends on your definition of "truly" new since any idea could be argued to be a mix of all past ideas. My personal feeling when I engage in these conversations is that we humans have a cognitive bias to ascribe a human remixing of an old idea to intelligence, but an AI-model remixing of an old idea as lookup. Indeed, basically every revolutionary idea is a mix of past ideas if you look closely enough. AI is a great example. To the 'lay person' AI is novel! It's new. It can talk to you! It's amazing. But for people who've been in this field for a while, it's an incremental improvement over linear algebra, topology, functional spaces, etc.
- vlovich123 2y ago
  LLMs are probably better than you at tasks you're not good at. There's a huge gulf between a domain expert and an LLM though. If there weren't, all the humans in companies would be fired right now and replaced. Similarly, OpenAI and Anthropic are paying engineers a metric fuckton of money to work there. If LLMs were that big of a game changer right, they wouldn't be paying that much. Or if you make the argument that only the best humans are getting hired, they're still hiring interns & junior engineers. If that were the case those would be being replaced by LLMs and they're not. You're basically ignoring all the experts saying "LLMs suck at all these things that even beginning domain experts don't suck at" to generate your claim & then ignoring all evidence to the contrary. And you're ignoring the ways in which LLMs fall on their face to be creative that aren't language-based. Creative problem solving in ways they haven't been trained on is out of their domain while fully squarely in the domain of human intelligence. > You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant Computers can do arithmetic really quickly but that's not intelligence but a person computing that quickly is considered a savant. You've built up an erroneous dichotomy in your head.
  - anon291 2y ago
    But that's exactly it, right. There are some people excellent for being the expert in one field and some people are excellent because they're extremely competent at many fields. LLMs are the latter. Sure, for any domain expert, you can easily get an LLM to trip on something. But just the shear amount of things it is above average at puts it easily into the top echelon of humans. > You're basically ignoring all the experts saying "LLMs suck at all these things that even beginning domain experts don't suck at" to generate your claim & then ignoring all evidence to the contrary. Domain expertise is not the only form of intelligence. The most interesting things often lie at the intersections of domains. As I said in another comment. There are a variety of ways to judge intillegence, and no one quantifiable metric. It's like asking if Einstein is better than Mozart. I don't know... their fields are so different. However, I think it's pretty safe to say that the modern slate of LLMs fall into the top 10% of human intelligence, simply for their breath of knowledge and ability to synthesize ideas at the cross-section of any wide number of fields.

anon291 2y ago

You've hit on the idea that intelligence is not quantifiable by one metric. I completely agree. But you're holding a much different goal for AI than for average people. Modern LLMs are able to produce insights much faster and more accurately than most people (you think you could pass the retrieval tasks in the way that the LLMs do (reading the whole text)?... I really encourage people to try). By that metric (insights/speed), I think they far surpass even the most brilliant. You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant.

elashri 2y ago
I would argue the opposite actually. The same way we don't call someone who are able to do arithmetic calculations very fast a genius if they can't think in more useful mathematical way and construct novel ideas. The samething is happening here, these tools are useful in retrieving and processing current information at high speeds but intelligence is not about the ability to process some data at high speed and then recall them. This is what we actually call servant. It is the ability to build on top this knowledge retrieval and use reason to create new ideas is a closer definition of intelligence and would be better goal.
- anon291 2y ago
  Let's step back. 1. The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. 2. Most 'new ideas' are rehashes of old ones. 3. If you set the temperature up on an LLM, it will absolutely come up with new ideas. Expecting an LLM to make a scientific discover a la einstein is ... a bit much, don't you think [1]? When it comes to 'everyday' creativity, such as short poems, songs, recipes, vacation itineraries, etc. ChatGPT is more capable than the vast majority of people. Literally, ask ChatGPT to write you a song about _____, and it will come up with something creative. Ask it for a recipe with ridiculous ingredients and see what it does. It'll make things you've never seen before, generate an image for you and even come up with a neologism if you ask it too. It's insanely creative. [1] Although I have walked chatgpt through various theoretical physics scenarios and it will create new math for you.
  - vlovich123 2y ago
    > The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. Depends on your definition of "truly" new since any idea could be argued to be a mix of all past ideas. But I see truly new ideas all the time without going down in the history books because most new ideas are incrementally building on what came before or are extremely niche and only a very few turn out to be a massive turning point which has a broad impact which is also only usually evident in retrospect (e.g. blue LEDs was basically trial and error and almost an approach that was given up on, transistors were believed to be impactful but not a huge revolution for computing like they turned out to be, etc etc).
    - anon291 2y ago
      > Depends on your definition of "truly" new since any idea could be argued to be a mix of all past ideas. My personal feeling when I engage in these conversations is that we humans have a cognitive bias to ascribe a human remixing of an old idea to intelligence, but an AI-model remixing of an old idea as lookup. Indeed, basically every revolutionary idea is a mix of past ideas if you look closely enough. AI is a great example. To the 'lay person' AI is novel! It's new. It can talk to you! It's amazing. But for people who've been in this field for a while, it's an incremental improvement over linear algebra, topology, functional spaces, etc.
vlovich123 2y ago
LLMs are probably better than you at tasks you're not good at. There's a huge gulf between a domain expert and an LLM though. If there weren't, all the humans in companies would be fired right now and replaced. Similarly, OpenAI and Anthropic are paying engineers a metric fuckton of money to work there. If LLMs were that big of a game changer right, they wouldn't be paying that much. Or if you make the argument that only the best humans are getting hired, they're still hiring interns & junior engineers. If that were the case those would be being replaced by LLMs and they're not. You're basically ignoring all the experts saying "LLMs suck at all these things that even beginning domain experts don't suck at" to generate your claim & then ignoring all evidence to the contrary. And you're ignoring the ways in which LLMs fall on their face to be creative that aren't language-based. Creative problem solving in ways they haven't been trained on is out of their domain while fully squarely in the domain of human intelligence. > You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant Computers can do arithmetic really quickly but that's not intelligence but a person computing that quickly is considered a savant. You've built up an erroneous dichotomy in your head.
- anon291 2y ago
  But that's exactly it, right. There are some people excellent for being the expert in one field and some people are excellent because they're extremely competent at many fields. LLMs are the latter. Sure, for any domain expert, you can easily get an LLM to trip on something. But just the shear amount of things it is above average at puts it easily into the top echelon of humans. > You're basically ignoring all the experts saying "LLMs suck at all these things that even beginning domain experts don't suck at" to generate your claim & then ignoring all evidence to the contrary. Domain expertise is not the only form of intelligence. The most interesting things often lie at the intersections of domains. As I said in another comment. There are a variety of ways to judge intillegence, and no one quantifiable metric. It's like asking if Einstein is better than Mozart. I don't know... their fields are so different. However, I think it's pretty safe to say that the modern slate of LLMs fall into the top 10% of human intelligence, simply for their breath of knowledge and ability to synthesize ideas at the cross-section of any wide number of fields.