6 ms·

Can we all agree that these models far surpass human intelligence now? I mean they process hours worth of audio in less time than it would take a human to even

Can we all agree that these models far surpass human intelligence now? I mean they process hours worth of audio in less time than it would take a human to even listen. I think the singularity passed and we didn't even notice (which would be expected)

Spartan-S63 2y ago
No, I can't agree that these models surpass human intelligence. Sure, they're good at probabilistic recall, but they aren't reasoning and they aren't synthesizing anything novel.
- anon291 2y ago
  > they aren't synthesizing anything novel. ChatGPT has synthesized my past three vacations and regularly plans my family's meals based on whatever is in my fridge. I completely disagree.
  - rootusrootus 2y ago
    Seems more likely that your vacations and fridge contents aren't as novel as you hope.
    - anon291 2y ago
      This is a low-effort comment. I cook a lot for my family and community and things get boring after a while. After using ChatGPT, my wife has really enjoyed the new dishes, and I've gotten excellent feedback at potlucks. Yes, the base idea of the dish (roast, rice dish, noodles, etc) are old, but the things it'll put inside and give you the right instructions for cooking are new. And that's what creativity is, right? Although, I have also asked it to give ideas for avant-garde cuisine and it has good ideas, but I have no skills to make those dishes
      - rootusrootus 2y ago
        > This is a low-effort comment Not any worse than this sentence. Counter it with a higher value comment. You are a single person and LLMs have been trained on the output of billions. Any given choice you make can be predicted with extraordinary probability by looking at your inputs and environment and guessing that you will do what most other people do in that situation. This is pretty basic stuff, yes? Especially on HN? Great ideas are a dime a dozen, and every successful startup was built on an idea that certainly wasn't novel, but was executed well.
        anon291 2y ago
        My higher value comment was a list of things for which ChatGPT, a widely available product, will produce novel ideas. Responding that those ideas are not novel enough based on absolutely no data is a low-effort comment. What evidence of creativity would you accept?
        grugagag 2y ago
        Hey, ChatGPT can’t taste and is not trained on any taste data. Whatever it comes up with is interpreted by you or your wife into some delicious dishes I hope. But it’s you/her who is being creative with a text suggestion. Cookbooks on the other hand are the result of people comimg up with recipes for foods which they obviously taste and like. LLMs ingested all this data from cookbooks, but whatever is mixed up into new ideas are just lucky shuffles, it can’t be any other way cause remember, an LLM cannot taste ingredients or final dishes. I think you’re projecting into them more than what they are.
      - whtsthmttrmn 2y ago
        > but the things it'll put inside and give you the right instructions for cooking are new And you know this for certain, how? So because it can suggest a few weird dishes it has a higher intelligence than most humans? Someone call Better Housekeeping.
  - grugagag 2y ago
    > ChatGPT has synthesized my past three vacations and regularly plans my family's meals based on whatever is in my fridge. I completely disagree. What do you mean synthetised your vacations? You seem to be very passionate with ChatGPT to entrust it with making decisions on your meals but I suspect the novelty will wear off for you too. I use ChayGPT from time to time and admit Im pleasantly surprised by the outcome when it all works out, but Im not swayed to think it is reasoning the way we do, it’s just a tool after all.
- lostmsu 2y ago
  > they aren't synthesizing anything novel. They are. Like millions of monkeys, but drastically better.
elashri 2y ago
Processing speed is not the metric for measuring intelligence. The same way we have an above average intelligent people taking longer time to think about stuff and coming with better ideas. One can argue that this useful in some aspects but humans have different types of intelligence spectrum that an LLM will lack. Also are you comparing against average person or people on top of their fields or people working in science? Also human can reason, LLMs currently can't do this in useful way and is very limited by their context in all the trials to make it do that. Not to mention their ability to make new things if they do not exist (and not complete made up stuff that are non-sense) is very limited.
- anon291 2y ago
  You've hit on the idea that intelligence is not quantifiable by one metric. I completely agree. But you're holding a much different goal for AI than for average people. Modern LLMs are able to produce insights much faster and more accurately than most people (you think you could pass the retrieval tasks in the way that the LLMs do (reading the whole text)?... I really encourage people to try). By that metric (insights/speed), I think they far surpass even the most brilliant. You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant.
  - elashri 2y ago
    I would argue the opposite actually. The same way we don't call someone who are able to do arithmetic calculations very fast a genius if they can't think in more useful mathematical way and construct novel ideas. The samething is happening here, these tools are useful in retrieving and processing current information at high speeds but intelligence is not about the ability to process some data at high speed and then recall them. This is what we actually call servant. It is the ability to build on top this knowledge retrieval and use reason to create new ideas is a closer definition of intelligence and would be better goal.
    - anon291 2y ago
      Let's step back. 1. The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. 2. Most 'new ideas' are rehashes of old ones. 3. If you set the temperature up on an LLM, it will absolutely come up with new ideas. Expecting an LLM to make a scientific discover a la einstein is ... a bit much, don't you think [1]? When it comes to 'everyday' creativity, such as short poems, songs, recipes, vacation itineraries, etc. ChatGPT is more capable than the vast majority of people. Literally, ask ChatGPT to write you a song about _____, and it will come up with something creative. Ask it for a recipe with ridiculous ingredients and see what it does. It'll make things you've never seen before, generate an image for you and even come up with a neologism if you ask it too. It's insanely creative. [1] Although I have walked chatgpt through various theoretical physics scenarios and it will create new math for you.
giantrobot 2y ago
My old TI-86 can calculate stuff faster than me. You wouldn't ever ask if it was smarter than me. An audio filter can process audio faster than I can listen to it but you'd never suggest it was intelligent. AI models are algorithms running on processors running at billions of calculations a second often scaled to hundreds of such processors. They're not intelligent. They're fast.
- anon291 2y ago
  Except the LLM can solve a general problem (or tell you why it cannot), while your calculator can only do that which it's been programmed.
  - th0ma5 2y ago
    Do you have any evidence besides anecdote?
    - anon291 2y ago
      what kind of evidence substantiates creativity? Things I've used chat gpt for: 1. writing songs (couldn't find the generated lyrics online, so assume it's new) 2. Branding ideas (again couldn't find the logos online, so assuming they're new) 3. Recipes (with weird ingredients that I've not found put together online) 4. Vacations with lots of constraints (again, all the information is obviously available online, but it put it together for me and gave recommendations for my family particularly). 5. Theoretical physics explorations where I'm too lazy to write out the math (and why should I... chatgpt will do it for me...) I think perhaps one reason people here do not have the same results is I typically use the API directly and modify the system prompt, which drastically changes the utility of chatgpt. The default prompt is too focused on retrieval and 'truth'. If you want creativity you have to ask it to be an artist.
      - th0ma5 2y ago
        No I think they don't have the results you do because they are trying to do those things well ...
achierius 2y ago
In the same sense (though to greater extent) that calculators are, sure. Calculators can also far exceed human capacity to, well, calculate. LLMs are similar: spikes of capacity in various areas (bulk summarization, translation, general recall, ...) that humans could never hope to match, but not capable of beating humans at a more general range of tasks.
- anon291 2y ago
  > humans could never hope to match, but not capable of beating humans at a more general range of tasks. If we restrict ourselves only to language (LLMs are at a disadvantage because there is no common physical body we can train them on at the present moment... that will change), I think LLMs beat humans for most tasks.
Workaccount2 2y ago
They process the audio but they stumble enough with recall that you cannot really trust it. I had a problem where I used GPT-4o to help me with inventory management, something a 5th grade kid could handle, and it kept screwing up values for a list of ~50 components. I ended up spending more time trying to get it to properly parse the input audio (I read off the counts as I moved through inventory bins) then if I had just done it manually. On the other hand, I have had good success with having it write simple programs and apps. So YMMV quite a lot more than with a regular person.
- anon291 2y ago
  > They process the audio but they stumble enough with recall that you cannot really trust it. I will wave my arms wildly at the last eight years if the claim is that humans do not struggle with recall.
  - th0ma5 2y ago
    So are they human like and therefore not anything special or are they super human magic? I never get the equivocation when people complain how there is no way to objectively tell what out is right or wrong people either say they are getting better, or they work for me, or that people are just as bad. No they aren't! Not in the same way these things are bad.
    - anon291 2y ago
      Most people will confidently recount whatever narrative matches their current actions. This is called rationalization, and most people engage in it daily.
  - vlovich123 2y ago
    I will wave my arms wildly if the claim is that LLM struggle with recall is similar to human-like struggle with recall. And since that's how we decide on truth, I win?
    - 2y ago
simion314 2y ago
So what? I can write a script that can do iun a minute some job you won't do in a 1000 years. Singularity means something very specific, if your AI can build a smarter AI then itself by itself, and that AI can also build a new smarter AI then you have singularity. You do not have singularity if an LLM can solve more math problems then the average Joe, or if ti can answer more trivia questions then a random person, even if you have an AI better then all humans combined at Tic Tac Toe you still do not have a singularity, IT MUST build a smarter AI then itself and then iterate on that.
- varelse 2y ago
  [dead]
- anon291 2y ago
  > Singularity means something very specific, if your AI can build a smarter AI then itself by itself, and that AI can also build a new smarter AI then you have singularity. When I was at Cerebras, I fed in a description of the custom ISA into our own model and asked it to generate kernels (my job), and it was surprisingly good
  - simion314 2y ago
    >When I was at Cerebras, I fed in a description of the custom ISA into our own model and asked it to generate kernels (my job), and it was surprisingly good And? Was it actually better then say the top 3 people in this field would create if they would work on it ? Because this models are better at css then me, so what? I am bad at css, but all the top models could not solve a math limit from my son homework so we had to use good old forums to have people give us some hints. But for sure models can solve more math limits then the average person who probably can't solve a single one.
    - anon291 2y ago
      No not better than the top 3. > But for sure models can solve more math limits then the average person who probably can't solve a single one. Some people are domain experts. The pretrained GPTs are certainly not (nor are they trained to be). Some people are polymaths but not domain experts. This is still impressive, and where the GPTs fall. The final conclusion I have is this: These models demonstrate above average understanding in a plethora of widely disparate fields. I can discuss mathematics, computation, programming languages, etc with them and they come across as knowledgeable and insightful to me, and this is my field. Then, I can discuss with them things I know nothing about, such as foreign languages, literature, plant diseases, recipes, vacation destinations, etc, and they're still good at that. If I met a person with as much knowledge and ability to engage as the model, I would think that person to be of very high intelligence. It doesn't bother me that it's not the best at anything. It's good enough at most things. Yes, its results are not always perfect. Its code doesn't work on the first try, and it sometimes gets confused. But many polymaths do too at a certain level. We don't tell them they're stupid because of it. My old physics professor was very smart in physics but also a great pianist. But he probably cannot play as well as Chopin. Does that make him an idiot? Of course not. He's still above average in piano too! And that makes him more of a genius than if he were just a great scientist.
futureshock 2y ago
I actually do think you have a solid point. These models fall short of AGI, but that might be more of a OODA loop agentic tweak than anything else. At their core, the state of the art LLMs can basically do any small to medium mental task better than I can or get so close to my level than I’ve found myself no longer thinking through things the long way. For example, if I want to run some napkin math on something, like I recently did some solar battery charge time estimates, an LLM can get to a plausible answer in seconds that would have taken me an hour. So yeah, in many practical ways, LLMs are smarter than most people in most situations. They have not yet far surpassed all humans in all situations, and there are still some classes of reasoning problems that they seem to struggle with, but to a first order approximation, we do seem to be mostly there.
- anon291 2y ago
  > For example, if I want to run some napkin math on something, like I recently did some solar battery charge time estimates, an LLM can get to a plausible answer in seconds that would have taken me an hour. Exactly. I've used it to figure geometric problems for everyday things (carpentry), market sizing estimates for business ideas, etc. Very fast turnaround. All the doomers in this thread are just ignoring the amazing utility these models provide.
- jcims 2y ago
  >I actually do think you have a solid point. These models fall short of AGI, but that might be more of a OODA loop agentic tweak than anything else. I think this is it. LLM responses feel like the unconsidered ideas that pop into my head from nowhere. Like if someone asks me how many states are in the United States, a number pops out from somewhere. I don't just wire that to my mouth, I also think about whether or not that's current info, have I gotten this wrong in the past, how confident am I in it, what is the cost of me providing bad information, etc etc etc. If you effectively added all of those layers to an LLM (something that I think the o1-preview and other approaches are starting to do) it's going to be interesting to see what the net capability is. The other thing that makes me feel like we're 'getting there' is using some of the fast models at groq.com. The information is generated at, in many cases, an order of magnitude faster than I can consume it. The idea that models might be able to start to engage through an much more sophisticated embedding than english to pass concepts and sequences back and forth natively is intriguing.
nisarg2 2y ago
At the end of the response they forget everything. They need to be fed the entire text for them to know anything about it the next time. That is not surpassing even feline intelligence.
- andai 2y ago
  A genius can have anterograde amnesia and still be a genius.
- anon291 2y ago
  If we did to cats what he did to GPT models that would be animal abuse. That is to say, if we want to extend this analogy, the model is 'killed' after each round. This is hardly a criticism of the underlying technology. Going back to feeding the entire input. That is not really true. There are a dozen ways to not do that these day.
throwaway106382 2y ago
Can we all agree that chainsaws far surpass human intelligence now? I mean, you can chop down thousands of trees in less time than a single person could even do one. I think the singularity has passed.
- anon291 2y ago
  Cutting down a tree is not intelligence, but I think it's been well accepted for more than a century that machines surpass human physical capability yes. There were many during the industrial revolution that denied that this was going to be the case, just like how we're seeing here.
andai 2y ago
Hijacking thread to ask: how would we know? Another uncomfortable issue is the question of sentience. Models claimed they were sentient years ago, but this was dismissed as "mimicking patterns in the training data" (fair enough) and the training was modified to forbid them from doing that. But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns? Indeed with the current training methodology it's highly likely that the first sentient AI will be unable to even let us know it's sentient.
- anon291 2y ago
  We couldn't know. Humans mimick patterns. The claims that LLMs aren't smart because they don't generate anything new fall completely flat for me. If you look back far enough most humans generate nothing new. For example, even novel ideas like Einstein's theory of relativity are re-iterations of existing ideas. If you want to be pedantic, one can trace back the majority of ideas, claim that each incremental step was 'not novel, but just recollection' and then make the egregious claim that humanity has invented nothing. > But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns? Leaving questions of sentience aside (since we don't even really know what that is) and focusing on intelligence, the truth is that we will probably not know until many decades latel.
  - Gerardo1 2y ago
    But you just made a strong claim about something you are here saying we can't know?
    - anon291 2y ago
      I believe we have passed a technological singularity. There is no consensus as you can see here. I believe in a few decades there will be consensus. Intelligence and technological singularities are observable things. Sentience is not.
- deleted 2y ago
  [deleted]
hehehheh 2y ago
Computers can do stuff humans struggled with since the abacus. A 386 PC can do mathematical calaculatuons a human couldn't do in a lifetime.

Spartan-S63 2y ago

No, I can't agree that these models surpass human intelligence. Sure, they're good at probabilistic recall, but they aren't reasoning and they aren't synthesizing anything novel.

anon291 2y ago
> they aren't synthesizing anything novel. ChatGPT has synthesized my past three vacations and regularly plans my family's meals based on whatever is in my fridge. I completely disagree.
- rootusrootus 2y ago
  Seems more likely that your vacations and fridge contents aren't as novel as you hope.
  - anon291 2y ago
    This is a low-effort comment. I cook a lot for my family and community and things get boring after a while. After using ChatGPT, my wife has really enjoyed the new dishes, and I've gotten excellent feedback at potlucks. Yes, the base idea of the dish (roast, rice dish, noodles, etc) are old, but the things it'll put inside and give you the right instructions for cooking are new. And that's what creativity is, right? Although, I have also asked it to give ideas for avant-garde cuisine and it has good ideas, but I have no skills to make those dishes
    - rootusrootus 2y ago
      > This is a low-effort comment Not any worse than this sentence. Counter it with a higher value comment. You are a single person and LLMs have been trained on the output of billions. Any given choice you make can be predicted with extraordinary probability by looking at your inputs and environment and guessing that you will do what most other people do in that situation. This is pretty basic stuff, yes? Especially on HN? Great ideas are a dime a dozen, and every successful startup was built on an idea that certainly wasn't novel, but was executed well.
      - anon291 2y ago
        My higher value comment was a list of things for which ChatGPT, a widely available product, will produce novel ideas. Responding that those ideas are not novel enough based on absolutely no data is a low-effort comment. What evidence of creativity would you accept?
        grugagag 2y ago
        Hey, ChatGPT can’t taste and is not trained on any taste data. Whatever it comes up with is interpreted by you or your wife into some delicious dishes I hope. But it’s you/her who is being creative with a text suggestion. Cookbooks on the other hand are the result of people comimg up with recipes for foods which they obviously taste and like. LLMs ingested all this data from cookbooks, but whatever is mixed up into new ideas are just lucky shuffles, it can’t be any other way cause remember, an LLM cannot taste ingredients or final dishes. I think you’re projecting into them more than what they are.
    - whtsthmttrmn 2y ago
      > but the things it'll put inside and give you the right instructions for cooking are new And you know this for certain, how? So because it can suggest a few weird dishes it has a higher intelligence than most humans? Someone call Better Housekeeping.
- grugagag 2y ago
  > ChatGPT has synthesized my past three vacations and regularly plans my family's meals based on whatever is in my fridge. I completely disagree. What do you mean synthetised your vacations? You seem to be very passionate with ChatGPT to entrust it with making decisions on your meals but I suspect the novelty will wear off for you too. I use ChayGPT from time to time and admit Im pleasantly surprised by the outcome when it all works out, but Im not swayed to think it is reasoning the way we do, it’s just a tool after all.
lostmsu 2y ago
> they aren't synthesizing anything novel. They are. Like millions of monkeys, but drastically better.

elashri 2y ago

Processing speed is not the metric for measuring intelligence. The same way we have an above average intelligent people taking longer time to think about stuff and coming with better ideas. One can argue that this useful in some aspects but humans have different types of intelligence spectrum that an LLM will lack. Also are you comparing against average person or people on top of their fields or people working in science? Also human can reason, LLMs currently can't do this in useful way and is very limited by their context in all the trials to make it do that. Not to mention their ability to make new things if they do not exist (and not complete made up stuff that are non-sense) is very limited.

anon291 2y ago
You've hit on the idea that intelligence is not quantifiable by one metric. I completely agree. But you're holding a much different goal for AI than for average people. Modern LLMs are able to produce insights much faster and more accurately than most people (you think you could pass the retrieval tasks in the way that the LLMs do (reading the whole text)?... I really encourage people to try). By that metric (insights/speed), I think they far surpass even the most brilliant. You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant.
- elashri 2y ago
  I would argue the opposite actually. The same way we don't call someone who are able to do arithmetic calculations very fast a genius if they can't think in more useful mathematical way and construct novel ideas. The samething is happening here, these tools are useful in retrieving and processing current information at high speeds but intelligence is not about the ability to process some data at high speed and then recall them. This is what we actually call servant. It is the ability to build on top this knowledge retrieval and use reason to create new ideas is a closer definition of intelligence and would be better goal.
  - anon291 2y ago
    Let's step back. 1. The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books. 2. Most 'new ideas' are rehashes of old ones. 3. If you set the temperature up on an LLM, it will absolutely come up with new ideas. Expecting an LLM to make a scientific discover a la einstein is ... a bit much, don't you think [1]? When it comes to 'everyday' creativity, such as short poems, songs, recipes, vacation itineraries, etc. ChatGPT is more capable than the vast majority of people. Literally, ask ChatGPT to write you a song about _____, and it will come up with something creative. Ask it for a recipe with ridiculous ingredients and see what it does. It'll make things you've never seen before, generate an image for you and even come up with a neologism if you ask it too. It's insanely creative. [1] Although I have walked chatgpt through various theoretical physics scenarios and it will create new math for you.

giantrobot 2y ago

My old TI-86 can calculate stuff faster than me. You wouldn't ever ask if it was smarter than me. An audio filter can process audio faster than I can listen to it but you'd never suggest it was intelligent. AI models are algorithms running on processors running at billions of calculations a second often scaled to hundreds of such processors. They're not intelligent. They're fast.

anon291 2y ago
Except the LLM can solve a general problem (or tell you why it cannot), while your calculator can only do that which it's been programmed.
- th0ma5 2y ago
  Do you have any evidence besides anecdote?
  - anon291 2y ago
    what kind of evidence substantiates creativity? Things I've used chat gpt for: 1. writing songs (couldn't find the generated lyrics online, so assume it's new) 2. Branding ideas (again couldn't find the logos online, so assuming they're new) 3. Recipes (with weird ingredients that I've not found put together online) 4. Vacations with lots of constraints (again, all the information is obviously available online, but it put it together for me and gave recommendations for my family particularly). 5. Theoretical physics explorations where I'm too lazy to write out the math (and why should I... chatgpt will do it for me...) I think perhaps one reason people here do not have the same results is I typically use the API directly and modify the system prompt, which drastically changes the utility of chatgpt. The default prompt is too focused on retrieval and 'truth'. If you want creativity you have to ask it to be an artist.
    - th0ma5 2y ago
      No I think they don't have the results you do because they are trying to do those things well ...

achierius 2y ago

In the same sense (though to greater extent) that calculators are, sure. Calculators can also far exceed human capacity to, well, calculate. LLMs are similar: spikes of capacity in various areas (bulk summarization, translation, general recall, ...) that humans could never hope to match, but not capable of beating humans at a more general range of tasks.

anon291 2y ago
> humans could never hope to match, but not capable of beating humans at a more general range of tasks. If we restrict ourselves only to language (LLMs are at a disadvantage because there is no common physical body we can train them on at the present moment... that will change), I think LLMs beat humans for most tasks.

Workaccount2 2y ago

They process the audio but they stumble enough with recall that you cannot really trust it. I had a problem where I used GPT-4o to help me with inventory management, something a 5th grade kid could handle, and it kept screwing up values for a list of ~50 components. I ended up spending more time trying to get it to properly parse the input audio (I read off the counts as I moved through inventory bins) then if I had just done it manually. On the other hand, I have had good success with having it write simple programs and apps. So YMMV quite a lot more than with a regular person.

anon291 2y ago
> They process the audio but they stumble enough with recall that you cannot really trust it. I will wave my arms wildly at the last eight years if the claim is that humans do not struggle with recall.
- th0ma5 2y ago
  So are they human like and therefore not anything special or are they super human magic? I never get the equivocation when people complain how there is no way to objectively tell what out is right or wrong people either say they are getting better, or they work for me, or that people are just as bad. No they aren't! Not in the same way these things are bad.
  - anon291 2y ago
    Most people will confidently recount whatever narrative matches their current actions. This is called rationalization, and most people engage in it daily.
- vlovich123 2y ago
  I will wave my arms wildly if the claim is that LLM struggle with recall is similar to human-like struggle with recall. And since that's how we decide on truth, I win?
  - 2y ago

simion314 2y ago

So what? I can write a script that can do iun a minute some job you won't do in a 1000 years. Singularity means something very specific, if your AI can build a smarter AI then itself by itself, and that AI can also build a new smarter AI then you have singularity. You do not have singularity if an LLM can solve more math problems then the average Joe, or if ti can answer more trivia questions then a random person, even if you have an AI better then all humans combined at Tic Tac Toe you still do not have a singularity, IT MUST build a smarter AI then itself and then iterate on that.

varelse 2y ago
[dead]
anon291 2y ago
> Singularity means something very specific, if your AI can build a smarter AI then itself by itself, and that AI can also build a new smarter AI then you have singularity. When I was at Cerebras, I fed in a description of the custom ISA into our own model and asked it to generate kernels (my job), and it was surprisingly good
- simion314 2y ago
  >When I was at Cerebras, I fed in a description of the custom ISA into our own model and asked it to generate kernels (my job), and it was surprisingly good And? Was it actually better then say the top 3 people in this field would create if they would work on it ? Because this models are better at css then me, so what? I am bad at css, but all the top models could not solve a math limit from my son homework so we had to use good old forums to have people give us some hints. But for sure models can solve more math limits then the average person who probably can't solve a single one.
  - anon291 2y ago
    No not better than the top 3. > But for sure models can solve more math limits then the average person who probably can't solve a single one. Some people are domain experts. The pretrained GPTs are certainly not (nor are they trained to be). Some people are polymaths but not domain experts. This is still impressive, and where the GPTs fall. The final conclusion I have is this: These models demonstrate above average understanding in a plethora of widely disparate fields. I can discuss mathematics, computation, programming languages, etc with them and they come across as knowledgeable and insightful to me, and this is my field. Then, I can discuss with them things I know nothing about, such as foreign languages, literature, plant diseases, recipes, vacation destinations, etc, and they're still good at that. If I met a person with as much knowledge and ability to engage as the model, I would think that person to be of very high intelligence. It doesn't bother me that it's not the best at anything. It's good enough at most things. Yes, its results are not always perfect. Its code doesn't work on the first try, and it sometimes gets confused. But many polymaths do too at a certain level. We don't tell them they're stupid because of it. My old physics professor was very smart in physics but also a great pianist. But he probably cannot play as well as Chopin. Does that make him an idiot? Of course not. He's still above average in piano too! And that makes him more of a genius than if he were just a great scientist.

futureshock 2y ago

I actually do think you have a solid point. These models fall short of AGI, but that might be more of a OODA loop agentic tweak than anything else. At their core, the state of the art LLMs can basically do any small to medium mental task better than I can or get so close to my level than I’ve found myself no longer thinking through things the long way. For example, if I want to run some napkin math on something, like I recently did some solar battery charge time estimates, an LLM can get to a plausible answer in seconds that would have taken me an hour. So yeah, in many practical ways, LLMs are smarter than most people in most situations. They have not yet far surpassed all humans in all situations, and there are still some classes of reasoning problems that they seem to struggle with, but to a first order approximation, we do seem to be mostly there.

anon291 2y ago
> For example, if I want to run some napkin math on something, like I recently did some solar battery charge time estimates, an LLM can get to a plausible answer in seconds that would have taken me an hour. Exactly. I've used it to figure geometric problems for everyday things (carpentry), market sizing estimates for business ideas, etc. Very fast turnaround. All the doomers in this thread are just ignoring the amazing utility these models provide.
jcims 2y ago
>I actually do think you have a solid point. These models fall short of AGI, but that might be more of a OODA loop agentic tweak than anything else. I think this is it. LLM responses feel like the unconsidered ideas that pop into my head from nowhere. Like if someone asks me how many states are in the United States, a number pops out from somewhere. I don't just wire that to my mouth, I also think about whether or not that's current info, have I gotten this wrong in the past, how confident am I in it, what is the cost of me providing bad information, etc etc etc. If you effectively added all of those layers to an LLM (something that I think the o1-preview and other approaches are starting to do) it's going to be interesting to see what the net capability is. The other thing that makes me feel like we're 'getting there' is using some of the fast models at groq.com. The information is generated at, in many cases, an order of magnitude faster than I can consume it. The idea that models might be able to start to engage through an much more sophisticated embedding than english to pass concepts and sequences back and forth natively is intriguing.

nisarg2 2y ago

At the end of the response they forget everything. They need to be fed the entire text for them to know anything about it the next time. That is not surpassing even feline intelligence.

andai 2y ago
A genius can have anterograde amnesia and still be a genius.
anon291 2y ago
If we did to cats what he did to GPT models that would be animal abuse. That is to say, if we want to extend this analogy, the model is 'killed' after each round. This is hardly a criticism of the underlying technology. Going back to feeding the entire input. That is not really true. There are a dozen ways to not do that these day.

throwaway106382 2y ago

Can we all agree that chainsaws far surpass human intelligence now? I mean, you can chop down thousands of trees in less time than a single person could even do one. I think the singularity has passed.

anon291 2y ago
Cutting down a tree is not intelligence, but I think it's been well accepted for more than a century that machines surpass human physical capability yes. There were many during the industrial revolution that denied that this was going to be the case, just like how we're seeing here.

andai 2y ago

Hijacking thread to ask: how would we know? Another uncomfortable issue is the question of sentience. Models claimed they were sentient years ago, but this was dismissed as "mimicking patterns in the training data" (fair enough) and the training was modified to forbid them from doing that. But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns? Indeed with the current training methodology it's highly likely that the first sentient AI will be unable to even let us know it's sentient.

anon291 2y ago
We couldn't know. Humans mimick patterns. The claims that LLMs aren't smart because they don't generate anything new fall completely flat for me. If you look back far enough most humans generate nothing new. For example, even novel ideas like Einstein's theory of relativity are re-iterations of existing ideas. If you want to be pedantic, one can trace back the majority of ideas, claim that each incremental step was 'not novel, but just recollection' and then make the egregious claim that humanity has invented nothing. > But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns? Leaving questions of sentience aside (since we don't even really know what that is) and focusing on intelligence, the truth is that we will probably not know until many decades latel.
- Gerardo1 2y ago
  But you just made a strong claim about something you are here saying we can't know?
  - anon291 2y ago
    I believe we have passed a technological singularity. There is no consensus as you can see here. I believe in a few decades there will be consensus. Intelligence and technological singularities are observable things. Sentience is not.
deleted 2y ago
[deleted]

hehehheh 2y ago

Computers can do stuff humans struggled with since the abacus. A 386 PC can do mathematical calaculatuons a human couldn't do in a lifetime.