4 ms·

"More powerful" here just means larger (another commenter says faster chips, but you don't even need that if you spend the next ten years training a model, alth

by rhtgrg 4y ago

"More powerful" here just means larger (another commenter says faster chips, but you don't even need that if you spend the next ten years training a model, although that could be a more accurate representation of what Nvidia is trying to say, since the alternative is a no-brainer). AI isn't really "smart," and it seems unlikely we're going to get there anytime soon (in other words, the "singularity" is still firmly out of our grasp). What we're seeing today is the fruit of training models with large swaths of data, and the limits of this approach are predictably based on the availability of such data and the resources required to convert them into these gigantic models. Framed in those terms, the "limit" we should be considering ought to be more about utility than how many resources we can afford to throw at models that might ultimately have no use. Discussions around these topics tend to get dramatized, though. In my opinion we haven't made a lick of qualitative progress in recent years in the "IQ" of AI, but we do get better at utilizing what we have, and making quantitative progress in building ever larger models.

pixl97 4y ago
Please lay out your definition of smart? For the task they are given, take textual tokens as input and make coherent output, the models seem 'smart enough' for me. The problem here is all the text in the world is still one dimension of information. The emergence of human intelligence, at least I believe, isn't from one dimensional information, but from being able to get feedback and prune the bits of information that are inconsistent with reality. There is no continuous feedback loop in our current LLMs to prune/down weight bad information in that fashion that the mind works in. There is also the "the LLM universe is what humans tell it", if we feed it bad data that's incorrectly weighted, then we'll get bad output. There is currently no scientific method model internal for the AI to test its predictions on reality. I believe all the above issues are solvable, but no idea on the time and resources necessary to accomplish them.
- rhtgrg 4y ago
  > Please lay out your definition of smart? Firstly, I agree that "smart enough" is what we should focus on here, as I tried to emphasize with my focus on utility. Secondly, I'm assuming your intent is to ask me when I would consider an AI to be "smart" in the context of my comment. I think I should avoid that question, and perhaps even retract my statement. There is plenty of evidence that ChatGPT is smarter than some humans. All of your 'problems', as you might realize, are equally applicable to some people. Putting that aside, a model would need to at least be able to internalize any corrections made (by a user, and not necessarily only in the context of training) and reliably produce results consistent with such an internalization, for me to consider it smart-er (i.e., an improvement on today's AI IQ). To be smart without qualification, it would need to go further, into the realm of being able to hold opposing ideas while functioning consistent to a given user's expectations (this one will be big, because it will act as a defense against trolls and misinformation). Bonus points if it's able to improve upon its own learning algorithms and ruminate on ideas (unsupervised learning that is inspired by previous input and not stochastic in nature) without necessitating constant external dialogue.
  - pixl97 4y ago
    Ya, as Bing showed us AI rampency is an issue that we've not tackled even in a tiny prompt space. Going to be interesting to see how we figure out how to deal with that.
tim333 4y ago
>In my opinion we haven't made a lick of qualitative progress in recent years in the "IQ" of AI We've gone from not very good to beats all humans at chess and go, and passing exams in medicine and law, all of which are things that I can't do myself. When AI can do all things better than a human would that qualify as a lick of progress?

by rhtgrg 4y ago

pixl97 4y ago

Please lay out your definition of smart? For the task they are given, take textual tokens as input and make coherent output, the models seem 'smart enough' for me. The problem here is all the text in the world is still one dimension of information. The emergence of human intelligence, at least I believe, isn't from one dimensional information, but from being able to get feedback and prune the bits of information that are inconsistent with reality. There is no continuous feedback loop in our current LLMs to prune/down weight bad information in that fashion that the mind works in. There is also the "the LLM universe is what humans tell it", if we feed it bad data that's incorrectly weighted, then we'll get bad output. There is currently no scientific method model internal for the AI to test its predictions on reality. I believe all the above issues are solvable, but no idea on the time and resources necessary to accomplish them.

rhtgrg 4y ago
> Please lay out your definition of smart? Firstly, I agree that "smart enough" is what we should focus on here, as I tried to emphasize with my focus on utility. Secondly, I'm assuming your intent is to ask me when I would consider an AI to be "smart" in the context of my comment. I think I should avoid that question, and perhaps even retract my statement. There is plenty of evidence that ChatGPT is smarter than some humans. All of your 'problems', as you might realize, are equally applicable to some people. Putting that aside, a model would need to at least be able to internalize any corrections made (by a user, and not necessarily only in the context of training) and reliably produce results consistent with such an internalization, for me to consider it smart-er (i.e., an improvement on today's AI IQ). To be smart without qualification, it would need to go further, into the realm of being able to hold opposing ideas while functioning consistent to a given user's expectations (this one will be big, because it will act as a defense against trolls and misinformation). Bonus points if it's able to improve upon its own learning algorithms and ruminate on ideas (unsupervised learning that is inspired by previous input and not stochastic in nature) without necessitating constant external dialogue.
- pixl97 4y ago
  Ya, as Bing showed us AI rampency is an issue that we've not tackled even in a tiny prompt space. Going to be interesting to see how we figure out how to deal with that.

tim333 4y ago

>In my opinion we haven't made a lick of qualitative progress in recent years in the "IQ" of AI We've gone from not very good to beats all humans at chess and go, and passing exams in medicine and law, all of which are things that I can't do myself. When AI can do all things better than a human would that qualify as a lick of progress?