5 ms·

Seems like a long-winded way of saying, "no". ;)

by md_ 3y ago

jackmott42 3y ago
It is telling though, that a year ago myself and I think most programmers would have just quickly said "nope definitely not", whereas now you kind of have to get long winded to say no, and many of us are not saying no but maybe AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it. What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such? Gonna be fun.
- uoaei 3y ago
  > many of us are not saying no but maybe I'm not sure if this is a controversial stance, but not all opinions are equal. HN commenters span a wide range of competencies, but very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to.
  - UIUC_06 3y ago
    Gene Wilder meme: Tell me all about how your opinions are better than everyone else's!
    - uoaei 3y ago
      You will note I did not proffer any of my own. To avoid Dunning-Kruger, you must know what things you don't have sufficient knowledge about and avoid speaking as if you do. Also please don't "meme" on HN, it degrades discussion quality very quickly. There are other venues for that.
      - UIUC_06 3y ago
        oh, I beg your pardon. You did say "very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to." I thought you meant that your opinions are worth listening to, unlike the other HN'ers. Are they not?
        uoaei 3y ago
        That's pretty irrelevant within the context of this comment. I said what I said; I didn't say anything else. I have thoughts about this and related topics in my comment history, but still focus pretty tightly on saying things I can reasonably back up, avoiding speculation about things where my understanding is murkier. So in that wider sense, yes, I am living my values.
- md_ 3y ago
  I think it's pretty surprising, and counterintuitive, how powerful generative language models are. It turns out that a lot of intellectual tasks are sort of weakly simulatable by just doing a great job at next-token prediction. I don't think that means that "good next-token prediction" is identical to "AGI". As Yan LeCun says, I think LLMs are an off-ramp on the highway to AGI. But they are powerful at specific tasks. Unfortunately, instead of evaluating their actual utility at a specific task, people just seem to think "if we throw AI at the problem, we will solve it," which is sort of like every other tech bubble ever.
  - version_five 3y ago
    It turns out that a lot of intellectual tasks are sort of weakly simulatable by just doing a great job at next-token prediction. Yes, this is a good way of putting it. I've been saying for years, it's less that we're making big discoveries about what "AI" can do, and more that we're showing that many things humans do that appear complex actually reduce to something pretty simple. But that simple thing is still just fitting a pattern. It's the cases where it doesn't work, even if it only fails 1% of the time, that define the difference between pattern matching and actual human intelligence. One thing that follows from that (that people don't like) is that we actually need to move goalposts about how intelligence is defined. "Pass a turing test" is not very valuable now. And as mode tasks are shown to be possible with pattern matching / next token prediction, we need to further refine tests away from these tasks to settle on a good definition of what separates human intelligence. It should be obvious that the distinction is there, but it's still tough to nail down. (I'd argue that by defining a "task" you've already done most of the work to solving it, so it's not to exciting to learn that AI can finish the job)
    - jackmott42 3y ago
      I am not convinced actual human intelligence is all that different. Not all people do the kind of reasoning we talk about when we try to distinguish between GPT and humans, and even those of us humans that CAN do it often don't bother. We may find that augmenting something as simple or nearly as simple as ChatGPT with a few extra tools to handle step by step reasoning (see the Wolfram plugin) may get you there.

Seems like a long-winded way of saying, "no". ;)

by md_ 3y ago

Seems like a long-winded way of saying, "no". ;)

It is telling though, that a year ago myself and I think most programmers would have just quickly said "nope definitely not", whereas now you kind of have to get long winded to say no, and many of us are not saying no but maybe AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it. What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such? Gonna be fun.

uoaei 3y ago
> many of us are not saying no but maybe I'm not sure if this is a controversial stance, but not all opinions are equal. HN commenters span a wide range of competencies, but very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to.
- UIUC_06 3y ago
  Gene Wilder meme: Tell me all about how your opinions are better than everyone else's!
  - uoaei 3y ago
    You will note I did not proffer any of my own. To avoid Dunning-Kruger, you must know what things you don't have sufficient knowledge about and avoid speaking as if you do. Also please don't "meme" on HN, it degrades discussion quality very quickly. There are other venues for that.
    - UIUC_06 3y ago
      oh, I beg your pardon. You did say "very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to." I thought you meant that your opinions are worth listening to, unlike the other HN'ers. Are they not?
      - uoaei 3y ago
        That's pretty irrelevant within the context of this comment. I said what I said; I didn't say anything else. I have thoughts about this and related topics in my comment history, but still focus pretty tightly on saying things I can reasonably back up, avoiding speculation about things where my understanding is murkier. So in that wider sense, yes, I am living my values.
md_ 3y ago
I think it's pretty surprising, and counterintuitive, how powerful generative language models are. It turns out that a lot of intellectual tasks are sort of weakly simulatable by just doing a great job at next-token prediction. I don't think that means that "good next-token prediction" is identical to "AGI". As Yan LeCun says, I think LLMs are an off-ramp on the highway to AGI. But they are powerful at specific tasks. Unfortunately, instead of evaluating their actual utility at a specific task, people just seem to think "if we throw AI at the problem, we will solve it," which is sort of like every other tech bubble ever.
- version_five 3y ago
  It turns out that a lot of intellectual tasks are sort of weakly simulatable by just doing a great job at next-token prediction. Yes, this is a good way of putting it. I've been saying for years, it's less that we're making big discoveries about what "AI" can do, and more that we're showing that many things humans do that appear complex actually reduce to something pretty simple. But that simple thing is still just fitting a pattern. It's the cases where it doesn't work, even if it only fails 1% of the time, that define the difference between pattern matching and actual human intelligence. One thing that follows from that (that people don't like) is that we actually need to move goalposts about how intelligence is defined. "Pass a turing test" is not very valuable now. And as mode tasks are shown to be possible with pattern matching / next token prediction, we need to further refine tests away from these tasks to settle on a good definition of what separates human intelligence. It should be obvious that the distinction is there, but it's still tough to nail down. (I'd argue that by defining a "task" you've already done most of the work to solving it, so it's not to exciting to learn that AI can finish the job)
  - jackmott42 3y ago
    I am not convinced actual human intelligence is all that different. Not all people do the kind of reasoning we talk about when we try to distinguish between GPT and humans, and even those of us humans that CAN do it often don't bother. We may find that augmenting something as simple or nearly as simple as ChatGPT with a few extra tools to handle step by step reasoning (see the Wolfram plugin) may get you there.