7 ms·

This idea that ChatGPT is "intelligent" is so absurd it's getting tiring. You do realize that the code you see regurgitated is most likely some permuted variant

by dvt 4y ago

This idea that ChatGPT is "intelligent" is so absurd it's getting tiring. You do realize that the code you see regurgitated is most likely some permuted variant of a question/answer on Stack Overflow or a pull request on Github, right? ChatGPT can't even do basic categorization[1] reliably, but you think it can understand code? [1] https://i.imgur.com/nw6rstc.png https://i.imgur.com/nw6rstc.png

topicseed 4y ago
Does it matter though? Perhaps all code we tend to write it's just regurgitated snippets from stuff we read, or stuff that's been written about enough time to make it "simple" for AI to reuse. Perhaps the goal isn't to use AI to write up an entire repo, but to go through easy utils so the highly paid engineer can focus on the core logic, the core USP of the business.
wizeman 4y ago
> You do realize that the code you see regurgitated is most likely some permuted variant of a question/answer on Stack Overflow or a pull request on Github, right? What do you think coders do? Did they learn to code by themselves, without ever looking at any preexisting code, or what?
- wizzwizz4 4y ago
  > What do you think coders do? I've seen enough programmers who can't program, but most of them can at least be taught to program. (The look on their faces when they finally start to get it… It's one of the best things.) Most people working on the Linux kernel can actually program. Most software engineering work is just plumbing existing libraries together, according to a specification given by a non-programmer. The hard part is translating the business requirements into something that a computer could understand; the exact syntax can be handled by a search engine, or a predictive text algorithm. ChatGPT can't write a kernel device driver, and it can't act as a no-code tool for non-programmers. Those are the hard parts. > “To tapping machine with hammer, 10s.; to knowing where to tap it, £10; total, £10. 10s.” https://quoteinvestigator.com/2017/03/06/tap/ https://quoteinvestigator.com/2017/03/06/tap/
  - wizeman 4y ago
    > ChatGPT can't write a kernel device driver, and it can't act as a no-code tool for non-programmers. Those are the hard parts. Oh, do I have news for you then. Look at what I just did with ChatGPT in 30 seconds (and I did not cherry-pick, these were the first answers I got!): https://gist.github.com/wizeman/b269be035308994be745025fc3378f04 https://gist.github.com/wizeman/b269be035308994be745025fc337... Now to be fair, the code is probably not totally correct, as probably there are parts still missing/wrong and there might even be compilation errors or other problems. But here's the important part: you can tell which errors or problems you've observed and ChatGPT will fix these problems for you. Exactly like what a programmer does. And sure, it cannot yet do this at scale, such as in implementing a huge kernel driver like a GPU driver. But at this rate, give it a few years and an improved version might just be able to do anything a programmer does, perhaps even autonomously if we allow it to interact with a computer like a human does.
    - dvt 4y ago
      > Look at what I just did with ChatGPT in 30 seconds (and I did not cherry-pick, these were the first answers I got!): Weird flex, as that code is like 90% boilerplate[1]. Everyone was freaking out about Copilot and no one seriously ended up using it because it just generates buggy (or copyrighted) code. It can't even handle writing unit tests with decent coverage (which is arguably the most repetitive/boring software engineering task). [1] https://github.com/ngtkt0909/linux-kernel-module-template/blob/develop/00.hello/src/hello.c https://github.com/ngtkt0909/linux-kernel-module-template/bl...
      - dragonwriter 4y ago
        > no one seriously ended up using it [citation needed] I mean, I’ve seen people claiming to use it and that it has significantly accelerating their work. On what are you basing the conclusion that it has no serious use?
        vbezhenar 4y ago
        I do use it and I'm very picky when it comes to writing code. Here's example of tiny webapp I wrote recently: https://github.com/vbezhenar/pwgen/blob/main/pwgen.html https://github.com/vbezhenar/pwgen/blob/main/pwgen.html Of course it wasn't Copilot writing it, but it definitely helps with boring parts. Like I'd write const charactersElement = document.getElementById('characters'); and rest 10 lines will be written with Copilot with minimal assistance. It's like having stupid but diligent assistant who's happy to copy&paste&adapt parts of code. I can't claim that I often use fully generated Copilot functions. Sometimes I do, often with significant rework, but that's because, as I said, I'm very picky. I paid GitHub $100 already and don't regret it. Though I think that Copilot has plenty of features ahead. For example finding obvious issues in the code would be very useful. Like typos. Another issue with Copilot is that it only generates new code. Imagine that I need to edit 10 similar lines. I edit one line and I'd like Copilot to offer other edits. Also UI is lacking. Like it generates 10 lines but I only like first line. Now I have to add 10 lines and delete 9. But I'm sure that those are obvious directions.
- delusional 4y ago
  How do you think programming actually began?
- philjohn 4y ago
  I started programming in the 80's on a Commodore 128. I had the Commodore BASIC manual, and just went from there.
  - wizeman 4y ago
    > I started programming in the 80's on a Commodore 128. I had the Commodore BASIC manual, and just went from there. ChatGPT can also learn about Commodore programming if you feed it the Commodore BASIC manual during training. My guess is that it could learn even more/better if you'd allow it to interact with a Commodore 128 computer like you did when you were learning!
    - Baeocystin 4y ago
      No need to feed it. I had it converting between the C=128's BASIC 7 and C=64's BASIC 2 without any additional context. Did a better job than 14 year old me had done back in the day, too.
rightbyte 4y ago
I am not expecting it to do novel research. It seems to yank out algorithms explained to it in plain text as good as I would write them in 30 minutes. In the hand of a newbie programmer with like one week of learning experience this thing will allow them to make usable programs. I don't think this thing will hit us with years of experience in the short term but entry level programmers might be in real danger of being flooded if OpenAI makes some "low code" IDE that just works and parses the whole set of code in the project when evaluating queries.
broast 4y ago
I personally do. I don't really find it meaningful when people show a simple buggy output. Show me a hundred iterations of you intending on helping it achieve the correct output on the same question. Even if it doesn't learn as fast as a human your age, doesn't mean it's not learning the same underlying patterns that we apply for our own abstract understanding. I haven't been shown any evidence that human understanding is different than this other than humans are much more refined with more dedicated components
edouard-harris 4y ago
Can you give a concrete example of an output ChatGPT could give — or an experiment one could run on it — that would persuade you that it's intelligent? (For whatever definition of "intelligent" you prefer.) If you can't think of such a test, then it seems strange indeed to confidently claim that ChatGPT has failed to satisfy an untestable condition. But if you can think of such a test, and any human fails it — does that mean the human who fails it isn't intelligent?
- dvt 4y ago
  You're putting the cart before the horse. Searle's Chinese Room thought experiment argues with great precision why (solely) symbol-manipulating systems don't (and will never) think. The process of cognition is deeply internal and most likely involves self-reflection, internal states, and a sense of "I-ness" which seems to be unique to mental things (unlike Searle, I'm a substance dualist so I'm admittedly a bit biased). Ergo, I'd say, at the very least, an intelligent & thinking process would need to (also) operate on non-symbolic things (which rules out computational systems), maybe on qualia, but there's probably other primitives as well.
  - baq 4y ago
    You’re dodging the question. Have you tried this thing? You can declare yourself whatever, it doesn’t care if it’s a computational system, it just does things which are hard to describe as purely recombining what it’s already seen and not for the lack of trying.
    - dvt 4y ago
      > hard to describe as purely recombining I take issue with this; imo its output looks exactly like what a neural network being fed terrabytes and terrabytes of natural language then recombining it would look like. But either way, you're making the same mistake: looking at behavior and affirming the consequent (namely: it outputs smart-looking text, therefore it must be intelligent). But this is a mistake. Its behavior implies nothing about underlying processes.
jchw 4y ago
In the real world, it tends to perform better, though. Frankly, it also probably isn't as good as "mental math" as some humans, but that's also not what it's optimized for. However, it's able to perform roughly as well as an average human for practice SAT tests, so it doesn't seem that it really has general issues with intelligence test questions, even if they are more contrived than most real world problems. Nobody ever said that ChatGPT demonstrates intelligence similar to humans in every way, but again: who the hell cares? People are going to be saying this right into their own layoffs at this rate. I'm a career SWE, so I'm not really looking forward to it anymore than anyone else. I also wonder: is there a chance you had more dialogue, but only presented this interaction?
pyinstallwoes 4y ago
This idea that Rodger the new hire on the team is "intelligent" is so absurd it's getting tiring. You do realize that the code you see regurgitated is most likely some permuted variant of a question/answer on Stack Overflow or a pull request on Github, right? Rodger can't even do basic categorization[1] reliably, but you think Rodger can understand code?

by dvt 4y ago

topicseed 4y ago

Does it matter though? Perhaps all code we tend to write it's just regurgitated snippets from stuff we read, or stuff that's been written about enough time to make it "simple" for AI to reuse. Perhaps the goal isn't to use AI to write up an entire repo, but to go through easy utils so the highly paid engineer can focus on the core logic, the core USP of the business.

wizeman 4y ago

> You do realize that the code you see regurgitated is most likely some permuted variant of a question/answer on Stack Overflow or a pull request on Github, right? What do you think coders do? Did they learn to code by themselves, without ever looking at any preexisting code, or what?

wizzwizz4 4y ago
> What do you think coders do? I've seen enough programmers who can't program, but most of them can at least be taught to program. (The look on their faces when they finally start to get it… It's one of the best things.) Most people working on the Linux kernel can actually program. Most software engineering work is just plumbing existing libraries together, according to a specification given by a non-programmer. The hard part is translating the business requirements into something that a computer could understand; the exact syntax can be handled by a search engine, or a predictive text algorithm. ChatGPT can't write a kernel device driver, and it can't act as a no-code tool for non-programmers. Those are the hard parts. > “To tapping machine with hammer, 10s.; to knowing where to tap it, £10; total, £10. 10s.” https://quoteinvestigator.com/2017/03/06/tap/ https://quoteinvestigator.com/2017/03/06/tap/
- wizeman 4y ago
  > ChatGPT can't write a kernel device driver, and it can't act as a no-code tool for non-programmers. Those are the hard parts. Oh, do I have news for you then. Look at what I just did with ChatGPT in 30 seconds (and I did not cherry-pick, these were the first answers I got!): https://gist.github.com/wizeman/b269be035308994be745025fc3378f04 https://gist.github.com/wizeman/b269be035308994be745025fc337... Now to be fair, the code is probably not totally correct, as probably there are parts still missing/wrong and there might even be compilation errors or other problems. But here's the important part: you can tell which errors or problems you've observed and ChatGPT will fix these problems for you. Exactly like what a programmer does. And sure, it cannot yet do this at scale, such as in implementing a huge kernel driver like a GPU driver. But at this rate, give it a few years and an improved version might just be able to do anything a programmer does, perhaps even autonomously if we allow it to interact with a computer like a human does.
  - dvt 4y ago
    > Look at what I just did with ChatGPT in 30 seconds (and I did not cherry-pick, these were the first answers I got!): Weird flex, as that code is like 90% boilerplate[1]. Everyone was freaking out about Copilot and no one seriously ended up using it because it just generates buggy (or copyrighted) code. It can't even handle writing unit tests with decent coverage (which is arguably the most repetitive/boring software engineering task). [1] https://github.com/ngtkt0909/linux-kernel-module-template/blob/develop/00.hello/src/hello.c https://github.com/ngtkt0909/linux-kernel-module-template/bl...
    - dragonwriter 4y ago
      > no one seriously ended up using it [citation needed] I mean, I’ve seen people claiming to use it and that it has significantly accelerating their work. On what are you basing the conclusion that it has no serious use?
      - vbezhenar 4y ago
        I do use it and I'm very picky when it comes to writing code. Here's example of tiny webapp I wrote recently: https://github.com/vbezhenar/pwgen/blob/main/pwgen.html https://github.com/vbezhenar/pwgen/blob/main/pwgen.html Of course it wasn't Copilot writing it, but it definitely helps with boring parts. Like I'd write const charactersElement = document.getElementById('characters'); and rest 10 lines will be written with Copilot with minimal assistance. It's like having stupid but diligent assistant who's happy to copy&paste&adapt parts of code. I can't claim that I often use fully generated Copilot functions. Sometimes I do, often with significant rework, but that's because, as I said, I'm very picky. I paid GitHub $100 already and don't regret it. Though I think that Copilot has plenty of features ahead. For example finding obvious issues in the code would be very useful. Like typos. Another issue with Copilot is that it only generates new code. Imagine that I need to edit 10 similar lines. I edit one line and I'd like Copilot to offer other edits. Also UI is lacking. Like it generates 10 lines but I only like first line. Now I have to add 10 lines and delete 9. But I'm sure that those are obvious directions.
delusional 4y ago
How do you think programming actually began?
philjohn 4y ago
I started programming in the 80's on a Commodore 128. I had the Commodore BASIC manual, and just went from there.
- wizeman 4y ago
  > I started programming in the 80's on a Commodore 128. I had the Commodore BASIC manual, and just went from there. ChatGPT can also learn about Commodore programming if you feed it the Commodore BASIC manual during training. My guess is that it could learn even more/better if you'd allow it to interact with a Commodore 128 computer like you did when you were learning!
  - Baeocystin 4y ago
    No need to feed it. I had it converting between the C=128's BASIC 7 and C=64's BASIC 2 without any additional context. Did a better job than 14 year old me had done back in the day, too.

rightbyte 4y ago

I am not expecting it to do novel research. It seems to yank out algorithms explained to it in plain text as good as I would write them in 30 minutes. In the hand of a newbie programmer with like one week of learning experience this thing will allow them to make usable programs. I don't think this thing will hit us with years of experience in the short term but entry level programmers might be in real danger of being flooded if OpenAI makes some "low code" IDE that just works and parses the whole set of code in the project when evaluating queries.

broast 4y ago

I personally do. I don't really find it meaningful when people show a simple buggy output. Show me a hundred iterations of you intending on helping it achieve the correct output on the same question. Even if it doesn't learn as fast as a human your age, doesn't mean it's not learning the same underlying patterns that we apply for our own abstract understanding. I haven't been shown any evidence that human understanding is different than this other than humans are much more refined with more dedicated components

edouard-harris 4y ago

Can you give a concrete example of an output ChatGPT could give — or an experiment one could run on it — that would persuade you that it's intelligent? (For whatever definition of "intelligent" you prefer.) If you can't think of such a test, then it seems strange indeed to confidently claim that ChatGPT has failed to satisfy an untestable condition. But if you can think of such a test, and any human fails it — does that mean the human who fails it isn't intelligent?

dvt 4y ago
You're putting the cart before the horse. Searle's Chinese Room thought experiment argues with great precision why (solely) symbol-manipulating systems don't (and will never) think. The process of cognition is deeply internal and most likely involves self-reflection, internal states, and a sense of "I-ness" which seems to be unique to mental things (unlike Searle, I'm a substance dualist so I'm admittedly a bit biased). Ergo, I'd say, at the very least, an intelligent & thinking process would need to (also) operate on non-symbolic things (which rules out computational systems), maybe on qualia, but there's probably other primitives as well.
- baq 4y ago
  You’re dodging the question. Have you tried this thing? You can declare yourself whatever, it doesn’t care if it’s a computational system, it just does things which are hard to describe as purely recombining what it’s already seen and not for the lack of trying.
  - dvt 4y ago
    > hard to describe as purely recombining I take issue with this; imo its output looks exactly like what a neural network being fed terrabytes and terrabytes of natural language then recombining it would look like. But either way, you're making the same mistake: looking at behavior and affirming the consequent (namely: it outputs smart-looking text, therefore it must be intelligent). But this is a mistake. Its behavior implies nothing about underlying processes.

jchw 4y ago

In the real world, it tends to perform better, though. Frankly, it also probably isn't as good as "mental math" as some humans, but that's also not what it's optimized for. However, it's able to perform roughly as well as an average human for practice SAT tests, so it doesn't seem that it really has general issues with intelligence test questions, even if they are more contrived than most real world problems. Nobody ever said that ChatGPT demonstrates intelligence similar to humans in every way, but again: who the hell cares? People are going to be saying this right into their own layoffs at this rate. I'm a career SWE, so I'm not really looking forward to it anymore than anyone else. I also wonder: is there a chance you had more dialogue, but only presented this interaction?

pyinstallwoes 4y ago

This idea that Rodger the new hire on the team is "intelligent" is so absurd it's getting tiring. You do realize that the code you see regurgitated is most likely some permuted variant of a question/answer on Stack Overflow or a pull request on Github, right? Rodger can't even do basic categorization[1] reliably, but you think Rodger can understand code?