3 ms·

Hm. I've not had any problems using GPT4 with code that it has never seen. But sure, I'll try that... Edit: I took the GIO Hello World sample. I fed it into th

Hm. I've not had any problems using GPT4 with code that it has never seen. But sure, I'll try that... Edit: I took the GIO Hello World sample. I fed it into the GPT-4 API. And I swear to God, first try, first click: https://gist.github.com/FeepingCreature/bde583aa8a2e8e5e1c569190af5aa3f8 https://gist.github.com/FeepingCreature/bde583aa8a2e8e5e1c56... Near-perfect success. One extra import, sure, I'll give it that one, LLMs can't correct when they notice they don't actually need an import in the code. But then it works. Gist link cause I use OpenRouter against the GPT-4 API, maybe that's why? Maybe the "stochastic parrot" vs "oncoming apocalypse" perspective genuinely represents that ..... GPT just doesn't work for some people?? Did they fuck the ChatGPT4 web interface enough somehow so that it's just inept, but somehow not affect the API? Edit: > In my research, 35/37 attempts don't even compile. It tries the same mistakes over and over again. It fails to make reasonable assessment of the compilation errors. Important note: there's a theory that if you've got it making mistakes a few times, it thinks that it's playing "the sort of AI that fucks up a lot" and starts making more mistakes, reinforcing its role. Start over fresh if it seems strangely incapable. Like, it is absolutely possible to use GPT-4 in a way that results in it systematically being incapable of the most basic tasks. These things are not reliably competent; but they're occasionally competent! In my experience, even with completely novel tasks in completely novel environments, the competence is "in there" and can often be elicited with dedicated poking. That's why I think LLMs are enough for a takeoff with enough scale (and basic online learning probably): in my opinion, it's not a matter of attaining the skills but removing the roadblocks. It's like the joke about seeing an old man playing chess with a labrador, and the labrador just carefully picking up figures and moving them around the board, every time in a legal move, and you say "that's amazing, a chess-playing dog!" And the old man scoffs and says, "Nonsense. His endgame is rubbish." Three years ago we didn't think a dog could understand pawn promotion at all. edit: Hey, share your repros? I'm genuinely curious what's going on here now. edit: > Watch how it can't even spell. This is a specific issue with the current generation of LLMs and should be thought of more as a disorder than a fundamental inability. LLMs don't actually ever see letters, they see BPEs, which consist of one up to n letters. For instance, the word "corporation" looks to the AI more like "<27768>". So like- yes. It can not spell. It fundamentally, architecturally, cannot perceive letters in the words you give it; its ability to manually split words into letters is based on chance memorization. Instead, try getting it to split the word into letters in code.

trimethylpurine 3y ago
I see. This is no longer a good example because of the additional functionality and documentation that's been added to the Gio package since my experiment took place. So, in this case, I can't show you behind the curtain without a lot more effort. But even given your example, you've essentially told me about a parrot that says "four!" when someone says "two plus two." Even if the same parrot can recite the entire US constitution backwards and forwards, it doesn't actually understand what it's saying. I just spent a few minutes on YouTube watching parrots. They can be very convincing with the right training. For me it's the same thing. It's uncanny, but it's not mutiny. It's barely even in the direction of intelligence.
- FeepingCreature 3y ago
  This just seems a bizarre argument to me. I've written entire (fully novel!) programs with these things. If this is a parrot, we should hire parrots for software development and worry about them being the next dominant species. Like, sure it doesn't know things that have zero documentation. I'm not saying it's human-level yet! That's never been the argument! Look, you pulled this example out. You said it doesn't even work 37/38 times. Do you have another example? I just really don't understand what the basis for this argument is. I've found the thing a capable programmer within its limits!
  - trimethylpurine 3y ago
    You got my curiosity going so I went for the experiment again. It turns out my example is still very much valid. It's still not able to do basic work with the Gio package despite great documentation from the Gio folks, if you ask me. In this case, the LLM is struggling to version match. https://chat.openai.com/share/d5e39e52-9140-4f74-9fef-04fc34ba65ad https://chat.openai.com/share/d5e39e52-9140-4f74-9fef-04fc34... I cut this one short because I'm not that interested to go 40 questions again. I have better things to do and I have already seen the outcome. It's clear the LLM is working as intended, which is neither for programming, nor for problem solving. It's giving the next most probable response. That's truly great for repeating and reintegrating existing solutions. But repeating is not what's needed for the "kill all humans" scenario. That scenario requires outreasoning humans. Processing power is not remotely the issue. I've never seen a theoretical solution that purports to give a machine the ability to actually reason, and because of that, I don't believe that ChatGPT, or any other LLM, even begins to approach "intelligence." If you are having success with it, I think it's because you're helping it more than you'd like to admit, or your solutions might not be as novel as you think. Probably the prior, but I must admit, I've wrongly thought I was working on new stuff many times before. I think that's more common than we think. Or at least I'd like to think that so I can feel good about myself. In any case, I won't be hiring ChatGPT or parrots; and for the same reason. I need developers that can follow directions, solve problems in better ways than what is readily available, and do it faster than my competitors. And I'm not even in some cutting edge business, I'm just building integrations, specifically in the manufacturing and distribution industry. Even in something so mundane, I've hardly found ChatGPT to be truly useful for much more than a GitHub search tool, and to suggest cocktails when my liquor selection runs scarce. Worth $20? Sure. But it's certainly not scaring me.

trimethylpurine 3y ago

I see. This is no longer a good example because of the additional functionality and documentation that's been added to the Gio package since my experiment took place. So, in this case, I can't show you behind the curtain without a lot more effort. But even given your example, you've essentially told me about a parrot that says "four!" when someone says "two plus two." Even if the same parrot can recite the entire US constitution backwards and forwards, it doesn't actually understand what it's saying. I just spent a few minutes on YouTube watching parrots. They can be very convincing with the right training. For me it's the same thing. It's uncanny, but it's not mutiny. It's barely even in the direction of intelligence.

FeepingCreature 3y ago
This just seems a bizarre argument to me. I've written entire (fully novel!) programs with these things. If this is a parrot, we should hire parrots for software development and worry about them being the next dominant species. Like, sure it doesn't know things that have zero documentation. I'm not saying it's human-level yet! That's never been the argument! Look, you pulled this example out. You said it doesn't even work 37/38 times. Do you have another example? I just really don't understand what the basis for this argument is. I've found the thing a capable programmer within its limits!
- trimethylpurine 3y ago
  You got my curiosity going so I went for the experiment again. It turns out my example is still very much valid. It's still not able to do basic work with the Gio package despite great documentation from the Gio folks, if you ask me. In this case, the LLM is struggling to version match. https://chat.openai.com/share/d5e39e52-9140-4f74-9fef-04fc34ba65ad https://chat.openai.com/share/d5e39e52-9140-4f74-9fef-04fc34... I cut this one short because I'm not that interested to go 40 questions again. I have better things to do and I have already seen the outcome. It's clear the LLM is working as intended, which is neither for programming, nor for problem solving. It's giving the next most probable response. That's truly great for repeating and reintegrating existing solutions. But repeating is not what's needed for the "kill all humans" scenario. That scenario requires outreasoning humans. Processing power is not remotely the issue. I've never seen a theoretical solution that purports to give a machine the ability to actually reason, and because of that, I don't believe that ChatGPT, or any other LLM, even begins to approach "intelligence." If you are having success with it, I think it's because you're helping it more than you'd like to admit, or your solutions might not be as novel as you think. Probably the prior, but I must admit, I've wrongly thought I was working on new stuff many times before. I think that's more common than we think. Or at least I'd like to think that so I can feel good about myself. In any case, I won't be hiring ChatGPT or parrots; and for the same reason. I need developers that can follow directions, solve problems in better ways than what is readily available, and do it faster than my competitors. And I'm not even in some cutting edge business, I'm just building integrations, specifically in the manufacturing and distribution industry. Even in something so mundane, I've hardly found ChatGPT to be truly useful for much more than a GitHub search tool, and to suggest cocktails when my liquor selection runs scarce. Worth $20? Sure. But it's certainly not scaring me.