6 ms·

Sounds like GPT 3.5 to me, version 4 is much better at code. Also, as a general rule you shouldn't trust its code output, but it's a good way to brainstorm or

Sounds like GPT 3.5 to me, version 4 is much better at code. Also, as a general rule you shouldn't trust its code output, but it's a good way to brainstorm or get started on coding something. It can give useful tips or replace stack overflow. I almost never ask it to generate code for me, but I ask it questions about existing code very often. That seems to work very well for me.

bearjaws 2y ago
GPT4 still sucks if you are using anything modern. It's knowledge of sveltekit still seems weighted to pre 1.0, it hallucinates libraries for shadcn, or radix-ui. You have to prefix your prompts with "Please use modern JS" otherwise it won't use .filter or .reduce sometimes. I agree its a good replacement for Stackoverflow, but it is often pitched as a replacement for coding. I've found using aider as the best compromise, let it build something small and revert if it doesn't work.
- water9 2y ago
  So basically, you’re trying to use stuff that came later than the data set ChatGPT was trained on and complaining it doesn’t know things that are happening in the future..
  - iamacyborg 2y ago
    How is a user supposed to know what it is and is not trained on?
    - creesch 2y ago
      In this context, it is safe to assume it isn't trained on stuff that was not available when it was trained. OpenAI lists that cutoff date next to each model: https://platform.openai.com/docs/models/continuous-model-upgrades https://platform.openai.com/docs/models/continuous-model-upg...
  - not2b 2y ago
    filter and reduce have been in JS for more than a decade now.
  - matsemann 2y ago
    But how will this affect what devs work on in the future? Will new frameworks be so hampered by this that no one will use them? I tried asking this a few days ago but got little response: > If I ask some LLM/GPT a react question I get good responses. If I ask it about a framework released after the training data was obtained, it will either not know or hallucinate. Or if it's a lesser known framework the quality will be worse than for a known framework. Same with other things like hardware manuals not being trained on yet etc. > As more and more devs rely on AI tools in their work flows, will emerging tech have a bigger hurdle than before to be adopted? Will we regress to the mean? https://news.ycombinator.com/item?id=40263033 https://news.ycombinator.com/item?id=40263033
  - hsbauauvhabzb 2y ago
    Which presumably will become a problem when gpt can use old scraper data or data full of gpt output and effectively starts eating itself and producing even worse results?
  - bearjaws 2y ago
    All the things I mentioned are from 2018-2022...
StewardMcOy 2y ago
Sorry to hijack your reply, but I've been having a lot of trouble with ChatGPT4 for code. I don't actually incorporate LLM-generated code into my work, but I often converse with ChatGPT and check what it's capable of every couple weeks so I can stay informed. By far, the most common problem is that I will ask ChatGPT 4 to write a small problem that does X, and it will create a skeleton. Lots of functions that implement at most some paramater validation and then contain comments like // Implement the rest of this function. With enough prodding, I can sometimes get it to actually fill in the function implementations one-by-one. But often times it will implement part of it and then put another one of those "implement the rest" comments in. Like for example, if I asked it to write a function to draw three circles, after prodding, it would write code to draw the first circle and then stop. Obviously, the answer here is for it to create a function that draws 1 circle and call it 3 times, but I can't fault it for taking my instructions very literally. Even when I do get it to implement full functions, when I later ask it for the full code listing, it's back to printing out functions that have "implement this" comments in them. Perhaps the fault is mine for asking it to do tasks that there aren't hundreds of implementations of on the internet.
- simonw 2y ago
  I've seen lots of people complain about this (the "ChatGPT is being lazy" bug, which OpenAI themselves said they would look at late last year) but I've not seen it myself. I wonder if it comes down to prompting style? What kind of prompts are you using that get these responses?
  - StewardMcOy 2y ago
    I didn't consider prompting style. Perhaps I'm too polite? It's a bit difficult because I usually engage in conversation first to sound out whether my understanding of the problem is sound and whether it agrees on the approach to take. So the chat is quite large before I get to the prompts, and the prompts reference earlier chat messages. But let's see if I can pull up some good examples from my history... OK, here's one. I was initially talking with it about how to best simulate restrictions from older consoles in an indie game. We chatted a bit how to most efficiently do paletted drawing on modern graphics architectures in various APIs (e.g. Vulkan, SDL). I got to asking it how exactly the SNES stored data in VRAM, and asked it to list all the PPU registers in the SNES and what they're used for. It gave me a correct answer (I double-checked). I then asked this: > Can you write a header for a C++ file named PPU.hpp with methods to read and write these registers? And it responded with this. Note that the response includes a couple "add more as needed" comments. Here's a basic outline for PPU.hpp: #ifndef PPU_HPP #define PPU_HPP #include <cstdint> class PPU { public: // Constructor and Destructor PPU(); ~PPU(); // Methods to read and write PPU registers uint8_t readRegister(uint16_t address); void writeRegister(uint16_t address, uint8_t value); // Specific methods for different PPU functionalities void setMode(uint8_t mode); void setBGTileSize(uint8_t bg, bool size16x16); void setBGCharacterBaseAddress(uint8_t bg, uint16_t address); void setBGScreenBaseAddress(uint8_t bg, uint16_t address); void setMosaicSize(uint8_t size); void setBGScroll(uint8_t bg, uint16_t horizontalOffset, uint16_t verticalOffset); // Add more methods as needed... private: // Memory-mapped PPU registers // Assuming a representation of the PPU register space uint8_t ppuRegisters[/* size of the PPU register space */]; // Helper methods to manipulate specific bits/flags in the registers // Example: void setBit(uint16_t address, uint8_t bit, bool value); // Example: bool getBit(uint16_t address, uint8_t bit); // Add more helper methods as needed... }; #endif // PPU_HPP
chrisjj 2y ago
> Sounds like GPT 3.5 to me It was. > version 4 Paywalled. > is much better at code. I'll post the question as comment to root, for you to try if you want.

bearjaws 2y ago

GPT4 still sucks if you are using anything modern. It's knowledge of sveltekit still seems weighted to pre 1.0, it hallucinates libraries for shadcn, or radix-ui. You have to prefix your prompts with "Please use modern JS" otherwise it won't use .filter or .reduce sometimes. I agree its a good replacement for Stackoverflow, but it is often pitched as a replacement for coding. I've found using aider as the best compromise, let it build something small and revert if it doesn't work.

water9 2y ago
So basically, you’re trying to use stuff that came later than the data set ChatGPT was trained on and complaining it doesn’t know things that are happening in the future..
- iamacyborg 2y ago
  How is a user supposed to know what it is and is not trained on?
  - creesch 2y ago
    In this context, it is safe to assume it isn't trained on stuff that was not available when it was trained. OpenAI lists that cutoff date next to each model: https://platform.openai.com/docs/models/continuous-model-upgrades https://platform.openai.com/docs/models/continuous-model-upg...
- not2b 2y ago
  filter and reduce have been in JS for more than a decade now.
- matsemann 2y ago
  But how will this affect what devs work on in the future? Will new frameworks be so hampered by this that no one will use them? I tried asking this a few days ago but got little response: > If I ask some LLM/GPT a react question I get good responses. If I ask it about a framework released after the training data was obtained, it will either not know or hallucinate. Or if it's a lesser known framework the quality will be worse than for a known framework. Same with other things like hardware manuals not being trained on yet etc. > As more and more devs rely on AI tools in their work flows, will emerging tech have a bigger hurdle than before to be adopted? Will we regress to the mean? https://news.ycombinator.com/item?id=40263033 https://news.ycombinator.com/item?id=40263033
- hsbauauvhabzb 2y ago
  Which presumably will become a problem when gpt can use old scraper data or data full of gpt output and effectively starts eating itself and producing even worse results?
- bearjaws 2y ago
  All the things I mentioned are from 2018-2022...

StewardMcOy 2y ago

Sorry to hijack your reply, but I've been having a lot of trouble with ChatGPT4 for code. I don't actually incorporate LLM-generated code into my work, but I often converse with ChatGPT and check what it's capable of every couple weeks so I can stay informed. By far, the most common problem is that I will ask ChatGPT 4 to write a small problem that does X, and it will create a skeleton. Lots of functions that implement at most some paramater validation and then contain comments like // Implement the rest of this function. With enough prodding, I can sometimes get it to actually fill in the function implementations one-by-one. But often times it will implement part of it and then put another one of those "implement the rest" comments in. Like for example, if I asked it to write a function to draw three circles, after prodding, it would write code to draw the first circle and then stop. Obviously, the answer here is for it to create a function that draws 1 circle and call it 3 times, but I can't fault it for taking my instructions very literally. Even when I do get it to implement full functions, when I later ask it for the full code listing, it's back to printing out functions that have "implement this" comments in them. Perhaps the fault is mine for asking it to do tasks that there aren't hundreds of implementations of on the internet.

simonw 2y ago
I've seen lots of people complain about this (the "ChatGPT is being lazy" bug, which OpenAI themselves said they would look at late last year) but I've not seen it myself. I wonder if it comes down to prompting style? What kind of prompts are you using that get these responses?
- StewardMcOy 2y ago
  I didn't consider prompting style. Perhaps I'm too polite? It's a bit difficult because I usually engage in conversation first to sound out whether my understanding of the problem is sound and whether it agrees on the approach to take. So the chat is quite large before I get to the prompts, and the prompts reference earlier chat messages. But let's see if I can pull up some good examples from my history... OK, here's one. I was initially talking with it about how to best simulate restrictions from older consoles in an indie game. We chatted a bit how to most efficiently do paletted drawing on modern graphics architectures in various APIs (e.g. Vulkan, SDL). I got to asking it how exactly the SNES stored data in VRAM, and asked it to list all the PPU registers in the SNES and what they're used for. It gave me a correct answer (I double-checked). I then asked this: > Can you write a header for a C++ file named PPU.hpp with methods to read and write these registers? And it responded with this. Note that the response includes a couple "add more as needed" comments. Here's a basic outline for PPU.hpp: #ifndef PPU_HPP #define PPU_HPP #include <cstdint> class PPU { public: // Constructor and Destructor PPU(); ~PPU(); // Methods to read and write PPU registers uint8_t readRegister(uint16_t address); void writeRegister(uint16_t address, uint8_t value); // Specific methods for different PPU functionalities void setMode(uint8_t mode); void setBGTileSize(uint8_t bg, bool size16x16); void setBGCharacterBaseAddress(uint8_t bg, uint16_t address); void setBGScreenBaseAddress(uint8_t bg, uint16_t address); void setMosaicSize(uint8_t size); void setBGScroll(uint8_t bg, uint16_t horizontalOffset, uint16_t verticalOffset); // Add more methods as needed... private: // Memory-mapped PPU registers // Assuming a representation of the PPU register space uint8_t ppuRegisters[/* size of the PPU register space */]; // Helper methods to manipulate specific bits/flags in the registers // Example: void setBit(uint16_t address, uint8_t bit, bool value); // Example: bool getBit(uint16_t address, uint8_t bit); // Add more helper methods as needed... }; #endif // PPU_HPP

chrisjj 2y ago

> Sounds like GPT 3.5 to me It was. > version 4 Paywalled. > is much better at code. I'll post the question as comment to root, for you to try if you want.