6 ms·

It's easily verifiable, but it may still waste time. I've had many cases where ChatGPT makes up functions that do exactly what I need, but then I find out these

It's easily verifiable, but it may still waste time. I've had many cases where ChatGPT makes up functions that do exactly what I need, but then I find out these functions don't actually exist. This may not happen very often for super popular languages like Python or Javascript where training data is huge, but it happens all the time for the long-tail of languages. In those cases, it would've been faster for me to do a regular search. I do agree with the overall point though. If you understand when to use it and when it's more likely to give you nonsensical answers, it can save a huge amount of time. But when I ask it about a topic that I don't know enough about to immediately verify the answer myself I'm forced to double check the answers for validity, which kind of defeats the purpose. The best queries to ChatGPT are cases where I know what the answer should look like, I just forgot the syntax or some details. Bash scripts or Kubernetes manifests are examples here, I know them, I just keep forgetting the keywords because I only touch them every few weeks. And don't get me started about asking ChatGPT about more general topics in e.g. economics or finance. What you get is a well-written summary of popular news and reddit opinions, which is dangerous if it's presented as "the truth" - The big mistake here is that the training procedure assumes that the amount of data correlates with correctness, which isn't true for many topics that involve politics or similar kinds of incentives where people and news spread what conveniently benefits them and gets clicks.

wildpeaks 4y ago
Wasting time and having to be constantly vigilant is exhausting and a slippery slope that makes it easier to fall for deceptive content and settling for "I don't know, it's probably close enough" instead of insisting on precision and accuracy. Humans take a lot of shortcuts (such as believing more easily the same facts presented with a confident tone) and the "firehose of bs" exploits it: this was already the case before generative AI, but AI amplifies the industrial-scale imbalance between the time needed to generate partially incorrect data and the amount of time/energy required to validate.
- discreteevent 4y ago
  Agreed that it is a slippery slope. Programming is understanding - like writing or teaching is understanding. To really understand something, we must construct it ourselves. We will be inclined to skip this step. This comment sums it up well: > Salgat 8 days ago > The problem with ML is that it's pattern recognition, it's an approximation. Code is absolute, it's logic that is interpreted very literally and very exactly. This is what makes it so dangerous for coding; it creates code that's convincing to humans but with deviations that allow for all sorts of bugs. And the worst part is, since you didn't write the code, you may not have the skills (or time) to figure out if those bugs exist https://news.ycombinator.com/item?id=34140585 https://news.ycombinator.com/item?id=34140585
  - MetaWhirledPeas 4y ago
    > To really understand something, we must construct it ourselves. I think the real power of these bots will be to lead us down this path, as opposed to it doing everything for us. We can ask it to justify and explain its solution and it will do its best. If we're judicious with this we can use it to build our own understanding and just trash the AI's output.
- alpaca128 4y ago
  How is that worse than having to look at every online post's date to estimate whether the solution is out of date? Or two StackOverflow results where one is incorrectly marked as duplicate and in the other the person posting the answer is convinced that the question is wrong. ChatGPT can completely cut out the online search and give an answer directly about things like compiler errors, and elaborate further on any detail in the answer. I think that 2-3 further GPT generations down the line it will be worth the time for some applications. The problem I see is less the overall quality of responses but people overestimating on where it can be used productively. But that will always be a problem with new tech, see Tesla drivers who regularly take a nap in the car because it didn't crash yet.
  - mannykannot 4y ago
    Unless the training of ChatGPT has a mechanism to excise the influence of now out-of-date training input, it will become increasingly more likely to give an outdated response as time goes by. Does its training have this capability?
    - brookst 4y ago
      Yes. The trick is to use it as an LLM and not a procedural, transactional data set. For instance, “how do I create a new thread in Python”. Then ask “how do I create a new thread in Python 3.8”. The answers will (probably) be different. Any interface to chatgpt or similar can help users craft good prompts this way. It just takes thinking about the problem a little differently. One wildly inefficient but illustrative approach is to use chatgpt itself to optimize the queries. For the Python threading example, I just asked it “ A user is asking a search engine ‘how do I create threads in Python’. What additional information will help ensure the results are most useful to the user?”. The results: > The user's current level of programming experience and knowledge of Python > The specific version of Python being used > The desired use case for the threads (e.g. parallel processing, concurrent execution) > Any specific libraries or modules the user wants to use for thread creation > The operating system the user is running on (as this may affect the availability of certain threading options) So if you imagine something like Google autocomplete, but running this kind of optimization advice while the user builds their query, the AI can help guide the user to being specific enough to get the most relevant results.
janandonly 4y ago
I am foreseeing a future in which programming language designers match the most sought after functions in google/bing/chatgpt and then implement those that do not yet exist because apparently there is a real need for those.
- tgv 4y ago
  You can also call it an artificially created need. Many functions exist, but have a different name.
- VirusNewbie 4y ago
  Yes, I had the same thought. LLM’s might be instrumental in new language design. If it can understand the most common structures being used, it makes sense to build libraries, macros, or language features.
weird-eye-issue 4y ago
It does this for Python and JS too
atmosx 4y ago
I agree, chatGPT3 shines when the operator has domain knowledge. Otherwise it's a hit or miss.
bambax 4y ago
I agree. ChatGPT is really really bad. It just makes up stuff and wraps its fabrications in an air of authority. A "bullshit sandwich" if you will. When one tells people this we get the reply "but so do random blogs! or reddit comments!". Well yes, but they're just random blogs and reddit comments, often peppered with syntactic and spelling mistakes, non sequiturs, and other absurdities. Nobody would take them seriously. ChatGPT is very different. It doesn't say "this random redditor says this, and this other random redditor says the exact opposite, so IDK, I'm just a machine, please make up your mind". What it says is "this is the absolute truth that I, a 'large language model', have been able to extract from the vast amount of information I have been trained on. You can rely on it with confidence." I'm sorry to sound hyperbolic but this cannot end well.
- int_19h 4y ago
  ChatGPT doesn't say anything of the sort. In fact, it will vehemently insist that what it says is not necessarily true or accurate if you challenge it.
  - bambax 4y ago
    I'm sorry but this is demonstrably false. I have posted examples of this on HN before. Yes, if you tell ChatGPT that it's wrong, in some cases it says "I'm sorry" and tries again (and produces some other random guess). But if you ask it "are you sure?" it invariably affirms that yes, it's sure and it's in the right.
    - int_19h 4y ago
      Hm, you're right. I'm pretty sure that it wasn't so gung-ho when I played with it earlier, but now even very explicit instructions along the lines of "you should only answer "yes" if it is absolutely certain that this is the correct answer" still give this response. Ditto for prompts like "is it possible that your answer was incorrect?"
- whodev 4y ago
  I like bouncing my code problems off ChatGPT, it can give me an answer and I don't feel bad if I forgot something simple. The issue is I've had it give me completely wrong code only for it to be like "I'm sorry" and provide a second incorrect response.
throwaway4aday 4y ago
Using a purpose built (or trained I guess) model for code generation would likely have better results. GitHub copilot is useful for this reason. I find ChatGPT for code is mainly useful if you want to instruct it in natural language to make subsequent changes to the output.
themk 4y ago
If you follow up about the nonexistent function, it will often implement it for you. The other thing that I've had success with is asking for references for the information, which will often link you to the relevant docs.
rmbyrro 4y ago
If you ask, there's a good chance ChatGPT can create that function for you. Just tell it: "That function `xyz()` doesn't exist in the library, can you write it for me?"

wildpeaks 4y ago

Wasting time and having to be constantly vigilant is exhausting and a slippery slope that makes it easier to fall for deceptive content and settling for "I don't know, it's probably close enough" instead of insisting on precision and accuracy. Humans take a lot of shortcuts (such as believing more easily the same facts presented with a confident tone) and the "firehose of bs" exploits it: this was already the case before generative AI, but AI amplifies the industrial-scale imbalance between the time needed to generate partially incorrect data and the amount of time/energy required to validate.

discreteevent 4y ago
Agreed that it is a slippery slope. Programming is understanding - like writing or teaching is understanding. To really understand something, we must construct it ourselves. We will be inclined to skip this step. This comment sums it up well: > Salgat 8 days ago > The problem with ML is that it's pattern recognition, it's an approximation. Code is absolute, it's logic that is interpreted very literally and very exactly. This is what makes it so dangerous for coding; it creates code that's convincing to humans but with deviations that allow for all sorts of bugs. And the worst part is, since you didn't write the code, you may not have the skills (or time) to figure out if those bugs exist https://news.ycombinator.com/item?id=34140585 https://news.ycombinator.com/item?id=34140585
- MetaWhirledPeas 4y ago
  > To really understand something, we must construct it ourselves. I think the real power of these bots will be to lead us down this path, as opposed to it doing everything for us. We can ask it to justify and explain its solution and it will do its best. If we're judicious with this we can use it to build our own understanding and just trash the AI's output.
alpaca128 4y ago
How is that worse than having to look at every online post's date to estimate whether the solution is out of date? Or two StackOverflow results where one is incorrectly marked as duplicate and in the other the person posting the answer is convinced that the question is wrong. ChatGPT can completely cut out the online search and give an answer directly about things like compiler errors, and elaborate further on any detail in the answer. I think that 2-3 further GPT generations down the line it will be worth the time for some applications. The problem I see is less the overall quality of responses but people overestimating on where it can be used productively. But that will always be a problem with new tech, see Tesla drivers who regularly take a nap in the car because it didn't crash yet.
- mannykannot 4y ago
  Unless the training of ChatGPT has a mechanism to excise the influence of now out-of-date training input, it will become increasingly more likely to give an outdated response as time goes by. Does its training have this capability?
  - brookst 4y ago
    Yes. The trick is to use it as an LLM and not a procedural, transactional data set. For instance, “how do I create a new thread in Python”. Then ask “how do I create a new thread in Python 3.8”. The answers will (probably) be different. Any interface to chatgpt or similar can help users craft good prompts this way. It just takes thinking about the problem a little differently. One wildly inefficient but illustrative approach is to use chatgpt itself to optimize the queries. For the Python threading example, I just asked it “ A user is asking a search engine ‘how do I create threads in Python’. What additional information will help ensure the results are most useful to the user?”. The results: > The user's current level of programming experience and knowledge of Python > The specific version of Python being used > The desired use case for the threads (e.g. parallel processing, concurrent execution) > Any specific libraries or modules the user wants to use for thread creation > The operating system the user is running on (as this may affect the availability of certain threading options) So if you imagine something like Google autocomplete, but running this kind of optimization advice while the user builds their query, the AI can help guide the user to being specific enough to get the most relevant results.

janandonly 4y ago

I am foreseeing a future in which programming language designers match the most sought after functions in google/bing/chatgpt and then implement those that do not yet exist because apparently there is a real need for those.

tgv 4y ago
You can also call it an artificially created need. Many functions exist, but have a different name.
VirusNewbie 4y ago
Yes, I had the same thought. LLM’s might be instrumental in new language design. If it can understand the most common structures being used, it makes sense to build libraries, macros, or language features.

weird-eye-issue 4y ago

It does this for Python and JS too

atmosx 4y ago

I agree, chatGPT3 shines when the operator has domain knowledge. Otherwise it's a hit or miss.

bambax 4y ago

I agree. ChatGPT is really really bad. It just makes up stuff and wraps its fabrications in an air of authority. A "bullshit sandwich" if you will. When one tells people this we get the reply "but so do random blogs! or reddit comments!". Well yes, but they're just random blogs and reddit comments, often peppered with syntactic and spelling mistakes, non sequiturs, and other absurdities. Nobody would take them seriously. ChatGPT is very different. It doesn't say "this random redditor says this, and this other random redditor says the exact opposite, so IDK, I'm just a machine, please make up your mind". What it says is "this is the absolute truth that I, a 'large language model', have been able to extract from the vast amount of information I have been trained on. You can rely on it with confidence." I'm sorry to sound hyperbolic but this cannot end well.

int_19h 4y ago
ChatGPT doesn't say anything of the sort. In fact, it will vehemently insist that what it says is not necessarily true or accurate if you challenge it.
- bambax 4y ago
  I'm sorry but this is demonstrably false. I have posted examples of this on HN before. Yes, if you tell ChatGPT that it's wrong, in some cases it says "I'm sorry" and tries again (and produces some other random guess). But if you ask it "are you sure?" it invariably affirms that yes, it's sure and it's in the right.
  - int_19h 4y ago
    Hm, you're right. I'm pretty sure that it wasn't so gung-ho when I played with it earlier, but now even very explicit instructions along the lines of "you should only answer "yes" if it is absolutely certain that this is the correct answer" still give this response. Ditto for prompts like "is it possible that your answer was incorrect?"
whodev 4y ago
I like bouncing my code problems off ChatGPT, it can give me an answer and I don't feel bad if I forgot something simple. The issue is I've had it give me completely wrong code only for it to be like "I'm sorry" and provide a second incorrect response.

throwaway4aday 4y ago

Using a purpose built (or trained I guess) model for code generation would likely have better results. GitHub copilot is useful for this reason. I find ChatGPT for code is mainly useful if you want to instruct it in natural language to make subsequent changes to the output.

themk 4y ago

If you follow up about the nonexistent function, it will often implement it for you. The other thing that I've had success with is asking for references for the information, which will often link you to the relevant docs.

rmbyrro 4y ago

If you ask, there's a good chance ChatGPT can create that function for you. Just tell it: "That function `xyz()` doesn't exist in the library, can you write it for me?"