5 ms·

ChatGPT for sure can be creative, but it lacks 1 thing: real life world experience. For example, as a senior software developer, I'm not convinced pure unit te

ChatGPT for sure can be creative, but it lacks 1 thing: real life world experience. For example, as a senior software developer, I'm not convinced pure unit testing is always the way to go. I was thinking about writing my own alternative and how to weigh the benefits against the drawbacks. I was querying ChatGPT about it, but it's just able to restate the current "common knowledge". So yes, ChatGPT can come up with new concepts. It however, cannot find out the problems, issues and unexpected consequences of applying it in the real world. Somehow, these things will have to be fed into it again, in either the form of text, images or video. 99% of current writing can be replaced by ChatGPT. But this 1% of things only people with real world experience can come up with, it cannot do.

edwardso 4y ago
Yeah, with my interaction with ChatGPT it just spits out general concepts that almost make it seem like it exhaustively covers the prompt, but from those concepts sometimes there'll be 1 or 2 that just won't be applicable to the reality of the real world. Large language model lacks the acuteness to anticipate any unintended consequences and that 1% of edge case is where things might truly matter.
pillefitz 4y ago
ChatGPT lists (common) downsides, remedies and situations when to avoid unit testing. Do you have the expected answer in mind, and could chatGPT have possibly deduced this from your prompt?
- koonsolo 4y ago
  My problem and solution is probably too wide to explain here, so let's take an easier example. TDD where you write the test before the code, is a widespread concept now. Imagine a time before this was the case. Would ChatGPT be able to come up with such a concept? Probably. Would it be able to really understand all the consequences of such a setup? I highly doubt it. Even if it did, would we trust it? In my opinion, such things can only be tested in the real world, where you work with real projects, developers and testers, and gather all the impacts. I don't think a LLM could derive such things.
  - throwaway4aday 4y ago
    If you can't explain it in a comment then how did you explain it in the prompt(s)?
golol 4y ago
Once you have an agentic AI with years of experience your claim is disproven. I consider this as conceptually possible given a 10-100x increase in scale.

edwardso 4y ago

Yeah, with my interaction with ChatGPT it just spits out general concepts that almost make it seem like it exhaustively covers the prompt, but from those concepts sometimes there'll be 1 or 2 that just won't be applicable to the reality of the real world. Large language model lacks the acuteness to anticipate any unintended consequences and that 1% of edge case is where things might truly matter.

pillefitz 4y ago

ChatGPT lists (common) downsides, remedies and situations when to avoid unit testing. Do you have the expected answer in mind, and could chatGPT have possibly deduced this from your prompt?

koonsolo 4y ago
My problem and solution is probably too wide to explain here, so let's take an easier example. TDD where you write the test before the code, is a widespread concept now. Imagine a time before this was the case. Would ChatGPT be able to come up with such a concept? Probably. Would it be able to really understand all the consequences of such a setup? I highly doubt it. Even if it did, would we trust it? In my opinion, such things can only be tested in the real world, where you work with real projects, developers and testers, and gather all the impacts. I don't think a LLM could derive such things.
- throwaway4aday 4y ago
  If you can't explain it in a comment then how did you explain it in the prompt(s)?

golol 4y ago

Once you have an agentic AI with years of experience your claim is disproven. I consider this as conceptually possible given a 10-100x increase in scale.