2 ms·

> they're great for writing functions and explaining hard to understand errors Are they? I'd be interested to hear your experience on this. So far for me they

> they're great for writing functions and explaining hard to understand errors Are they? I'd be interested to hear your experience on this. So far for me they have only really been able to summarise what I could find from the top few results searching online. They do a good job of summarising that, and might be quicker, but that's been it. However, when I encounter an actually tricky issue, like a threading bug or a null pointer exception/type error sort of generic issue that's 5 levels removed from its source, these tools never manage it. Despite prompting saying I don't need a NullPointerException explained to me, figure out how this is null, the results are poor. This might be my biases speaking, but it really does feel like I'm speaking to something that's good at transforming words and paragraphs into different formats but which has no actual understanding of the code.

floodle 3y ago
> So far for me they have only really been able to summarise what I could find from the top few results searching online This has been my experience, except that the chat interface gets me to exactly the answer to my question considerably faster than a search engine. I see these tools as search engines with much better user interfaces and customised responses.
- danpalmer 3y ago
  Yeah, I guess that there's just not a lot of value for me in "what does this error message mean" because once you've worked with a language or framework for any reasonable amount of time you learn them and they sort of disappear into the background. LLMs do seem good for learning new systems though.
WhiteNoiz3 3y ago
I've had success with asking "what are some possible bugs with this block of code".. Sometimes it spots errors that I wouldn't have thought of or at least gives some ideas for things to check. Fixing tricky bugs often requires collecting additional information - stepping through code, looking at values of variables and making sure they are what you expect, etc. It's an iterative process and AI tools would need to be able to do the same thing - most humans wouldn't be able to solve errors like that just by looking at the code, and neither would an LLM. I see the same issues with people who think AI is going to make scientific discoveries - it can't do that because making discoveries requires collecting data until something is certain or we have a clear picture. At that point, you don't need AI. AI won't be making discoveries until we can automate that entire process of forming a hypothesis, testing it / doing experiments, collecting data, refining your hypothesis, etc.

floodle 3y ago

> So far for me they have only really been able to summarise what I could find from the top few results searching online This has been my experience, except that the chat interface gets me to exactly the answer to my question considerably faster than a search engine. I see these tools as search engines with much better user interfaces and customised responses.

danpalmer 3y ago
Yeah, I guess that there's just not a lot of value for me in "what does this error message mean" because once you've worked with a language or framework for any reasonable amount of time you learn them and they sort of disappear into the background. LLMs do seem good for learning new systems though.

WhiteNoiz3 3y ago

I've had success with asking "what are some possible bugs with this block of code".. Sometimes it spots errors that I wouldn't have thought of or at least gives some ideas for things to check. Fixing tricky bugs often requires collecting additional information - stepping through code, looking at values of variables and making sure they are what you expect, etc. It's an iterative process and AI tools would need to be able to do the same thing - most humans wouldn't be able to solve errors like that just by looking at the code, and neither would an LLM. I see the same issues with people who think AI is going to make scientific discoveries - it can't do that because making discoveries requires collecting data until something is certain or we have a clear picture. At that point, you don't need AI. AI won't be making discoveries until we can automate that entire process of forming a hypothesis, testing it / doing experiments, collecting data, refining your hypothesis, etc.