6 ms·

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprink

I asked ChatGPT how to add a JSDoc type to a Vue 2 prop and it gave me a wrong answer. There have been several times where I’ve asked it questions and it sprinkles in well disguised misinformation. These tools are impressive but they definitely have limitations.

golol 3y ago
ChatGPT (3.5) is not state of the art
seanthemon 3y ago
Curious, how does it do when you give it examples / use playground.openai? I find that no examples often leads to the same result as a befuddled junior, but with examples often it gains confidence. Also I find playground to give me much better code snippets than chat sometimes
- cbozeman 3y ago
  You've identified the problem inherent in these models right now, though. There's a lot of "extraneous" information and details that appear useless and unrelated, but that long-time developers have tucked away in their brains (or really anyone who has done something at a high level for a long time), that turns out to be incredibly useful; generally these people don't require examples - they just know what the right answer is, because they've been exposed to a variety of problems over a long career or lifespan. That's where LLMs need to be to be truly "useful". I think if we get them to that point, we'll really have something useful on our hands.
  - dontupvoteme 3y ago
    That's where good money and societal upheaval will come up, private models for corporations trained on their entire codebase and documentation. Both young and old shall feel the pain then.
    - HPsquared 3y ago
      The codebase and documentation usually don't capture the full context of the development process. There is still a lot that isn't written down.
      - dontupvoteme 3y ago
        I agree, but the MBAs will probably not.
  - seanthemon 3y ago
    Very insightful point - do you think that OpenAI and Microsoft are aiming at developers so that we may spill this knowledge by N-shotting the chat query? I.e if the AI doesn't understand some fundamental knowledge, often i'll teach it so that it can be more effective at helping me, inadvertently I spill some of this tucked knowledge.
    - cbozeman 3y ago
      I don't think they have any desire to use this knowledge for their own gain, since they explicitly say that your data will not be used to train their future models - both OpenAI is saying that about their trainable ChatGPT 3.5 Turbo model, and Azure OpenAI Service was clear on that as well. I think the future is extremely well-trained base models that are then fine-tuned on specific domain knowledge. I'm already seeing that with Meta's Llama 2 personally at home. My company has access to Azure OpenAI Service's GPT-4 trainable model that we've been working with on all our documentation, and the results look very promising so far. But, this idea of the All-Knowing Oracle that can answer any question you have is all well and good, but it's so far beyond our current processing power as to be a pointless endeavor at the moment. Will we get there eventually? Yeah, sure. According to Jim Keller as of around 2019/2020 when he was speaking on Lex Fridman's podcast, he said we still have room to go 1,000,000 times smaller - meaning chips. I think we probably need two more orders of magnitude in processing power of the strongest high-end GPUs before we're there. NVIDIA's H100 is a great achievement, but we need something about 100x as fast, and I think we'll still need dozens of them working in parallel to train the model that can answer all these questions. All of this though is a moot point, because it's the lawyers that have already slowed down progress. OpenAI is terrified of being sued. Altman couches it behind terms like, "AI Safety", "AI Alignment", etc., but it's fear. It's all stemming from fear. And it's all stemming from people just not "getting it". We're entering a new age of upheaval, and there's going to be rogue AIs that tell people to go kill themselves to reduce climate change. You know why? Because there are humans that tell people to go kill themselves to reduce climate change. These models are language models. We taught them how to think, and we taught them how to think like us, so it's no surprise to me whatsoever that they behave like us - meaning they occasionally lie and they occasionally go off the rails and go a little crazy. We have become gods and we have made a creation in our own image. Most of the time it's awesome, sometimes it's a little wild and wacky.
jasfi 3y ago
Are you using GPT-4? If not, it's understandable. If you don't pay for ChatGPT, you get GPT-3.5. You can also get access to GPT-4 if you use the playground.
- andrepd 3y ago
  Why is it always the same reply? Yes, GPT4 is as useless as GPT3.5 on any non trivial task.
  - input_sh 3y ago
    My guess is people feel the need to self-justify their $20/month subscription.
    - aenis 3y ago
      My guess is most people into AI don't even remember that they are paying a $20/month for this. We do a lot of experiments involving gpt3.5, 4, claude-v2, titan-large, and palm2, and for what it's worth, on our real production workloads gpt4 shines. We can make Palm2 produce decent results with a lot of extra effort, and claude-v2 is passable but gpt4 does not disappoint. This is low-grade knowledge management stuff, and we are not using it as a information-retrieval system - but for basic 'cognitive' tasks where all the information needed is provided in the prompt. I'd not rely on it for info retrieval tasks such as the examples quoted above - its knowledge base is highly compressed, after all.
  - Kiro 3y ago
    > Why is it always the same reply? When there's one definitive answer to something that people keep repeating there's a slight chance that it's actually true. Shocking, I know.
    - andrepd 3y ago
      Well but it's not. Also, your reasoning would imply e.g. Qanon is true based on the number of people repeating the same thing, so that's pretty weak.
jstummbillig 3y ago
Everything has limitations. The linked piece is very much a statement about limited potential, not the current state. In this context, this stance is akin to saying "I asked a 6yo this math question and it gave me a wrong answer, there's definitely limitations". Sure – but who would use that observation to assess where the future potential cap of the kid is?
ryoshu 3y ago
True. I can't do a full table scan on an non-indexed column in a RDBMS with billions of rows on a single machine without it grinding to a halt. Tools have limitations.
- dclowd9901 3y ago
  Disingenuous: given the proper processing, you’d get a 100% deterministic and reliable answer. How do you get that out of an LLM? What tool is any good if it doesn’t work 100% of the time predictably?
  - dontupvoteme 3y ago
    Right now one generation is expensive on these big mainframe models. Query many times on the right model(s) for the question and the correct answer will be there 99.999% of the time as the other hallucinations will be thrown out.
  - saberience 3y ago
    How many tools do you know of that work 100% of the time. As someone that’s worked in software 20 years now, I don’t think there is anything from Linux to eMacs to visual studio to Java which doesn’t have bugs, issues, crashes, shitty stuff, and yet we expect a revolutionary and amazing technology to generate code perfectly? It’s funny those the LLM haters keep raising the bar to a level that no other software can reach. Chatgpt is a tool like any other and just like a hammer, it can be misused, or it can be incredibly useful if used well. I personally find chatgpt mind boggling and astounding and use it every day multiple times for both coding and non coding purposes. But it’s totally normal and reasonable to me to expect bugs, do you really expect visual studio to run on large solutions and never have crashes or memory issues or slowness? If so you’re going to be disappointed.
    - ted_bunny 3y ago
      Poor is the carpenter!
dontupvoteme 3y ago
You're asking a model which is just a big unfocused and highly alignment taxed mush of the entire internet across all language, regardless of if its natural or computer, then (maybe) sliced across 8 experts. It can also simulate a Zizek vs Wittgenstein argument over Russian literature. the fact that it can usually write executing computer code is nothing short of magic to me. What one would want is something like LLaMA2-Coding-Vue2, maybe with a LoRA for the library or concept. I wrote a big word salad about all the other things you could do but suffice it to say there's no reason anyone should be limited to a single pass on a monolithic without automatic context window augmentation nor automatic code checking & regeneration & model escalation (e.g. query out to a 200B coding model or something).
- gorbypark 3y ago
  I think this concept will be the future of coding. You'll have something like CodeLLaMA-javascript trained specifically for the language, then you'll have LoRA-like files distributed with/for each library. I can image something like the definitely-typed project for typescript types, but instead of doing `yarn add @types/someLibrary` you'll be able to do `yarn add @loramodels/someLibrary`. I'm javascript centric these days, but the same concept should work for most languages with a package manager. Some of these features could even be integrated into say yarn/npm directly, you currently have a devDependencies section of a package.json..I could imaging having something along the lines of a "llmDependencies" section to define which main model and version to use and which "library" models to use.
GolDDranks 3y ago
I don't encounter a lot of "small", one-copy-paste-size problems in my daily work that I couldn't quickly solve myself, so I haven't found a lot of use for ChatGPT while coding, yet. (I reckon this is changing though.) However a few times there have been some mechanical refactoring-style grunt work I've delighted to have been able to let ChatGPT do. However, the rate ChatGPT is giving me subtly wrong results is just high enough that I end up cross-checking everything, and then it takes me a bit more time than it would've otherwise taken. Give it a year or two, maybe?
- layer8 3y ago
  > Give it a year or two, maybe? Maybe? It’s not clear what would bring a qualitative improvement, barring massive amounts of new training data.
  - GolDDranks 3y ago
    Based on the current front page of HackerNews, it's clear that quantitative improvement is happening all the time. I think that enough incremental progress will eventual feel like qualitative improvement. And it doesn't even need to be smarter models; improving prompts, IO systems etc. around the model might help. For example, I said I don't feel the need for help for copy-paste-sized problems, but inputing the whole repo seems to be achievable with some clever scaffolding with the IO around the model.

golol 3y ago

ChatGPT (3.5) is not state of the art

seanthemon 3y ago

Curious, how does it do when you give it examples / use playground.openai? I find that no examples often leads to the same result as a befuddled junior, but with examples often it gains confidence. Also I find playground to give me much better code snippets than chat sometimes

cbozeman 3y ago
You've identified the problem inherent in these models right now, though. There's a lot of "extraneous" information and details that appear useless and unrelated, but that long-time developers have tucked away in their brains (or really anyone who has done something at a high level for a long time), that turns out to be incredibly useful; generally these people don't require examples - they just know what the right answer is, because they've been exposed to a variety of problems over a long career or lifespan. That's where LLMs need to be to be truly "useful". I think if we get them to that point, we'll really have something useful on our hands.
- dontupvoteme 3y ago
  That's where good money and societal upheaval will come up, private models for corporations trained on their entire codebase and documentation. Both young and old shall feel the pain then.
  - HPsquared 3y ago
    The codebase and documentation usually don't capture the full context of the development process. There is still a lot that isn't written down.
    - dontupvoteme 3y ago
      I agree, but the MBAs will probably not.
- seanthemon 3y ago
  Very insightful point - do you think that OpenAI and Microsoft are aiming at developers so that we may spill this knowledge by N-shotting the chat query? I.e if the AI doesn't understand some fundamental knowledge, often i'll teach it so that it can be more effective at helping me, inadvertently I spill some of this tucked knowledge.
  - cbozeman 3y ago
    I don't think they have any desire to use this knowledge for their own gain, since they explicitly say that your data will not be used to train their future models - both OpenAI is saying that about their trainable ChatGPT 3.5 Turbo model, and Azure OpenAI Service was clear on that as well. I think the future is extremely well-trained base models that are then fine-tuned on specific domain knowledge. I'm already seeing that with Meta's Llama 2 personally at home. My company has access to Azure OpenAI Service's GPT-4 trainable model that we've been working with on all our documentation, and the results look very promising so far. But, this idea of the All-Knowing Oracle that can answer any question you have is all well and good, but it's so far beyond our current processing power as to be a pointless endeavor at the moment. Will we get there eventually? Yeah, sure. According to Jim Keller as of around 2019/2020 when he was speaking on Lex Fridman's podcast, he said we still have room to go 1,000,000 times smaller - meaning chips. I think we probably need two more orders of magnitude in processing power of the strongest high-end GPUs before we're there. NVIDIA's H100 is a great achievement, but we need something about 100x as fast, and I think we'll still need dozens of them working in parallel to train the model that can answer all these questions. All of this though is a moot point, because it's the lawyers that have already slowed down progress. OpenAI is terrified of being sued. Altman couches it behind terms like, "AI Safety", "AI Alignment", etc., but it's fear. It's all stemming from fear. And it's all stemming from people just not "getting it". We're entering a new age of upheaval, and there's going to be rogue AIs that tell people to go kill themselves to reduce climate change. You know why? Because there are humans that tell people to go kill themselves to reduce climate change. These models are language models. We taught them how to think, and we taught them how to think like us, so it's no surprise to me whatsoever that they behave like us - meaning they occasionally lie and they occasionally go off the rails and go a little crazy. We have become gods and we have made a creation in our own image. Most of the time it's awesome, sometimes it's a little wild and wacky.

jasfi 3y ago

Are you using GPT-4? If not, it's understandable. If you don't pay for ChatGPT, you get GPT-3.5. You can also get access to GPT-4 if you use the playground.

andrepd 3y ago
Why is it always the same reply? Yes, GPT4 is as useless as GPT3.5 on any non trivial task.
- input_sh 3y ago
  My guess is people feel the need to self-justify their $20/month subscription.
  - aenis 3y ago
    My guess is most people into AI don't even remember that they are paying a $20/month for this. We do a lot of experiments involving gpt3.5, 4, claude-v2, titan-large, and palm2, and for what it's worth, on our real production workloads gpt4 shines. We can make Palm2 produce decent results with a lot of extra effort, and claude-v2 is passable but gpt4 does not disappoint. This is low-grade knowledge management stuff, and we are not using it as a information-retrieval system - but for basic 'cognitive' tasks where all the information needed is provided in the prompt. I'd not rely on it for info retrieval tasks such as the examples quoted above - its knowledge base is highly compressed, after all.
- Kiro 3y ago
  > Why is it always the same reply? When there's one definitive answer to something that people keep repeating there's a slight chance that it's actually true. Shocking, I know.
  - andrepd 3y ago
    Well but it's not. Also, your reasoning would imply e.g. Qanon is true based on the number of people repeating the same thing, so that's pretty weak.

jstummbillig 3y ago

Everything has limitations. The linked piece is very much a statement about limited potential, not the current state. In this context, this stance is akin to saying "I asked a 6yo this math question and it gave me a wrong answer, there's definitely limitations". Sure – but who would use that observation to assess where the future potential cap of the kid is?

ryoshu 3y ago

True. I can't do a full table scan on an non-indexed column in a RDBMS with billions of rows on a single machine without it grinding to a halt. Tools have limitations.

dclowd9901 3y ago
Disingenuous: given the proper processing, you’d get a 100% deterministic and reliable answer. How do you get that out of an LLM? What tool is any good if it doesn’t work 100% of the time predictably?
- dontupvoteme 3y ago
  Right now one generation is expensive on these big mainframe models. Query many times on the right model(s) for the question and the correct answer will be there 99.999% of the time as the other hallucinations will be thrown out.
- saberience 3y ago
  How many tools do you know of that work 100% of the time. As someone that’s worked in software 20 years now, I don’t think there is anything from Linux to eMacs to visual studio to Java which doesn’t have bugs, issues, crashes, shitty stuff, and yet we expect a revolutionary and amazing technology to generate code perfectly? It’s funny those the LLM haters keep raising the bar to a level that no other software can reach. Chatgpt is a tool like any other and just like a hammer, it can be misused, or it can be incredibly useful if used well. I personally find chatgpt mind boggling and astounding and use it every day multiple times for both coding and non coding purposes. But it’s totally normal and reasonable to me to expect bugs, do you really expect visual studio to run on large solutions and never have crashes or memory issues or slowness? If so you’re going to be disappointed.
  - ted_bunny 3y ago
    Poor is the carpenter!

dontupvoteme 3y ago

You're asking a model which is just a big unfocused and highly alignment taxed mush of the entire internet across all language, regardless of if its natural or computer, then (maybe) sliced across 8 experts. It can also simulate a Zizek vs Wittgenstein argument over Russian literature. the fact that it can usually write executing computer code is nothing short of magic to me. What one would want is something like LLaMA2-Coding-Vue2, maybe with a LoRA for the library or concept. I wrote a big word salad about all the other things you could do but suffice it to say there's no reason anyone should be limited to a single pass on a monolithic without automatic context window augmentation nor automatic code checking & regeneration & model escalation (e.g. query out to a 200B coding model or something).

gorbypark 3y ago
I think this concept will be the future of coding. You'll have something like CodeLLaMA-javascript trained specifically for the language, then you'll have LoRA-like files distributed with/for each library. I can image something like the definitely-typed project for typescript types, but instead of doing `yarn add @types/someLibrary` you'll be able to do `yarn add @loramodels/someLibrary`. I'm javascript centric these days, but the same concept should work for most languages with a package manager. Some of these features could even be integrated into say yarn/npm directly, you currently have a devDependencies section of a package.json..I could imaging having something along the lines of a "llmDependencies" section to define which main model and version to use and which "library" models to use.

GolDDranks 3y ago

I don't encounter a lot of "small", one-copy-paste-size problems in my daily work that I couldn't quickly solve myself, so I haven't found a lot of use for ChatGPT while coding, yet. (I reckon this is changing though.) However a few times there have been some mechanical refactoring-style grunt work I've delighted to have been able to let ChatGPT do. However, the rate ChatGPT is giving me subtly wrong results is just high enough that I end up cross-checking everything, and then it takes me a bit more time than it would've otherwise taken. Give it a year or two, maybe?

layer8 3y ago
> Give it a year or two, maybe? Maybe? It’s not clear what would bring a qualitative improvement, barring massive amounts of new training data.
- GolDDranks 3y ago
  Based on the current front page of HackerNews, it's clear that quantitative improvement is happening all the time. I think that enough incremental progress will eventual feel like qualitative improvement. And it doesn't even need to be smarter models; improving prompts, IO systems etc. around the model might help. For example, I said I don't feel the need for help for copy-paste-sized problems, but inputing the whole repo seems to be achievable with some clever scaffolding with the IO around the model.