5 ms·

Actually I find it much better than that for exploratory purposes than just getting search terms. The ability to just keep asking questions for clarification i

Actually I find it much better than that for exploratory purposes than just getting search terms. The ability to just keep asking questions for clarification is something that the web was meant to provide with web links, but rarely does a good job of it. If it can simply act as a domain expert that I can talk to, it would be a huge win.

lolinder 4y ago
But it's not a domain expert: it's a language model designed and trained to produce language that could plausibly have been written by a human on the internet. At best it functions as a well-informed amateur, at worst it hallucinates nonsense but writes it in a way that is very convincing.
- hombre_fatal 4y ago
  To be fair, you just summarized human discourse especially on places like Reddit and HN.
  - deleted 4y ago
    [deleted]
  - bamboozled 4y ago
    This comment isn’t helpful or a retort sorry.
  - Xelynega 4y ago
    This makes me think of a quote from one of Dijkstra's lectures: "In the long run I expect computing science to transcend its parent disciplines, mathematics and logic, by effectively realizing a significant part of Leibniz's Dream of providing symbolic calculation as an alternative to human reasoning. (Please note the difference between "mimicking" and "providing an alternative to": alternatives are allowed to be better.)" When talking about a tool thats supposedly greater than humans, why should the shortcomings of humans be relevant? The tools we create to surpass our own capabilities should be greater than our own capabilities, not stunted by the same issues.
- kenjackson 4y ago
  It may not be a domain expert now, but it easily could be. For example if you took all the Linux kernel code, the code review comments, the docs, and several of the top books and blogs on kernel development — suddenly you have a system that may be great for new kernel developers to ask questions of. Especially in a community that often isn’t kind to people Jose asking “dumb questions”.
  - bcrosby95 4y ago
    Could it? There's lots of commentary about how it could easily be this or that, but I don't work in ML and have no clue whether it is actually easy or not to tweak ChatGPT to work in such ways. For example, in my experience ChatGPT isn't very "smart". It has a lot of knowledge but it can't infer any facts from that knowledge. When you ask it to write a program it has no real idea of what it actually does, and you can easily get it to add features it already added, or tell it something is a bug when it really isn't. This doesn't sound like the stuff you could make a domain expert out of, at least, not out of the box.
    - kenjackson 4y ago
      I'm not asking it to infer much, but to piece together a fair bit and understnad what I'm asking. For example, here's a chat I had with ChatGPT: "How does common subexpression elimination work?" It answered it correctly and gave some basic code examples to demonstrate the concept. Then I followed up with: "But can it do the elimination if the variables are flipped, but semantically equivalent, like 'y + x' in the example above?" It again gave what I would consider a correct answer. Note 'x + y' was what was eliminated previously, so I reversed it here. Then I asked: "Are there cases where the compiler might fail to eliminate common subexpressions?" And again a good answer. Now all of this could be found on the web somewhere, but for example the second question didn't show an obvious answer on Google when I searched for it. I'm sure I could find it, but I know this field well. If I was someone new to the field I'd probably spend a lot of time parsing useless articles to find something that answered the question in a way I could understand. I'm less concerned about it writing code (which is cool). For me, the ability to help me learn an area quickly is far more useful. It doesn't need novel answers, but the ability to understand what I'm asking and answer it. I think it's really close to being able to do this now.
- sebzim4500 4y ago
  Being a well-informed amateur in everything is pretty impressive though. ChatGPT will be extremely useful if it ever figures out how to say "I don't know".
  - lolinder 4y ago
    I don't believe there's any way for a LLM operating alone to recognize when it doesn't know something, because it has no concept of knowing something. Its one job is to predict the next word in a body of text. That's it. It's good enough at it that for at least half the people here it passes the Turing test with flying colors, but the only kind of confidence level it has is confidence in its prediction of the next word, not confidence in the information conveyed. If we were to take a language model and hook it up to another system that does have a concept of "knowing" something, I could see us getting somewhere useful—essentially a friendly-to-use search engine over an otherwise normal database.
    - kenjackson 4y ago
      I don't know if I buy this. It feels like your confidence in what you say is closely tied to "knowing". I'm sure there is more research to do here, but I'm not sure if there is a need to "tie" it to some other system. As it stands today there are definitely things ChatGPT doesn't know and will tell you so. For example, I asked it, why did Donald Trump spank his kids -- and it said, "I do not have information about the parenting practices of Donald Trump". That said, there are a lot of things it does get wrong, it would be nice for it be better at those. But I do think that, maybe much like humans, there will always be statements it makes, which are not true.
    - astrange 4y ago
      > Its one job is to predict the next word in a body of text. “Predicting the next word” and “writing” are the same thing; you’re just saying it writes answers in text. There’s nothing about that preventing it from reasoning, and its training goal was more than just “predict the next word” anyway.
  - collinmanderson 4y ago
- LouisSayers 4y ago
  It doesn't matter, it can still be extremely helpful. For instance, I had fragmented memories of a movie, described what I knew about it (about a boy who lived in a trainstation), and it helped me find a couple movies and then narrow in on the one I was looking for. These types of queries can be super painful with modern search engines but was easy with ChatGPT and a pleasant experience. I think people are thinking of this AI in the wrong way - where it is "an expert". To me, I like to think of it as a companion that helps us shape and refine our thoughts and ideas.
suby 4y ago
This is a good point. Something I also enjoy about it is the uniform interface. Each answer is presented the same, there's no parsing layout from different sites, or popup modals to dismiss, or long winded intro to get to the answer you're looking for. Of course you can't quite trust what you're told, so this is a bit moot.

lolinder 4y ago

But it's not a domain expert: it's a language model designed and trained to produce language that could plausibly have been written by a human on the internet. At best it functions as a well-informed amateur, at worst it hallucinates nonsense but writes it in a way that is very convincing.

hombre_fatal 4y ago
To be fair, you just summarized human discourse especially on places like Reddit and HN.
- deleted 4y ago
  [deleted]
- bamboozled 4y ago
  This comment isn’t helpful or a retort sorry.
- Xelynega 4y ago
  This makes me think of a quote from one of Dijkstra's lectures: "In the long run I expect computing science to transcend its parent disciplines, mathematics and logic, by effectively realizing a significant part of Leibniz's Dream of providing symbolic calculation as an alternative to human reasoning. (Please note the difference between "mimicking" and "providing an alternative to": alternatives are allowed to be better.)" When talking about a tool thats supposedly greater than humans, why should the shortcomings of humans be relevant? The tools we create to surpass our own capabilities should be greater than our own capabilities, not stunted by the same issues.
kenjackson 4y ago
It may not be a domain expert now, but it easily could be. For example if you took all the Linux kernel code, the code review comments, the docs, and several of the top books and blogs on kernel development — suddenly you have a system that may be great for new kernel developers to ask questions of. Especially in a community that often isn’t kind to people Jose asking “dumb questions”.
- bcrosby95 4y ago
  Could it? There's lots of commentary about how it could easily be this or that, but I don't work in ML and have no clue whether it is actually easy or not to tweak ChatGPT to work in such ways. For example, in my experience ChatGPT isn't very "smart". It has a lot of knowledge but it can't infer any facts from that knowledge. When you ask it to write a program it has no real idea of what it actually does, and you can easily get it to add features it already added, or tell it something is a bug when it really isn't. This doesn't sound like the stuff you could make a domain expert out of, at least, not out of the box.
  - kenjackson 4y ago
    I'm not asking it to infer much, but to piece together a fair bit and understnad what I'm asking. For example, here's a chat I had with ChatGPT: "How does common subexpression elimination work?" It answered it correctly and gave some basic code examples to demonstrate the concept. Then I followed up with: "But can it do the elimination if the variables are flipped, but semantically equivalent, like 'y + x' in the example above?" It again gave what I would consider a correct answer. Note 'x + y' was what was eliminated previously, so I reversed it here. Then I asked: "Are there cases where the compiler might fail to eliminate common subexpressions?" And again a good answer. Now all of this could be found on the web somewhere, but for example the second question didn't show an obvious answer on Google when I searched for it. I'm sure I could find it, but I know this field well. If I was someone new to the field I'd probably spend a lot of time parsing useless articles to find something that answered the question in a way I could understand. I'm less concerned about it writing code (which is cool). For me, the ability to help me learn an area quickly is far more useful. It doesn't need novel answers, but the ability to understand what I'm asking and answer it. I think it's really close to being able to do this now.
sebzim4500 4y ago
Being a well-informed amateur in everything is pretty impressive though. ChatGPT will be extremely useful if it ever figures out how to say "I don't know".
- lolinder 4y ago
  I don't believe there's any way for a LLM operating alone to recognize when it doesn't know something, because it has no concept of knowing something. Its one job is to predict the next word in a body of text. That's it. It's good enough at it that for at least half the people here it passes the Turing test with flying colors, but the only kind of confidence level it has is confidence in its prediction of the next word, not confidence in the information conveyed. If we were to take a language model and hook it up to another system that does have a concept of "knowing" something, I could see us getting somewhere useful—essentially a friendly-to-use search engine over an otherwise normal database.
  - kenjackson 4y ago
    I don't know if I buy this. It feels like your confidence in what you say is closely tied to "knowing". I'm sure there is more research to do here, but I'm not sure if there is a need to "tie" it to some other system. As it stands today there are definitely things ChatGPT doesn't know and will tell you so. For example, I asked it, why did Donald Trump spank his kids -- and it said, "I do not have information about the parenting practices of Donald Trump". That said, there are a lot of things it does get wrong, it would be nice for it be better at those. But I do think that, maybe much like humans, there will always be statements it makes, which are not true.
  - astrange 4y ago
    > Its one job is to predict the next word in a body of text. “Predicting the next word” and “writing” are the same thing; you’re just saying it writes answers in text. There’s nothing about that preventing it from reasoning, and its training goal was more than just “predict the next word” anyway.
- collinmanderson 4y ago
LouisSayers 4y ago
It doesn't matter, it can still be extremely helpful. For instance, I had fragmented memories of a movie, described what I knew about it (about a boy who lived in a trainstation), and it helped me find a couple movies and then narrow in on the one I was looking for. These types of queries can be super painful with modern search engines but was easy with ChatGPT and a pleasant experience. I think people are thinking of this AI in the wrong way - where it is "an expert". To me, I like to think of it as a companion that helps us shape and refine our thoughts and ideas.

suby 4y ago

This is a good point. Something I also enjoy about it is the uniform interface. Each answer is presented the same, there's no parsing layout from different sites, or popup modals to dismiss, or long winded intro to get to the answer you're looking for. Of course you can't quite trust what you're told, so this is a bit moot.