3 ms·

Aside from dumping a very large amount of compute and human-supervised learning into existing technology, what major progress has been made in the past six mont

Aside from dumping a very large amount of compute and human-supervised learning into existing technology, what major progress has been made in the past six months?

pixl97 4y ago
You mean things like GPT showing theory of mind at the level of a 9 year old, something that many people were arguing may not ever happen and/or if it did we were still a long way away from it just a few years ago?
- rootusrootus 4y ago
  Is that because it actually has a theory of mind, or because it fakes it really well based on human training data, or because the test is flawed?
edanm 4y ago
I think that's an interesting question that you won't be able to get an answer for, because most new things are still not accessible to the average HN reader. I mean, you can look up ML research papers to see what's new, but none of that is going to seem anywhere near as impactful as ChatGPT because it's at the level of research. It's not like when the first theoretical models that make up the basis for GPT came out, people were instantly sure where we'd end up in 5 years. I'm sure people within OpenAI or other AI companies would be able to tell you of very interesting advances, but they won't do so on HN until the things are public.
jstx1 4y ago
You don't need to be dismissive - demonstrating what can be achieved with a lot of data/compute and additional human-supervised learning is progress.
- rootusrootus 4y ago
  > You don't need to be dismissive Read it again, but this time turn off your preconception. I asked a question I wanted to hear a legitimate answer for. I am very interested to hear where this is headed (to the best of our ability to predict). The results we can get with sufficient computing power is quite interesting, but is it reaching the point of rapidly diminishing returns? What's the next step in this evolution, beyond making the training faster or adding more parameters to the model?
  - deleted 4y ago
    [deleted]
LordDragonfang 4y ago
I mean, Stable Diffusion came out mid August (almost exactly 6 months ago) and the past six months have seen tons of refinement by hobbyists, so we went from having what amounted to a tech demo with practically closed access to a really functional diffusion model that you can run on your average gaming machine. I'd say that's kind of major.
devmunchies 4y ago
Easy. We’re getting better at prompt engineering.

Aside from dumping a very large amount of compute and human-supervised learning into existing technology, what major progress has been made in the past six months?

pixl97 4y ago

You mean things like GPT showing theory of mind at the level of a 9 year old, something that many people were arguing may not ever happen and/or if it did we were still a long way away from it just a few years ago?

rootusrootus 4y ago
Is that because it actually has a theory of mind, or because it fakes it really well based on human training data, or because the test is flawed?

edanm 4y ago

I think that's an interesting question that you won't be able to get an answer for, because most new things are still not accessible to the average HN reader. I mean, you can look up ML research papers to see what's new, but none of that is going to seem anywhere near as impactful as ChatGPT because it's at the level of research. It's not like when the first theoretical models that make up the basis for GPT came out, people were instantly sure where we'd end up in 5 years. I'm sure people within OpenAI or other AI companies would be able to tell you of very interesting advances, but they won't do so on HN until the things are public.

jstx1 4y ago

You don't need to be dismissive - demonstrating what can be achieved with a lot of data/compute and additional human-supervised learning is progress.

rootusrootus 4y ago
> You don't need to be dismissive Read it again, but this time turn off your preconception. I asked a question I wanted to hear a legitimate answer for. I am very interested to hear where this is headed (to the best of our ability to predict). The results we can get with sufficient computing power is quite interesting, but is it reaching the point of rapidly diminishing returns? What's the next step in this evolution, beyond making the training faster or adding more parameters to the model?
- deleted 4y ago
  [deleted]

LordDragonfang 4y ago

I mean, Stable Diffusion came out mid August (almost exactly 6 months ago) and the past six months have seen tons of refinement by hobbyists, so we went from having what amounted to a tech demo with practically closed access to a really functional diffusion model that you can run on your average gaming machine. I'd say that's kind of major.

devmunchies 4y ago

Easy. We’re getting better at prompt engineering.