3 ms·

I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the righ

I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the right comparison, you should compare it to something like SQL or Google. Against that comparison, it stands out as very innovative, but it also seems like it has big potential to improve. If you could perfectly index a large database and carry out reasoning tasks, this is enough to replace a lot of jobs. ChatGPT is in this regard just on the boundary between a toy example and useful in prod. It's also worth noting that, up until now, people who have been saying this ("I think this is about where it peaks, it'll never do X") have always been proven wrong by the passage of time. So I think we should assign lower credibility to the skeptics, insofar as their past predictions were incorrect.

Ajedi32 4y ago
What do you mean by "the way it's able to access data"? AFAIK ChatGPT is just a fine tuned version of GPT-3.5 optimized for chat[1], it isn't any different from GPT-2 in the way it accesses data. [1]: https://openai.com/blog/chatgpt/#methods https://openai.com/blog/chatgpt/#methods
- heartbeats 4y ago
  That's true, but since the model is bigger than GPT-2, I can basically use it as a query language for a database. If the model is smaller, I still do that, sure, but I can only ask "basic" questions about the English language and so on. But if it has more neurons, it's capable to "index" a lot of questions about e.g. proper Python usage.
tootie 4y ago
It's definitely made several huge leaps in ability. Not least of which is just compiling and processing the enormous corpus of training data. It's also definitely a major leap in NLP. Not just in terms of tech, but also UX in that it can funnel users into the relatively narrow set of prompts that it can understand and respond to perfectly. By comparison, Alexa is ten years old and barely understands half the things I ask and frequently goes way off topic in it's responses. Admittedly a very different product but in the same space.

Ajedi32 4y ago

What do you mean by "the way it's able to access data"? AFAIK ChatGPT is just a fine tuned version of GPT-3.5 optimized for chat[1], it isn't any different from GPT-2 in the way it accesses data. [1]: https://openai.com/blog/chatgpt/#methods https://openai.com/blog/chatgpt/#methods

heartbeats 4y ago
That's true, but since the model is bigger than GPT-2, I can basically use it as a query language for a database. If the model is smaller, I still do that, sure, but I can only ask "basic" questions about the English language and so on. But if it has more neurons, it's capable to "index" a lot of questions about e.g. proper Python usage.

tootie 4y ago

It's definitely made several huge leaps in ability. Not least of which is just compiling and processing the enormous corpus of training data. It's also definitely a major leap in NLP. Not just in terms of tech, but also UX in that it can funnel users into the relatively narrow set of prompts that it can understand and respond to perfectly. By comparison, Alexa is ten years old and barely understands half the things I ask and frequently goes way off topic in it's responses. Admittedly a very different product but in the same space.