3 ms·

We've had regular GPT for several years now, and nothing has happened. People predicted this with the OG Transformer model from OpenAI, and yet the fake news ap

We've had regular GPT for several years now, and nothing has happened. People predicted this with the OG Transformer model from OpenAI, and yet the fake news apocalypse didn't come upon us. > Won't it result in 1000s of blogs mushrooming over night from purported bots-come-experts with some kind of engineered slant? I don't say this to be rude, I really don't, but have you Googled something like "reverse a string python" at any point in the past 5 years? > Whole "online communities" for the unsuspecting to join initially made up of nothing but bots "talking" to each other, This has also existed for a long time, see SubredditSimulator etc. There are also sites that scrape forums and replace the usernames to do ad fraud.

tootie 4y ago
You could carry on a decent conversation with ELIZA in the 90s. ChatGPT is arguably less sophisticated since it's just using NN tricks rather than trying to emulate intelligence. I could be wrong but I think ChatGPT is just about the ceiling for what this technology can do. I don't mean to be a grump because it's incredibly impressive for what it is but I still think we're pretty far from a tipping point.
- heartbeats 4y ago
  I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the right comparison, you should compare it to something like SQL or Google. Against that comparison, it stands out as very innovative, but it also seems like it has big potential to improve. If you could perfectly index a large database and carry out reasoning tasks, this is enough to replace a lot of jobs. ChatGPT is in this regard just on the boundary between a toy example and useful in prod. It's also worth noting that, up until now, people who have been saying this ("I think this is about where it peaks, it'll never do X") have always been proven wrong by the passage of time. So I think we should assign lower credibility to the skeptics, insofar as their past predictions were incorrect.
  - Ajedi32 4y ago
    What do you mean by "the way it's able to access data"? AFAIK ChatGPT is just a fine tuned version of GPT-3.5 optimized for chat[1], it isn't any different from GPT-2 in the way it accesses data. [1]: https://openai.com/blog/chatgpt/#methods https://openai.com/blog/chatgpt/#methods
    - heartbeats 4y ago
      That's true, but since the model is bigger than GPT-2, I can basically use it as a query language for a database. If the model is smaller, I still do that, sure, but I can only ask "basic" questions about the English language and so on. But if it has more neurons, it's capable to "index" a lot of questions about e.g. proper Python usage.
  - tootie 4y ago
    It's definitely made several huge leaps in ability. Not least of which is just compiling and processing the enormous corpus of training data. It's also definitely a major leap in NLP. Not just in terms of tech, but also UX in that it can funnel users into the relatively narrow set of prompts that it can understand and respond to perfectly. By comparison, Alexa is ten years old and barely understands half the things I ask and frequently goes way off topic in it's responses. Admittedly a very different product but in the same space.

tootie 4y ago

You could carry on a decent conversation with ELIZA in the 90s. ChatGPT is arguably less sophisticated since it's just using NN tricks rather than trying to emulate intelligence. I could be wrong but I think ChatGPT is just about the ceiling for what this technology can do. I don't mean to be a grump because it's incredibly impressive for what it is but I still think we're pretty far from a tipping point.

heartbeats 4y ago
I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the right comparison, you should compare it to something like SQL or Google. Against that comparison, it stands out as very innovative, but it also seems like it has big potential to improve. If you could perfectly index a large database and carry out reasoning tasks, this is enough to replace a lot of jobs. ChatGPT is in this regard just on the boundary between a toy example and useful in prod. It's also worth noting that, up until now, people who have been saying this ("I think this is about where it peaks, it'll never do X") have always been proven wrong by the passage of time. So I think we should assign lower credibility to the skeptics, insofar as their past predictions were incorrect.
- Ajedi32 4y ago
  What do you mean by "the way it's able to access data"? AFAIK ChatGPT is just a fine tuned version of GPT-3.5 optimized for chat[1], it isn't any different from GPT-2 in the way it accesses data. [1]: https://openai.com/blog/chatgpt/#methods https://openai.com/blog/chatgpt/#methods
  - heartbeats 4y ago
    That's true, but since the model is bigger than GPT-2, I can basically use it as a query language for a database. If the model is smaller, I still do that, sure, but I can only ask "basic" questions about the English language and so on. But if it has more neurons, it's capable to "index" a lot of questions about e.g. proper Python usage.
- tootie 4y ago
  It's definitely made several huge leaps in ability. Not least of which is just compiling and processing the enormous corpus of training data. It's also definitely a major leap in NLP. Not just in terms of tech, but also UX in that it can funnel users into the relatively narrow set of prompts that it can understand and respond to perfectly. By comparison, Alexa is ten years old and barely understands half the things I ask and frequently goes way off topic in it's responses. Admittedly a very different product but in the same space.