3 ms·

> I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science ficti

> I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science fiction. If they told you that they made a perfect natural language parser, parsed the whole internet into logic and used that parsed data to make a giant text markov chain, you couldn't imagine that producing results similar to what we see today? The only new thing is that now we can work with natural language as it was code, and that lets us do a lot of cool things, but other than that there is nothing new. If humans structured their natural languages like we structure code then we would get all these abilities for free if we parsed the entire internet, but since we have the messy natural language we had to come up with the super complex solution we got today. You wouldn't need "programmers", since our natural language would be structured enough for computers to execute etc, and that would let us to structure up all posted human reasoning online and make all sorts of cool models based on that. Whenever you call ChatGPT see it as calling the logic in some internet web page, it could be a unix tutorial or a book etc, and then it runs random continuations of that. It isn't hard to imagine what happens when you do that. Is a perfect natural language parser cool? Yes. Was it unthinkable? Not really, many would say it wouldn't be possible, but if we said that we made it possible then figuring out the effects aren't super hard.

mannykannot 4y ago
I don't work in the field, so my responses are naive, but for what it is worth: > If they told you that they made a perfect natural language parser... I would not have guessed that the methods used would yield that, or even 'just' a natural-language parser that performs as well as humans. > ...parsed the whole internet into logic... I am not sure what this means, but parsing seems to yield facts about specific sentences, and that alone seems insufficient to enable part three... > ...and used that parsed data to make a giant text Markov chain. Clearly, that a process is or can be described / explained / analyzed as a Markov chain is not sufficient for it to produce grammatically-correct output that is at least locally sensible to a human reader, so my surprise was that the methods used constitute a process that does. > ...but if we said that we made it possible then figuring out the effects aren't super hard. Hindsight is a really powerful tool.
ogogmad 4y ago
The problem with what you're saying is: There's a lot of data on the Internet, but it's still finite. If you divide a finite number - however big - by infinity, you still get zero. To get something like GPT, you need to generalise to infinitely many possibilities using only a finite amount of data. Markov Chains are too simple an algorithm to produce these results. I think your comment shows what's called the "AI Effect", where a Black Swan event in AI research - like AlphaZero or GPT - is dismissed as predictable purely based on hindsight - even though the results surprised most experts.

mannykannot 4y ago

I don't work in the field, so my responses are naive, but for what it is worth: > If they told you that they made a perfect natural language parser... I would not have guessed that the methods used would yield that, or even 'just' a natural-language parser that performs as well as humans. > ...parsed the whole internet into logic... I am not sure what this means, but parsing seems to yield facts about specific sentences, and that alone seems insufficient to enable part three... > ...and used that parsed data to make a giant text Markov chain. Clearly, that a process is or can be described / explained / analyzed as a Markov chain is not sufficient for it to produce grammatically-correct output that is at least locally sensible to a human reader, so my surprise was that the methods used constitute a process that does. > ...but if we said that we made it possible then figuring out the effects aren't super hard. Hindsight is a really powerful tool.

ogogmad 4y ago

The problem with what you're saying is: There's a lot of data on the Internet, but it's still finite. If you divide a finite number - however big - by infinity, you still get zero. To get something like GPT, you need to generalise to infinitely many possibilities using only a finite amount of data. Markov Chains are too simple an algorithm to produce these results. I think your comment shows what's called the "AI Effect", where a Black Swan event in AI research - like AlphaZero or GPT - is dismissed as predictable purely based on hindsight - even though the results surprised most experts.