2 ms·

I smile every time I hear this argument. To me it’s obviously false and head in the sand. There was a moment, with GPT-2 where it felt exactly right, then with

I smile every time I hear this argument. To me it’s obviously false and head in the sand. There was a moment, with GPT-2 where it felt exactly right, then with GPT-3 where you could see a glimmer of something emerging. Now GPT-4 has something real going on. A solid example is the tool use. Package up a brand new, never existed before or seen in the training data plug-in, give a few paragraphs of human readable documentation, and suddenly GPT-4 can interact with other systems, databases, and models. It doesn’t have 10,000 examples of interacting with this novel plugin to predict next word off of, it just has a short English description, enough for a human to be able to use and understand the same plugin effectively. There’s a reason this is knocking the socks off many smart folks. It’s upending a lot of the sacred cows of human intelligence.

JieJie 4y ago
There seems to be a huge lag in news these days. I was reminded recently that GPT-4 was only released three weeks ago even fewer people have full access to the plugins that the author is describing here. Most people are still talking about an article they read in the New York Times about GPT-3.5 Turbo. Not that 3.5 Turbo didn't exhibit some reasoning abilities, but they were more difficult to suss out. GPT-4 can still seem pretty vacant sometimes and finding "magic" in it has a lot to do with building up a "state" where the model is generating reason. It does not happen by default. This will all be different in another month or two, as more people have access to the latest models and new models get released (assuming there is no agreement on a pause). Then there will be another lag as laypeople catch up. There are scenarios that may play out where this speeds up even further and whole generations of AI completely fly under the radar of public acknowledgement. Or this all collapses in a debris pile of hype. I feel like I am just along for the ride at this point, trying to keep my head above water. I reckon this is going to cause a tremendous amount of stress for a tremendous number of people.

JieJie 4y ago

There seems to be a huge lag in news these days. I was reminded recently that GPT-4 was only released three weeks ago even fewer people have full access to the plugins that the author is describing here. Most people are still talking about an article they read in the New York Times about GPT-3.5 Turbo. Not that 3.5 Turbo didn't exhibit some reasoning abilities, but they were more difficult to suss out. GPT-4 can still seem pretty vacant sometimes and finding "magic" in it has a lot to do with building up a "state" where the model is generating reason. It does not happen by default. This will all be different in another month or two, as more people have access to the latest models and new models get released (assuming there is no agreement on a pause). Then there will be another lag as laypeople catch up. There are scenarios that may play out where this speeds up even further and whole generations of AI completely fly under the radar of public acknowledgement. Or this all collapses in a debris pile of hype. I feel like I am just along for the ride at this point, trying to keep my head above water. I reckon this is going to cause a tremendous amount of stress for a tremendous number of people.