3 ms·

I think DALL-E is clearly just gluing things together that it found in a massive dataset, and doesn't have any understanding of the underlying concepts. I thoug

by avalys 4y ago

I think DALL-E is clearly just gluing things together that it found in a massive dataset, and doesn't have any understanding of the underlying concepts. I thought it was easy to see the signs of this in examining its output. Same for GPT-3. However, what's amazing about DALL-E and these other statistical, generative models to me is that it's made me think about how much of my daily thought processes are actually just gluing things together from some kind of fuzzy statistical model in my head. When I see an acquaintance on the street, I don't carefully consider and "think" about what to say to them. I just blurt something out from some database of stock greetings in my head - which are probably based on and weighted by how people have reacted in the past, what my own friends have used similar greetings, and what "cool" people say in TV and other media in similar circumstances. "Hey man how's it going?" If I was asked to draw an airplane, I don't "think" about what an airplane looks like from first principles - I can just synthesize one in my head and start drawing. There are tons of daily activities like this that we do like this that don't involve anything I'd call "intelligent thought." I have several relatives that, in the realm of political thought, don't seem to have anything more in their head than a GPT-3 models trained on Fox News (that, just like GPT-3, can't detect any logical contradictions between sentences). DALL-E has convinced me that even current deep learning models are probably very close to replicating the performance of a significant part of my brain. Not the most important part or the most "human" part, perhaps. But I don't see any major conceptual roadblock between that part and what we call conscious, intelligent thought. Just many more layers of connectivity, abstraction, and training data. Before DALL-E I didn't believe that simply throwing more compute at the AGI problem would one day solve it. Now I do.

dougmwne 4y ago
I think our intelligence has 2 components, our conscious selves that are good at formal reasoning and meaning making, and a massive neural network that is good at statistical inference. I suspect the reason we need less training data for learning than dall-e is because our conscious selves can tease out the formal relationships and models, rather than relying purely on inference which requires enough datapoints.
multibit 4y ago
>I have several relatives that, in the realm of political thought, don't seem to have anything more in their head than a GPT-3 models trained on Fox News (that, just like GPT-3, can't detect any logical contradictions between sentences). If more people were to realize that we're all probably like this, trained on some particular dataset (like mainstream vs reactionary news/opinion), I wonder if that would lead to a kind of common peace and understanding, perhaps stemming only from a deep nihilism.

by avalys 4y ago

dougmwne 4y ago

I think our intelligence has 2 components, our conscious selves that are good at formal reasoning and meaning making, and a massive neural network that is good at statistical inference. I suspect the reason we need less training data for learning than dall-e is because our conscious selves can tease out the formal relationships and models, rather than relying purely on inference which requires enough datapoints.

multibit 4y ago

>I have several relatives that, in the realm of political thought, don't seem to have anything more in their head than a GPT-3 models trained on Fox News (that, just like GPT-3, can't detect any logical contradictions between sentences). If more people were to realize that we're all probably like this, trained on some particular dataset (like mainstream vs reactionary news/opinion), I wonder if that would lead to a kind of common peace and understanding, perhaps stemming only from a deep nihilism.