3 ms·

It's not creative in the sense that it's clearly an automaton and has no awareness of culture, history, or context. It seems to operate like a huge multidimens

It's not creative in the sense that it's clearly an automaton and has no awareness of culture, history, or context. It seems to operate like a huge multidimensional map of an image space with some verbal tags stuck into it rather randomly, and with style transfer applied as an operator to merge different features in the results. What it needs is an equivalent map for the concepts behind the signposts, and also a mapping between concepts and images. The concept map would be very hard to train because a concept space that includes all of culture and history would be huge and messy. A large concept map with a huge set of associations is one of the features of educated humans. But the concept mesh is grounded in subjective experience, so it would need an equivalent layer for that.

visarga 4y ago
> What it needs is an equivalent map for the concepts behind the signposts, It's the CLIP text and image encoders of DALL-E 2. > and also a mapping between concepts and images. It's the "prior" module. You guessed right but DALL-E 2 already has these functionalities in its design. https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/04/DALL-E-2-architecture.jpg https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/04...
- astrange 4y ago
  It seems like it'd be easier to use if you could see the intermediate steps, i.e. what the CLIP encoder "thought" the text prompt meant, and how it "thought" to turn that into an image prior. That could be done by decoding both into text again. Also, if you could feed it some kind of "knowledge graph" of words and images rather than a sentence, you wouldn't be misunderstood the same way, but you'd presumably run into the limit of what its embeddings can contain.
HKH2 4y ago
I would argue that the risk-taking part of the creative process can also be indifferent to context. Highly impulsive processes need not be inclined to rationality. Yes there needs to be an evaluator to guide and contextualize the results, but when most of the mechanical parts can be done so quickly (often most of the effort of the creative process), that's bound to lead to far higher output and willingness to get creative work done. You're right that it in itself is not strictly creative because it has no intentionality, but I've no doubt it will lead to more creativity.

visarga 4y ago

> What it needs is an equivalent map for the concepts behind the signposts, It's the CLIP text and image encoders of DALL-E 2. > and also a mapping between concepts and images. It's the "prior" module. You guessed right but DALL-E 2 already has these functionalities in its design. https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/04/DALL-E-2-architecture.jpg https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/04...

astrange 4y ago
It seems like it'd be easier to use if you could see the intermediate steps, i.e. what the CLIP encoder "thought" the text prompt meant, and how it "thought" to turn that into an image prior. That could be done by decoding both into text again. Also, if you could feed it some kind of "knowledge graph" of words and images rather than a sentence, you wouldn't be misunderstood the same way, but you'd presumably run into the limit of what its embeddings can contain.

HKH2 4y ago

I would argue that the risk-taking part of the creative process can also be indifferent to context. Highly impulsive processes need not be inclined to rationality. Yes there needs to be an evaluator to guide and contextualize the results, but when most of the mechanical parts can be done so quickly (often most of the effort of the creative process), that's bound to lead to far higher output and willingness to get creative work done. You're right that it in itself is not strictly creative because it has no intentionality, but I've no doubt it will lead to more creativity.