3 ms·

Feels like you'd make more progress with discrete models working together. At the very simple level, Stable Diffusion is pretty terrible at words and fonts but

Feels like you'd make more progress with discrete models working together. At the very simple level, Stable Diffusion is pretty terrible at words and fonts but a reasonable model of where [patterns of pixels that look like] text might fit, and how big should it be. Add a second step which recognises Stable-Diffusion generated pseudotext blocks and replaces them with GPT-generated text to the same prompt set in an actual font scaled to match StableDiffusion's attempted font and you'll more likely get something that passes the "zoom in and try to read it" test. Though there may not be very much correspondence between the images and text. A more complex arrangement adds in a model which chooses which does high level structure to fit a prompt (subprompts on suitable topics for images and text for each page), a 'house style' model to pick fonts and copy/paste stuff like the RadioShack logo direct from its source material, plain old StableDiffusion to draw lots of individual pictures "cassette player deluxe RadioShack 1970" and plain old GPT to write the text which is typeset to the 'house style' model's specification, and probably an "observer" model that forces new iterations of really bad pages. Great thing about this magazine creation process is it also tends to work better for humans than having one person do everything!

uoaei 4y ago
The analogy in my mind is the heyday of x86 ubiquity vs the coming revolution of many SoCs in one device. Huge, all-purpose, massive-capacity NNs are the norm for the current advances in SOTA. This is probably because of a certain limit on development complexity: designing a complex system of richly interacting parts is hard, so as long as development of these sorts of tools is manual, it will be much easier just to shove more compute at the problem with singular, massive models. Compare this to the all-encompassing architecture hegemony of x86. This makes it relatively easy to write software that people will be able to use on devices they already own, and this reduction in developer effort and complexity is enormous for enabling rapid growth in the number of possible software programs that can be created per unit time. As ML design becomes more automated and functional, new possibilities open up regarding breaking out sub-tasks to individual, hyper-specialized tools and combining them into a resilient and capable whole that is more than the sum of its parts. That is the true power of automatic differentiation frameworks: when your system can be end-to-end differentiable throughout many devices and specialized functions, and you can train parts as easily as you can train the whole, you begin to observe the creation -- the growth, really -- of a new kind of digital cognition machine.
flemhans 4y ago
So the first "prompt" would be "write a series of prompts for an AI to use as steps to make variations of this image, and connect it together afterwards"

uoaei 4y ago

The analogy in my mind is the heyday of x86 ubiquity vs the coming revolution of many SoCs in one device. Huge, all-purpose, massive-capacity NNs are the norm for the current advances in SOTA. This is probably because of a certain limit on development complexity: designing a complex system of richly interacting parts is hard, so as long as development of these sorts of tools is manual, it will be much easier just to shove more compute at the problem with singular, massive models. Compare this to the all-encompassing architecture hegemony of x86. This makes it relatively easy to write software that people will be able to use on devices they already own, and this reduction in developer effort and complexity is enormous for enabling rapid growth in the number of possible software programs that can be created per unit time. As ML design becomes more automated and functional, new possibilities open up regarding breaking out sub-tasks to individual, hyper-specialized tools and combining them into a resilient and capable whole that is more than the sum of its parts. That is the true power of automatic differentiation frameworks: when your system can be end-to-end differentiable throughout many devices and specialized functions, and you can train parts as easily as you can train the whole, you begin to observe the creation -- the growth, really -- of a new kind of digital cognition machine.

flemhans 4y ago

So the first "prompt" would be "write a series of prompts for an AI to use as steps to make variations of this image, and connect it together afterwards"