3 ms·
But none of them do have a generic structure. For example, GPT-3 can't produce images from text prompts, and stable diffusion cannot generate language. The poss
by nuc1e0n 4y ago
But none of them do have a generic structure. For example, GPT-3 can't produce images from text prompts, and stable diffusion cannot generate language. The possible relationships of words are written into GPT-3's code, in python, by its developers. In a way all this proves is that that written language can convey meaning to people.
- Ologn 4y ago> GPT-3 can't produce images from text prompts Me: Give me the entire hexadecimal format of an example PNG. Only give me the hexadecimal format. GPT-3: 89504E470D0A1A0A0000000D49484452000000640000006408020000000065238226000000014944415478DAECFD07780D44204C60F81EADAEF777F7E7E62F1BDE7DEBDED710EC15C7AC81CEEC17069C59B99A1698BEE7A484D68FDE782A7C41A8A0E7D2A2C9B00A99F32FBCED
- nuc1e0n 4y agoThat's not a valid png. It's just plausible hex tokens. GPT-3 is confidently wrong yet again.
- Ologn 4y ago$ echo 89504E470D0A1A0A0000000D49484452000000640000006408020000000065238226000000014944415478DAECFD07780D44204C60F81EADAEF777F7E7E62F1BDE7DEBDED710EC15C7AC81CEEC17069C59B99A1698BEE7A484D68FDE782A7C41A8A0E7D2A2C9B00A99F32FBCED |xxd -r -p > output.png $ file output.png output.png: PNG image data, 100 x 100, 8-bit/color RGB, non-interlaced $ eog output.png Fatal error reading PNG image file: IHDR: CRC error It seems you are correct.
- nuc1e0n 4y agoYeah, I did something similar before replying. Now maybe GPT-3 could be modified to make PNGs, but someone would have to go do that.
- int_19h 4y agoChatGPT is the language center of the brain, so to speak. If we are trying to model how we believe we are doing things, you'd expect a separate model optimized for images running side by side, with the two interacting (much like Sydney is GPT interacting with search). And maybe even multiple of each, so that if you give the chatbot a high-level task, it will break it down into subtasks - like it already can - and then hand each task off to a separate instance.
- theGnuMe 4y agoYou can go from an image to text and vice versa. People have done it.
- nuc1e0n 4y agoYeah, with specifically crafted models.
- WithinReason 4y agoThere are generic models that can do both
- nuc1e0n 4y agohave been made to do both.
- visarga 4y agoDALL-E 1 used a GPT approach to generate both text and images. Images are divided into patches, about 1024 patches for one image. Each patch is like a text token. > We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. https://arxiv.org/abs/2102.12092 https://arxiv.org/abs/2102.12092 The moral - you can just stream together text and images into a GPT style model.
- nuc1e0n 4y agoI don't mean to say it can't be done. Only that it has to be made to be done.
- bsaul 4y agoI was under the impression that pretty much everyone was using the transformer architecture for anything now. Isn’t that the case ?