3 ms·
Having an AI convergence day, so I married my tinkering with VQGAN+CLIP with GPT-J with the following experiment. Prompt is in standard font, response is in i
by jcims 5y ago
Having an AI convergence day, so I married my tinkering with VQGAN+CLIP with GPT-J with the following experiment. Prompt is in standard font, response is in italics.
----
I started messing around with image generation with VQGAN and CLIP. It is really cool, but I ran out of things to prompt it with, you know just short phrases that describe a scene that could be rendered. I shortly thereafter saw an article about GPT-J and thought it would be cool to use AI to generate the prompts. So I go the demo up and running and asked it to generate a prompt by typing this into the notebook. The computer responded with the following:
Three cats chasing a balloon with laser beams shot at them from the ground.
I did not type in "three", so it really was just a blue-print of the problem. But this got me thinking. I had seen in different contexts that we humans will generally enter some rules when generating a creative response and act as that request for the request. For example, when I am writing a resume I am usually very mindful that I do not include any errors in the use of personal pronouns. I know that when I write "I will go to the fair", I know that I could come up with the following but not the former.
I will go to the fair
I then noticed that the prompt actually contains two main requests.
Three cats chasing a balloon
And this is where things got really interesting. I have noticed that sometimes I will be able to play the two requests against each other.
The sun shines on three cats chasing a balloon
In this example I was able to generate a response that contained two unrelated requests that were both fulfilled from the prompt. This isn't a requirement, if the AI or its creators want to keep a loop from being possible, the loop will likely be re-rolled if it is present.
----
I then took the phrase 'Three cats chasing a balloon with laser beams shot at them from the ground.' and generated an image with VQGAN+CLIP and it came up with the following:
https://imgur.com/KpAdZ3W https://imgur.com/KpAdZ3W
- ionwake 5y agoIve built some code that renders speech through a 3d avatar for youtube, maybe you would like to collaborate, it should be quite easy to plug everything you just described together. If so let me know and Ill paste my email thanks
- jcims 5y agoHey thanks for touching base. This is just me tinkering at the moment but I'll ping you in the future if it sticks.