4 ms·
Really like this project. Curious to learn more about the embedding process. Can you describe the process of generating the unique description a bit more?
by ajankelo 4y ago
Really like this project. Curious to learn more about the embedding process. Can you describe the process of generating the unique description a bit more?
- jmiran15 4y agoI used the Freesound.org api (https://freesound.org/help/developers/ https://freesound.org/help/developers/) to download a bunch of sounds to MongoDB. Specifically, I selected to include the ID, Name, Tags, Description, Previews, Images, and Analysis in the response. I then looped through all these sounds and combined the response info into a prompt for GPT Davinci, instructing it to create a descriptive paragraph about the sound. I combined Davinci's description with the Freesound.org descriptions and embedded it using Ada. I inserted the embeddings into Pinecone with the audio preview url and ID as metadata. Then I just embed the search query and compare it with the sound embeddings (Pinecone allows you to select what type of similarity, I used cosine) and return the 25 most similar sounds.
- naltroc 4y agoFascinating. What if instead of reading a sound library, you had a music generator on the backend? What might an integration look like if it used the existing samples for description, as input for a "more like this" kind of feature