8 ms·

Show HN: Riffusion with Lyrics

186 points by haykmartiros 3y ago

Almost a year ago someone stumbled upon our hobby project and posted it to HN (edit: https://news.ycombinator.com/item?id=33999162 https://news.ycombinator.com/item?id=33999162). It was magical to see so many of you enjoy something we built for our love of music. We’re back today with a real product and are excited to see what you think! riffusion.com lets you create original music via short, shareable audio clips called “riffs”. You describe the lyrics and a musical style to generate riffs complete with singing and custom artwork in a few seconds. From inspiring musicians, to wishing your mom a ‘good morning!’, we are having a lot of fun with riffs as a form of expression and communication. We hope you have as much of a blast traveling the latent space of music as we did making it. You can generate music from text, remix other riffs, or record your own lyric timings. A riff for HN: https://www.riffusion.com/riffs/ac7ff4c3-d86d-4dfa-884e-bb19ec6c4e40 https://www.riffusion.com/riffs/ac7ff4c3-d86d-4dfa-884e-bb19... Hayk + Seth

mdrzn 3y ago
"These lyrics didn't work, try mixing it up with something original" doesn't give enough clues to figure out WHY those lyrics didn't work and what I have to change.
drev 3y ago
Just like this radio ad in GTA vice city https://youtube.com/clip/UgkxdC87tLJ-QeYg-Qiue8PJpGdiiUFxOR1c?si=IJIxK_bJCHJuK88r https://youtube.com/clip/UgkxdC87tLJ-QeYg-Qiue8PJpGdiiUFxOR1...
fishkingsin 3y ago
Love to see multiple languages support
eknight15 3y ago
These are sweet: https://www.riffusion.com/riffs/48375c60-7b44-41d7-8aec-0ce536ea2682 https://www.riffusion.com/riffs/48375c60-7b44-41d7-8aec-0ce5... https://www.riffusion.com/riffs/205bd650-4453-4239-a5ec-0e723e90cebd https://www.riffusion.com/riffs/205bd650-4453-4239-a5ec-0e72...
speial 3y ago
Got an error when I tried to make a riff: TRPCClientError: JSON.parse: unexpected character at line 1 column 1 of the JSON data NextJS 3 TRPCClientError from B Caused by: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
al_be_back 3y ago
>> Almost a year ago someone stumbled upon our hobby project and posted it to HN which HN post was that?
- seth_ 3y ago
  This one! Was a wild day for us :) https://news.ycombinator.com/item?id=33999162 https://news.ycombinator.com/item?id=33999162
  - al_be_back 3y ago
    nice
speial 3y ago
Is it still open source? The GitHub seems kinda stalled with the last changes from 7 months ago?
- vlugorilla 3y ago
  Seems like it's no longer open source, as you say old repo seems abandoned, but I'd also like to know if it's somewhere else or why they decided to close it.
sisomercy 3y ago
LFG..again!!!
earthnail 3y ago
Riffusion is great! Wishing you the best of luck, Hayk and Seth. Question: what are role models of companies that generated a fun "toy" like yours (hope you don't mind the word; not meant negatively) and somehow figured out how to monetise it later?
- seth_ 3y ago
  We're happy to be building a toy! Your comment reminds me of this post: https://cdixon.org/2010/01/03/the-next-big-thing-will-start-out-looking-like-a-toy https://cdixon.org/2010/01/03/the-next-big-thing-will-start-... It's still really early innings for this technology, so we're happy to be learning and building fun technology helps people to do creative things. Main thing we're focused on is turning it from a toy you enjoy once, to something you come back to and dive deeper into (even if it's still just for fun).
  - earthnail 3y ago
    I agree, but my question is: what are specific companies that pulled this off well that you consider as role models? Not meaning to put you in a corner, just really curious about this - I love Riffusion and hope you guys succeed.
rhythmofrest 3y ago
I absolutely love this, and it nails genre really well. I think it’s optimised for English, right? The result of trying to get it to speak Spanish is pretty funny! An attempt at norteño/gaucho/polka: https://www.riffusion.com/riffs/28aeb4a7-cf41-4efc-9519-3a10db24eb21 https://www.riffusion.com/riffs/28aeb4a7-cf41-4efc-9519-3a10... Reggaeton: https://www.riffusion.com/riffs/1afbbb7a-9987-4e17-89af-18056c348f4e https://www.riffusion.com/riffs/1afbbb7a-9987-4e17-89af-1805...
- seth_ 3y ago
  Nice! It is best at pronouncing in English, but we've had a bunch of fun trying to get other languages too. Sometimes you can make things happen phonetically. Even for english words that it doesn't get right the first time haha
  - rhythmofrest 3y ago
    It produces this beautiful nonsense in Japanese also, and even correctly extracted the topic from the Japanese lyrics : https://www.riffusion.com/riffs/71fa41f3-1488-4e9e-8f99-5f814c878b55 https://www.riffusion.com/riffs/71fa41f3-1488-4e9e-8f99-5f81...
    - seth_ 3y ago
      Very cool. The latent space is a wild place.
- ramoz 3y ago
  country: https://www.riffusion.com/riffs/79a0fa02-5ee6-4fdf-bdba-9aaca2729f73 https://www.riffusion.com/riffs/79a0fa02-5ee6-4fdf-bdba-9aac...
unshavedyak 3y ago
This is kinda crazy to say for me.. but this is one of the very few apps i actually want an iOS app for. Which to me personally, signals the value of the service. Great job edit: Though it feels a bit weird to need my phone number from a desktop hah. Interesting mix of not-mobile-first-yes-mobile-first. GPU cost mitigation for bots i assume?
- seth_ 3y ago
  We would have an iOS app if we were better mobile engineers... hopefully someday we'll make one!
  - Alifatisk 3y ago
    I highly suggest Flutter if you are open for ideas!
Yenrabbit 3y ago
This is excellent! I've been turning my family's memorable sayings into smooth jazz. Congratulations
deleted 3y ago
[deleted]
cmaury 3y ago
Riffs need a downvote button. I'm not sure what the team's training/eval approach is, but you need a user signal for poor quality results, so you can improve over time. ex: https://www.riffusion.com/riffs/2a42259b-1557-4913-bc04-5f0a1b0a56bd https://www.riffusion.com/riffs/2a42259b-1557-4913-bc04-5f0a... Amazing progress since the original announcement.
abledon 3y ago
AMAZING
neom 3y ago
Fun. How long till I can write a full song, "riffusionise" it and export an .als into Ableton? Then we're really cooking with gas. I was surprisingly impressed at how good of a job it did. Nice work. https://www.riffusion.com/riffs/65768974-409b-4c93-943f-6f738cdf1eb8 https://www.riffusion.com/riffs/65768974-409b-4c93-943f-6f73...
m3kw9 3y ago
I don't see the appeal? It's pretty stty music, fine it has good novelty, but would I keep clicking it for 10 second riffs of mediocre songs? I do see a viral angle, and that is good because as an app dev, getting something viral is a home run hit. But I don't see it grabbing me to come back, hopefully there is going to be a lot more features to keep people coming back
- bemmu 3y ago
  I just lost an hour to this site, it's a lot of fun. Could have potential to generate a one-in-a-million hit that becomes a tiktok sensation just from the pure volume of these mediocre songs where one just happens to be a diamond. Check out my soundcloud https://www.riffusion.com/riffs/3a91e0e4-a069-4282-b060-c38950d131f5 https://www.riffusion.com/riffs/3a91e0e4-a069-4282-b060-c389...
  - m3kw9 3y ago
    I can see people attaching songs to TikTok, not! TikTok would have their own version out before they allow Riff to easily attach outside music on their platform. On a footnote, everyone of the songs seem to sound a bit derivative in a very “I’ve heard that somewhere before” vibe, you may get lyrics that vibes but it all seem “weak”. I’m just saying it as it is, not trying to be mean or putting the devs down, it’s just honest useful critique
    - earthnail 3y ago
      When the drum machine was released, tons of musicians said it sounded rubbish and not like real drums. Time will tell whether this falls in that same category, but generally it’s easy to underestimate new stuff.
      - vunderba 3y ago
        Mmm, this really isn't the same thing. With a drum machine you're still programming the riffs themselves and people's chief complaint was the sound itself, whereas I've listened to a number of samples from this and the quality is definitely there, but the lyrical melodies themselves are not particularly harmonically engaging or memorable. Like the voices are just all over the place on the scale. That being said, this is definitely a "meme" kind of thing rather than a tool for actual orchestration.
smusamashah 3y ago
Please add a way to delete riffs.
- haykmartiros 3y ago
  as of a few minutes ago, you can mark individual riffs as public or unlisted, in addition to the global setting!
isoprophlex 3y ago
I can't believe how much you managed to improve this since that first HN post. You seem to have solved the phase distortion problem completely, and the lyrics are just insane. A more fully featured UI to eg. align lyrics & append more (temporally consistent) generated music and you have a killer app. Super well done :)
moralestapia 3y ago
Really nice! Made this one, https://www.riffusion.com/riffs/48311e5a-ec7e-4e37-9631-3c766f1d2ddc https://www.riffusion.com/riffs/48311e5a-ec7e-4e37-9631-3c76... Some hallucinations at the end, but pretty good overall, and fun!
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
sjkoelle 3y ago
awesome, good job
gremlinsinc 3y ago
well.. f**k... I lost all hope of being productive today.
tinytera 3y ago
This is great! For remixing, are you guys actually saving the latent assets/embeddings? Or regenerating the music from some "seed"?
- haykmartiros 3y ago
  Remix is using latent space interpolation under the hood to generate three variations of a given riff with (light, medium and heavy) modifications
milkweed 3y ago
What fun! Made some awesome birthday greetings.
realrogerchen 3y ago
Sorry guys i gotta go. I need to return some video tapes: https://www.riffusion.com/riffs/ee65d404-536c-4f54-a7f1-0a388c1f0264 https://www.riffusion.com/riffs/ee65d404-536c-4f54-a7f1-0a38...
abledon 3y ago
I am blown away at how the lyrics sync up with the melodies... like some of the outputs are JINGLES. that a company would pay an agency like 300,000$ to come up with in 1995. The audio quality is so good, akin to an old VHS casette you'd find playing a commercial or something. Just incredible. feature requests: - Need a Delete button on the song tiles so we can garbage the bad generations we don't want to keep.
divyapatel12 3y ago
I am blown away by how well the country and opera genres work. Country: https://www.riffusion.com/riffs/746af7e4-0578-4688-9a5f-69e30f3a35ee https://www.riffusion.com/riffs/746af7e4-0578-4688-9a5f-69e3... Opera: https://www.riffusion.com/riffs/7bce0bd1-4faf-4919-ac66-a5d028bff8e4 https://www.riffusion.com/riffs/7bce0bd1-4faf-4919-ac66-a5d0...
- seth_ 3y ago
  Those are awesome! Sorry if the app is down for you or anyone else right now btw. Our servers are a little overloaded but we'll be back up in no time
dang 3y ago
Related ongoing thread: AI-generating music app Riffusion turns viral success into $4M in funding - https://news.ycombinator.com/item?id=37915393 https://news.ycombinator.com/item?id=37915393 - Oct 2023 (49 comments)
locusofself 3y ago
Are any of these music AI's trying to use VSTi instruments and the like? It seems like with the wealth of amazing quality instruments that just accept MIDI data, this would be a really good idea and require almost no compute power comparitively.
tsaiDavid 3y ago
Congrats! Love seeing this project take off in this way, it’s so exciting.
c420 3y ago
Everytime I try and remix smth it starts processing and then tells me the lyrics don't work and to create smth original. Edit: after playing with it for an hour, I agree with other posters that midi and/or .als export options would be very desirable. Worth paying for even.

186 points by haykmartiros 3y ago

mdrzn 3y ago

"These lyrics didn't work, try mixing it up with something original" doesn't give enough clues to figure out WHY those lyrics didn't work and what I have to change.

drev 3y ago

Just like this radio ad in GTA vice city https://youtube.com/clip/UgkxdC87tLJ-QeYg-Qiue8PJpGdiiUFxOR1c?si=IJIxK_bJCHJuK88r https://youtube.com/clip/UgkxdC87tLJ-QeYg-Qiue8PJpGdiiUFxOR1...

fishkingsin 3y ago

Love to see multiple languages support

eknight15 3y ago

These are sweet: https://www.riffusion.com/riffs/48375c60-7b44-41d7-8aec-0ce536ea2682 https://www.riffusion.com/riffs/48375c60-7b44-41d7-8aec-0ce5... https://www.riffusion.com/riffs/205bd650-4453-4239-a5ec-0e723e90cebd https://www.riffusion.com/riffs/205bd650-4453-4239-a5ec-0e72...

speial 3y ago

Got an error when I tried to make a riff: TRPCClientError: JSON.parse: unexpected character at line 1 column 1 of the JSON data NextJS 3 TRPCClientError from B Caused by: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

al_be_back 3y ago

>> Almost a year ago someone stumbled upon our hobby project and posted it to HN which HN post was that?

seth_ 3y ago
This one! Was a wild day for us :) https://news.ycombinator.com/item?id=33999162 https://news.ycombinator.com/item?id=33999162
- al_be_back 3y ago
  nice

speial 3y ago

Is it still open source? The GitHub seems kinda stalled with the last changes from 7 months ago?

vlugorilla 3y ago
Seems like it's no longer open source, as you say old repo seems abandoned, but I'd also like to know if it's somewhere else or why they decided to close it.

sisomercy 3y ago

LFG..again!!!

earthnail 3y ago

Riffusion is great! Wishing you the best of luck, Hayk and Seth. Question: what are role models of companies that generated a fun "toy" like yours (hope you don't mind the word; not meant negatively) and somehow figured out how to monetise it later?

seth_ 3y ago
We're happy to be building a toy! Your comment reminds me of this post: https://cdixon.org/2010/01/03/the-next-big-thing-will-start-out-looking-like-a-toy https://cdixon.org/2010/01/03/the-next-big-thing-will-start-... It's still really early innings for this technology, so we're happy to be learning and building fun technology helps people to do creative things. Main thing we're focused on is turning it from a toy you enjoy once, to something you come back to and dive deeper into (even if it's still just for fun).
- earthnail 3y ago
  I agree, but my question is: what are specific companies that pulled this off well that you consider as role models? Not meaning to put you in a corner, just really curious about this - I love Riffusion and hope you guys succeed.

rhythmofrest 3y ago

I absolutely love this, and it nails genre really well. I think it’s optimised for English, right? The result of trying to get it to speak Spanish is pretty funny! An attempt at norteño/gaucho/polka: https://www.riffusion.com/riffs/28aeb4a7-cf41-4efc-9519-3a10db24eb21 https://www.riffusion.com/riffs/28aeb4a7-cf41-4efc-9519-3a10... Reggaeton: https://www.riffusion.com/riffs/1afbbb7a-9987-4e17-89af-18056c348f4e https://www.riffusion.com/riffs/1afbbb7a-9987-4e17-89af-1805...

seth_ 3y ago
Nice! It is best at pronouncing in English, but we've had a bunch of fun trying to get other languages too. Sometimes you can make things happen phonetically. Even for english words that it doesn't get right the first time haha
- rhythmofrest 3y ago
  It produces this beautiful nonsense in Japanese also, and even correctly extracted the topic from the Japanese lyrics : https://www.riffusion.com/riffs/71fa41f3-1488-4e9e-8f99-5f814c878b55 https://www.riffusion.com/riffs/71fa41f3-1488-4e9e-8f99-5f81...
  - seth_ 3y ago
    Very cool. The latent space is a wild place.
ramoz 3y ago
country: https://www.riffusion.com/riffs/79a0fa02-5ee6-4fdf-bdba-9aaca2729f73 https://www.riffusion.com/riffs/79a0fa02-5ee6-4fdf-bdba-9aac...

unshavedyak 3y ago

This is kinda crazy to say for me.. but this is one of the very few apps i actually want an iOS app for. Which to me personally, signals the value of the service. Great job edit: Though it feels a bit weird to need my phone number from a desktop hah. Interesting mix of not-mobile-first-yes-mobile-first. GPU cost mitigation for bots i assume?

seth_ 3y ago
We would have an iOS app if we were better mobile engineers... hopefully someday we'll make one!
- Alifatisk 3y ago
  I highly suggest Flutter if you are open for ideas!

Yenrabbit 3y ago

This is excellent! I've been turning my family's memorable sayings into smooth jazz. Congratulations

deleted 3y ago

[deleted]

cmaury 3y ago

Riffs need a downvote button. I'm not sure what the team's training/eval approach is, but you need a user signal for poor quality results, so you can improve over time. ex: https://www.riffusion.com/riffs/2a42259b-1557-4913-bc04-5f0a1b0a56bd https://www.riffusion.com/riffs/2a42259b-1557-4913-bc04-5f0a... Amazing progress since the original announcement.

abledon 3y ago

AMAZING

neom 3y ago

Fun. How long till I can write a full song, "riffusionise" it and export an .als into Ableton? Then we're really cooking with gas. I was surprisingly impressed at how good of a job it did. Nice work. https://www.riffusion.com/riffs/65768974-409b-4c93-943f-6f738cdf1eb8 https://www.riffusion.com/riffs/65768974-409b-4c93-943f-6f73...

m3kw9 3y ago

I don't see the appeal? It's pretty stty music, fine it has good novelty, but would I keep clicking it for 10 second riffs of mediocre songs? I do see a viral angle, and that is good because as an app dev, getting something viral is a home run hit. But I don't see it grabbing me to come back, hopefully there is going to be a lot more features to keep people coming back

bemmu 3y ago
I just lost an hour to this site, it's a lot of fun. Could have potential to generate a one-in-a-million hit that becomes a tiktok sensation just from the pure volume of these mediocre songs where one just happens to be a diamond. Check out my soundcloud https://www.riffusion.com/riffs/3a91e0e4-a069-4282-b060-c38950d131f5 https://www.riffusion.com/riffs/3a91e0e4-a069-4282-b060-c389...
- m3kw9 3y ago
  I can see people attaching songs to TikTok, not! TikTok would have their own version out before they allow Riff to easily attach outside music on their platform. On a footnote, everyone of the songs seem to sound a bit derivative in a very “I’ve heard that somewhere before” vibe, you may get lyrics that vibes but it all seem “weak”. I’m just saying it as it is, not trying to be mean or putting the devs down, it’s just honest useful critique
  - earthnail 3y ago
    When the drum machine was released, tons of musicians said it sounded rubbish and not like real drums. Time will tell whether this falls in that same category, but generally it’s easy to underestimate new stuff.
    - vunderba 3y ago
      Mmm, this really isn't the same thing. With a drum machine you're still programming the riffs themselves and people's chief complaint was the sound itself, whereas I've listened to a number of samples from this and the quality is definitely there, but the lyrical melodies themselves are not particularly harmonically engaging or memorable. Like the voices are just all over the place on the scale. That being said, this is definitely a "meme" kind of thing rather than a tool for actual orchestration.

smusamashah 3y ago

Please add a way to delete riffs.

haykmartiros 3y ago
as of a few minutes ago, you can mark individual riffs as public or unlisted, in addition to the global setting!

isoprophlex 3y ago

I can't believe how much you managed to improve this since that first HN post. You seem to have solved the phase distortion problem completely, and the lyrics are just insane. A more fully featured UI to eg. align lyrics & append more (temporally consistent) generated music and you have a killer app. Super well done :)

moralestapia 3y ago

Really nice! Made this one, https://www.riffusion.com/riffs/48311e5a-ec7e-4e37-9631-3c766f1d2ddc https://www.riffusion.com/riffs/48311e5a-ec7e-4e37-9631-3c76... Some hallucinations at the end, but pretty good overall, and fun!

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]

sjkoelle 3y ago

awesome, good job

gremlinsinc 3y ago

well.. f**k... I lost all hope of being productive today.

tinytera 3y ago

This is great! For remixing, are you guys actually saving the latent assets/embeddings? Or regenerating the music from some "seed"?

haykmartiros 3y ago
Remix is using latent space interpolation under the hood to generate three variations of a given riff with (light, medium and heavy) modifications

milkweed 3y ago

What fun! Made some awesome birthday greetings.

realrogerchen 3y ago

Sorry guys i gotta go. I need to return some video tapes: https://www.riffusion.com/riffs/ee65d404-536c-4f54-a7f1-0a388c1f0264 https://www.riffusion.com/riffs/ee65d404-536c-4f54-a7f1-0a38...

abledon 3y ago

I am blown away at how the lyrics sync up with the melodies... like some of the outputs are JINGLES. that a company would pay an agency like 300,000$ to come up with in 1995. The audio quality is so good, akin to an old VHS casette you'd find playing a commercial or something. Just incredible. feature requests: - Need a Delete button on the song tiles so we can garbage the bad generations we don't want to keep.

divyapatel12 3y ago

I am blown away by how well the country and opera genres work. Country: https://www.riffusion.com/riffs/746af7e4-0578-4688-9a5f-69e30f3a35ee https://www.riffusion.com/riffs/746af7e4-0578-4688-9a5f-69e3... Opera: https://www.riffusion.com/riffs/7bce0bd1-4faf-4919-ac66-a5d028bff8e4 https://www.riffusion.com/riffs/7bce0bd1-4faf-4919-ac66-a5d0...

seth_ 3y ago
Those are awesome! Sorry if the app is down for you or anyone else right now btw. Our servers are a little overloaded but we'll be back up in no time

dang 3y ago

Related ongoing thread: AI-generating music app Riffusion turns viral success into $4M in funding - https://news.ycombinator.com/item?id=37915393 https://news.ycombinator.com/item?id=37915393 - Oct 2023 (49 comments)

locusofself 3y ago

Are any of these music AI's trying to use VSTi instruments and the like? It seems like with the wealth of amazing quality instruments that just accept MIDI data, this would be a really good idea and require almost no compute power comparitively.

tsaiDavid 3y ago

Congrats! Love seeing this project take off in this way, it’s so exciting.

c420 3y ago

Everytime I try and remix smth it starts processing and then tells me the lyrics don't work and to create smth original. Edit: after playing with it for an hour, I agree with other posters that midi and/or .als export options would be very desirable. Worth paying for even.