5 ms·
MusicGen-Looper: Generate fixed-bpm loops from text prompts
- KRAKRISMOTT 3y agoHow is this possible? I thought deep learning models struggle with approximating periodic functions like sin.
- bravura 3y agobigvgan paper (https://arxiv.org/pdf/2206.04658.pdf https://arxiv.org/pdf/2206.04658.pdf) "We introduce a proper inductive bias of periodicity to the generator by applying a recently proposed periodic activation called Snake function (Liu et al., 2020), defined as fα(x) = x + 1 α sin2 (αx), where α is a trainable parameter that controls the frequency of the periodic component of the signal and larger α gives higher frequency. The use of sin2 (x) ensures monotonicity and renders it amenable to easy optimization. Liu et al. (2020) demonstrates this periodic activation exhibits an improved extrapolation capability for temperature and financial data prediction."
- fagerhult 3y agoHere is the MusicGen paper from Facebook research: https://arxiv.org/abs/2306.05284 https://arxiv.org/abs/2306.05284 MusicGen is an LLM on top of EnCodec tokens, instead of working directly with audio. EnCodec is neural audio compression algorithm that encodes audio as tokens from a codebook. It's a really clever trick!
- IanCal 3y agoThe samples are outstanding. Even if they are cherry picked (not saying they are but even if) the output seems incredible. https://ai.honu.io/papers/musicgen/ https://ai.honu.io/papers/musicgen/
- deleted 3y ago[deleted]
- itairall 3y agoI would love to see something like track based MusicGen for single instruments/channels that can be mixed. Just seems like a fundamentally different problem than a photograph or painting.
- onion2k 3y agoI think it broke for me because I selected 140.06bpm. Adding 'step=1' to the range slider would help if the floating part was the cause.
- footlose_3815 3y ago"You must sign in to run this model" In this day and age, curiosity is not worth the risk.
- fagerhult 3y agoOops sorry about that, fixed now!
- nerdbert 3y agoRequired me to make an account on github? No thanks.
- JodieBenitez 3y agoThat thing refused to make me a 160 bpm chicago acid house loop... what a severe lack of imagination :-P
- dylan604 3y ago> what a severe lack of imagination I think you just described chicago acid house :-P
- tessierashpool 3y agofuck you
- dang 3y agoPlease don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- JodieBenitez 3y agoAhaha... granted. Upvoting this even though I still groove to old Phuture tracks.
- dylan604 3y agoI could see where it would be much more simple for an "AI" to generate a techno sound compared to something more melodic. I tried getting it to make things, and just had no luck with anything that sounded close to what was being requested. Some of it had a beat, but I don't think I could dance to it (can't tell since it's not actually loopable), so I gave it a 2
- marban 3y agoLong way to go before this is going to replace Splice et al.
- jnurmine 3y agoI got some unexpected melodic mellow Indian-sounding sitar-music with the input prompt "suomisaundi". Not at all what I expected, but quite nice nevertheless! It worked better with "suomisaundi psychedelic trance spugedelic". This is a neat idea in many ways.
- deleted 3y ago[deleted]
- dylan604 3y agois it just me that finds it weird that a thing that generates music loops can't loop the results in the provided player? Edit: am i mis-interpreting the term looper here? it just made an output with a fade out.
- verst 3y agoIt doesn't seem to like slow music. I jokingly gave it a prompt to create a wedding ceremony processional music for the entrance of the bride at ~60bpm (I'm getting married in 2.5 weeks). EDIT: Exact prompt "wedding procession bride entrance" File "predict.py", line 211, in predict raise ValueError( ValueError: Failed to generate a loop in the requested 60.23 bpm. Please try again. EDIT: At 52 bpm (exact) it seems to work. What it generated would not sound good if looped however. In terms of style.. it sounded a little music box like - celesta or so (think of the beginning of the Harry Potter soundtrack) with some sustained strings and pizzicato strings. That would be appropriate, except the rhythm and chords are fairly random and I wouldn't exactly call this musical :)
- Minor49er 3y agoPrompting with "harsh noise wall" resulted in some cool industrial breakbeats instead of raw noise. Looks like AI will not be taking Merzbow's job any time soon
- yoz 3y agoMany thanks for this joyous thing. I tried a few different prompts and always got something weird and interesting, but never quite what I wanted: Firstly, with temperature set to 2: “Amen break with a bag of spanners” (140 bpm): If the amen break is in there, I can’t tell. There does seem to be a kind of harp/bell thing doing the melody, though. “John Bonham with kettle drums” (90 bpm): Lots of guitar, subdued drums, but could definitely be late-period Zeppelin. Variation 2 is the exception: Zep at the start and end, long pause in the middle so John can drag his sticks along a LEGO oil tanker. “John Bonham with kettle drums and angry cat” (90 bpm): We are now inside the oil tanker. Now setting the temperature to 1: “Hardfloor in Luton Primark” (90 bpm): The bpm setting was an accidental leftover from the previous experiment, and the result sounds much more Primark than Hardfloor. “Portishead at cheezy funfair” (110 bpm): It’s a very folk-y funfair. Accordions? Organs? What the hell? Hours of fun! Again, many thanks!
- DropInIn 3y agoYep... Everything I tried didn't work.... Only altered the BPM and text prompt... Perhaps it's getting hugged to death...