17 ms·

MusicGen: Simple and controllable music generation

444 points by famouswaffles 3y ago

FpUser 3y ago
IS it me or I just simply do not hear any real music in the "examples". I tried quite a few and would not want to use any for listening.
bulbosaur123 3y ago
How do you run AudioCraft on Apple Silicon?
xfour 3y ago
Pretty good at transcribing the text, but the outputted music feels for lack of a better word “safe”. For example the kicking beat is way to generic and soft.
- deleted 3y ago
  [deleted]
skilled 3y ago
Except for the 2 min lowfi demo, I found the examples to be pretty bad. Sounds like the music is being played in a cardboard box in your garage’s corner.
- malux85 3y ago
  Just like the first generated AI images were full of blurry artefacts, Then 2 months later, they weren’t. You’ve got to start somewhere.
obiefernandez 3y ago
Music is already one of the most extremely devalued art forms given how oversupplied it is. Boggles the mind to think of the consequences of technology like this reaching quality levels where the differences between it and professionally produced music are imperceptible.
- Zetobal 3y ago
  [flagged]
  - istjohn 3y ago
    I downvoted because your comment is unnecessarily passive agressive and combative.
    - Zetobal 3y ago
      Is it? He stated something as a fact and I want him to take the viewpoint of other persons than himself and think about it. :)
      - amanaplanacanal 3y ago
        If there is a viewpoint you want people to think about, it would be much more productive to just state what it is.
        Zetobal 3y ago
        No, I don't want to influence him it's way easier to see the prejudice of other people if you let them think for themselves.
- rjh29 3y ago
  One consequence is bespoke music that changes dynamically, e.g. in games. There's also a relative dearth of royalty free music for independent content creators to use. AI would enable them to produce better content on a limited budget. People who enjoy creating music from scratch will be unaffected - recognition and financial rewards are tiny already for most.
- usaar333 3y ago
  If anything, the consequences seem minimal because it is so oversupplied.
rubicon33 3y ago
Maybe I'm just getting older but I feel like the quality of both music and film has seriously declined over the last 5-10 years. Maybe the good stuff is still out there but lost in a sea of average garbage that has surfaced to the top. Something tells me AI isn't going to rescue us either. I just sampled a bunch of these generated tracks and they immediately remind me of the average, mediocre, soul-lacking content that most music and film is today.
- chpatrick 3y ago
  There was always garbage, you just don't remember it.
- delusional 3y ago
  There's plenty of really good creative music. If you only watch Marvel and Top 20 hits you won't know it, but there's plenty of good stuff out there. I've really enjoyed the last couple of Bon Iver releases and my favorite artist, The Tallest Man On Earth, just released his new album Henry st. Containing some super personal tracks. The music is fantastic if you just look a little.
- rjh29 3y ago
  You sound old.
- brucethemoose2 3y ago
  A flood of garbage film/music has always existed, we just don't remember it because its uninteresting. However, I think modern rec algorithms (like the Netflix home page) are recommending more mediocre stuff than the old system, and the streaming boom did produce an abnormal glut of junk. Anyway I think AI is going to spawn a music remixing/game modding/tv extending renaissance. They perform much better when pointing them at a good source (as you can see with the melody conditioning samples, and other stuff like sd img2img and finetuned llms).
  - amanaplanacanal 3y ago
    It was a smaller flood though, when it required lots of money to record an album / make a movie. Gatekeepers kept most of it out. Now anybody can do it, so there is both a lot more chaff to sort through, and an outpouring of creativity.
rvz 3y ago
And here's the code: https://github.com/facebookresearch/audiocraft https://github.com/facebookresearch/audiocraft just so you know for the weights specifically: "The weights in this repository are released under the CC-BY-NC 4.0 license as found in the LICENSE_weights file." Combine it with AI voices and voice cloning and so begins the further devaluation of musicians and artists. Might as well accelerate it and see what happens. What could possibly go wrong?
seydor 3y ago
But how is GenA going to prompt this thing? how will they know what "80s music" is about and all. In other news, goodbye Youtube audio library, this is pretty good At 3.3B parameters this should be running locally, right Meta? (Yes it does, instructions on github) I 'm not sure I've seen any MIDI LLMs , wouldn't that be more fun to do ?
cwillu 3y ago
Would be interesting to see how well it can handle modulations: “play john lennon's "imagine" in a minor key”
ddmichael 3y ago
I may as a composer be biased but AI "generating" music is just sad. The hypocrisy is that musicians have been suing each other for intellectual property reasons, while this thing is being trained on everyone's music. The law should catch up on this. I get that it's going to improve but for now it's also just elevator/supermarket music.
- seydor 3y ago
  It can be sad, happy, energetic and all. It's no beethoven but there is a market for elevator music.lots of it
  - layer8 3y ago
    Elevator music was indeed what came to my mind when listening to the examples.
    - speedgoose 3y ago
      As someone deep into elevator music, I am very excited to try this model.
- esskay 3y ago
  It's an artform so its natural for a composer to have the same reaction as a painter would to Stable Diffusion for example. That being said - it's happening, and nothings going to stop it regardless of which side of the fence people sit on.
  - ddmichael 3y ago
    "That being said - it's happening, and nothings going to stop it" Well, the European Union is already working on a legal framework for AI. It happened with GDPR and it will happen again.
    - wilg 3y ago
      Sure, but that's not going to stop it.
marban 3y ago
Time to double down on my Roland synth
jimnotgym 3y ago
I just got excited at a mention of Summits On The Air and found a boring AI article. It's an amateur radio thing btw...You go up a mountain and make use of the good propogation to call other hams. You accrue points that are as valuable as HN points are!
Tenoke 3y ago
I was playing with it yesterday and it's not bad. I'd much rather use it for e.g. YouTube videos than risk getting copyright claimed for using something that already exists.
bratao 3y ago
Meta is truly on fire with the ML releases, outpacing Google and friends. Kudos to them! I'm genuinely thrilled about the potential release of LLaMA2.
- seydor 3y ago
  Lecun is rushing everything out the door before they are forced to say "Yes Senator" again
  - blululu 3y ago
    Possibly. FAIR has always been doing great work and making it public though (PyTorch is so big that we forget about it sometimes). Sadly 'we sell ads' is going to remain the case unless product people ask users to pony up some cash to use this tech. To be fair, I would totally chuck some cash to play with something like this and I can easily imagine a world in which this technology is used to power some bizarre social experiences like an online drum circle or some such.
    - deleted 3y ago
      [deleted]
- deleted 3y ago
  [deleted]
lucis 3y ago
I wonder if any of those services can generate editable output for a software like Ableton or Logic Pro. Seems to be more useful as an "assistant" for music producers, similar to how Copilot operates.
- NickC25 3y ago
  Google has actually released Magenta [0] which is a plugin within Ableton that can do just that. It's pretty cool, it generates interesting melodic content at the click of a button. [0] https://magenta.tensorflow.org/studio https://magenta.tensorflow.org/studio
mmaunder 3y ago
This is incredible!! For all the “AI is stealing our music” naysayers here consider that all art is derivative or it lacks context and makes it nonsensical, and artists learn too.
- ddmichael 3y ago
  [flagged]
  - deleted 3y ago
    [deleted]
  - dang 3y ago
    Can you please make your substantive points without personal swipes? https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=share%20know%20learn%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor... https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- tgv 3y ago
  I don't know why you're mentioning art being derivative, but that's not the thing that worries me. What worries me, is that a good enough model will take away the incentive to write music for many, and as a consequence it will also remove performers. This will reduce demand on music teaching and instruments, which will then both become nearly inaccessible. Since learning music isn't a question of following a few youtube videos, this will leave the world with just AI music. Jazz and classical music are probably exempt, since it relies on subsidies, their audiences care about the actual performance, and AI compositions will not draw enough of a crowd to make it financially interesting. But popular music will suffer, and that's what makes development of these models straight evil.
bottlepalm 3y ago
(and now for the rare take that isn't your typical cynical/jaded internet comment) Wow this is more than good enough to use for background music in video games, stores, commercials, etc.. You really could have super dynamic music in a video game for instance that changes based on the time of day, environment, situation, mood, etc.. all combined. Combine it with a LLM DJ and you could get some fun radio stations.
- jsheard 3y ago
  > You really could have super dynamic music in a video game for instance that changes based on the time of day, environment, situation, mood, etc.. all combined. Games can and do already do this, dynamic sequencing of music from a pool of stems has been common practice for a while. Maybe this could let you do it cheaper, and AI could go more granular by creating new stems on the fly, but the onus is still on the AI developers to show something which hits as hard as someone like Mick Gordons dynamic compositions. Infinite variety is of little value if the infinite space is full of infinitely boring, uninspired content.
  - bottlepalm 3y ago
    Like I said, cynical/jaded internet commenter - I don't need Mick Gordon's dynamic compositions, I just need some background music for my game that's good enough. And no you can't do this already: const musicSpeed = inFightSequence ? 'intense' : 'chill'; const musicPrompt = `drum and bass beat with ${musicSpeed} percussions`; playMusic(musicPrompt);
    - jsheard 3y ago
      I wish you luck with your game which constantly oscillates between https://youtu.be/dLkFh9Kn8AU?t=60 https://youtu.be/dLkFh9Kn8AU?t=60 and https://youtu.be/gQktj-WkgEo?t=75 https://youtu.be/gQktj-WkgEo?t=75 because you didn't put any thought into the music beyond telling the computer to make "chill DnB" and "intense DnB".
      - 3y ago
erwincoumans 3y ago
I've tried MusicLM and other Google AI music tools and they sounded very low quality/lo-fi. Seems Facebook isn't much better?
tumult 3y ago
I don’t understand what’s motivating certain types of people to continue working on these types of AI practical implementation projects. There’s nothing good for the world this (type of AI in particular) will offer. Maybe someone will say this will let people who aren’t musicians express themselves by creating music. That’s not true. It’s as true as hiring a musician to make a song for you, given a description. And nobody would say that the person who hired the musician was expressing themself.
- 52-6F-62 3y ago
  Exactly. What happened to automatic the boring things? Instead they seem to be all in on washing out any hope in creativity and pointing people to put all their hope in minting and munging “code”. It’s so myopic and short sighted it hurts my soul. I don’t understand at all. All that money, all that knowledge and talent… and this and stupid headsets strapped to peoples faces is the game? God dammit.
- panosfilianos 3y ago
  Let's say that ebooks now include metadata for soundtrack generation as you read them. Something like this model generates it real time based on the users reading speed etc. That'd be pretty cool for example.
  - tumult 3y ago
    That does sound cool, but you don’t need a purely generative AI to do this. Dealing with a reader who jumps around, re-reads paragraphs, flips back a few pages for a moment, etc. in a coherent way seems like the more difficult and interesting problem.
- arrosenberg 3y ago
  > Maybe someone will say this will let people who aren’t musicians express themselves by creating music. I'd say it lets me, a non-musician, create generic tracks for other projects that need music, but don't need the Lord of the Rings soundtrack. If you want the insane soundtrack, you still need an artist, as this project demonstrates.
ben_w 3y ago
So, CC-BY-NC licensed model weights, and they've made sure to license the training data. And some jurisdictions are saying that copyright cannot be claimed on the output of such models. Oh, to be a fly on the wall in RIAA corporate offices… Sans schadenfreude, I think this (depending on inference speed) could be perfect for dynamic content in games (including IRL games: LARP, escape rooms, table top games, etc.)
- seydor 3y ago
  Well , a lot of artists have sued other artists for plagiarizing. Now MusicGen will be called to testify in court and show is composing method. And if it can't prove innocence, it will be put in jail
- yyyk 3y ago
  >Oh, to be a fly on the wall in RIAA corporate offices… In all likelihood, they're ok with events. Games were never anywhere near their main revenue stream. Now the labour costs on what they're actually selling are dropping to zero. RIAA's future: 1) Use AI to fake a band. 2) Use AI to write music (maybe even lyrics). Don't really care if the AI is any good. 3) Distribute output widely, note that copyright still applies to the output. 4) Use media to generate hype (the critical step). This depends only on platform control/relations, and they have that. 5) Yea, other people could technically generate same quality dreck with AI, but it won't be (and legally can't be) exactly like the hyped dreck. Others can replicate nearly everything except the hype. 6) Since the costs are near zero just about every sale is pure profit. Basically, since Music can be replicated, they'll sell hype and belonging to a fan group instead.
  - LegitShady 3y ago
    Then they'll quickly be replaced because nothing about that is special - the RIAA exists because they were positioned to guard intellectual property that gave them a monopology on IP that was culturally significant. Making and marketing an AI band isn't even interesting. Someone will be doing it on twitch and youtube an anime vtuber ensemble before the RIAA even figure out any portion of it. The media hype is because of celebrity, and AI generated stuff can't be celebrity.
yantrams 3y ago
I'm just weirded about the fact that conversation about something AS EPIC AS THIS is so boring and rudderless here on hacker news of all places. I mean like YESTERDAY I did not have this superpower to summon something as majestic as say https://fb.watch/l4ssOD40M4/ https://fb.watch/l4ssOD40M4/ with a simple 'A quirky and skronky Aphex twin sample that just hits you' Edits: I woke up to this news delivered from Yann Lecun himself in the morning on facebook[1] and my gaped mouth can still be found for onlookers to witness I suppose! LIKE THIS IS IT FOLKS! Edit 2 All those back in my days muzzak folks lamenting about the quality of contemporary music can fuck right off because you clearly havent explored enough of the modern music landscape. Dont you dare blaspheme saying modern music has stagnated or some drivel like that. It is outright offensive to folks who are pushing the boundaries like say for example The Ex from Netherlands https://www.facebook.com/theexband https://www.facebook.com/theexband https://www.theex.nl/news.html https://www.theex.nl/news.html Just because you and the other soulless people you fraternize with are ignorant of all the innovative stuff thats going on, we have to suffer through your opinion on the state of pop culture?
- fullshark 3y ago
  Writing about music is like dancing about architecture
  - gilmore606 3y ago
    This quote always makes me wish someone would give dancing about architecture a good try.
- tomjakubowski 3y ago
  that's cool but also sounds nothing like aphex twin. sort of four tet-ish maybe
  - blensor 3y ago
    I may get down voted for this but it somehow reminds me of this: https://www.youtube.com/watch?v=fboNTcjJ8bo https://www.youtube.com/watch?v=fboNTcjJ8bo
emporas 3y ago
I used MusicGen yesterday to create 50 songs or so. Three of them sound pretty good [1][2][3]. MusicGen is definitely the best of four models of the presentation. I used the prompts differently than the article and i think i got better results. Suppose there is way to measure cardio beats or electricity spikes on the brain, and we configure the machine to generate music to increase cardio beats, or decrease them, or similarly increase electrical activity of the brain or decrease it. Then psychology might be deprecated, mood will be reduced to just a music channel. [1]https://soundcloud.com/kwstas-pramatias/lounge-owl https://soundcloud.com/kwstas-pramatias/lounge-owl [2]https://soundcloud.com/kwstas-pramatias/rock-glass-shattering https://soundcloud.com/kwstas-pramatias/rock-glass-shatterin... [3]https://soundcloud.com/kwstas-pramatias/rock-owl-howling https://soundcloud.com/kwstas-pramatias/rock-owl-howling
- layer8 3y ago
  Those three are pretty nice, but I’d say they are only a starting point, an inspiration, for creating a good song out of those themes. You are probably getting downvoted for your second paragraph which is a bit out there.
  - emporas 3y ago
    Err, yeah maybe a little bit out there. Yes of course they are the starting point, a good musician may take some samples and transform a music generation to a better song for sure. Some artists state that a painting is never complete, or a song is never complete. There is always room for innovation. The prompts i used, referenced real songwriters, and the model seems to know their songs. The article does not prompt it that way. So i guess there may be a little bit of IP infringement, but we need that, only for the first bunch. Next models will be trained on the best generations of previous models.
    - pcthrowaway 3y ago
      > Some artists state that a painting is never complete, or a song is never complete Kind of like how the software is never complete :P
lelandfe 3y ago
I mean this is shockingly good. The longer “lofi” example at the bottom sounds like it could have been a Boards of Canada demo. I’m very impressed.
- Jeff_Brown 3y ago
  On sound quality, yes. But neither this nor any of the others offered for comparison feels creative in the slightest. By contrast OpenAI's JukeBox, which is maybe two years old now, really comes up with fascinating ideas.
  - lelandfe 3y ago
    No, not just on sound quality. If that was an outro for a Boards of Canada or Ulrich Schnauss demo, I would not have batted an eye.
    - woah 3y ago
      No, it sounds like a car commercial.
finger 3y ago
I can’t find any requirements. Can you run it locally on a consumer GPU?
psycedelicAI 3y ago
But it’s meta……
muglug 3y ago
For the most part the new samples still sound like melodic nonsense — in all but one of the examples the melody doesn't fit properly with the chords underneath. It really does feel like the output of a music blender. The style transfer is the most interesting bit IMO, as you get a sense of how it hears the source examples. For example, when transferring the opening to the Bach Toccata all the new samples miss out the same passing note (the fifth note in the sequence). To a human ear that note is important, and could easily have been incorporated into the new samples, but it seemingly doesn't activate enough neurons for MusicGen to care.
- justdep 3y ago
  This is as bad as it'll ever be though.
  - muglug 3y ago
    Sure, but how good can it get with transformers? Is there some upper bound imposed by this overall approach? We don't know that yet, either.
cubefox 3y ago
Surely cherrypicked, but holy cow. Where will this end? Can you imagine what the HN front page will spit out in a few years?
- civilitty 3y ago
  > Can you imagine what the HN front page will spit out in a few years? A profitable startup? That might be asking too much though
sureglymop 3y ago
The drum and bass is genuinely amazing! Wow.
toasternz 3y ago
Music is hard to describe well without using artist names or references to specific songs. There isn't an alternative way to really describe things - "Airy EDM with tropical feel" doesn't cut it. This space will belong to scrappy shadowy decentralised organisations who let you type "give me a filtered french disco song using mizell brothers era johnny hammond jazz funk samples, lil uzi rapping, with a thundercat bassline and crooning"
_sys49152 3y ago
[dead]
cutler 3y ago
As if the current music scene hasn't already plummed the depths of banality. At this rate the stars of the 70s and 80s will be in business until they expire.
mtlmtlmtlmtl 3y ago
This is still pretty deep in the uncanny valley. None of the rock examples for example even sound very much like guitar. One sounds more like trance, the others seem more like metal than rock(though interestingly trance is a lot more similar to metal than you might think. It's just hard to notice at the surface level due to very different instruments). Then again, in this case I don't mind. I'm sure someone like Simon Posford could do some really wacky sampling based off of this. Don't see myself using it to make music just for my own listening though(not much of a composer). That's still a long ways off.
cutler 3y ago
Music died in the late 80s when the DJ supplanted the musician and sampling replaced originality. This is just part of the same trend.
- whynotmaybe 3y ago
  Technology killed the video star that killed the radio star
  - mkaic 3y ago
    which killed the stage star!
- NickC25 3y ago
  Sampling didn't replace originality. Look at the samples used by Daft Punk on their Interstellar 5555 album. Plenty of originality there.
  - Kye 3y ago
    This is correct. I have about 2TB of (royalty-free) samples that I pick and chop and place and resample when making music. I actually get more original the more material I have to work with.
JakeAl 3y ago
Well it's good for unlicensed YouTube music I guess.
odyssey7 3y ago
Infinite music is interesting from the angle that the music that we value is connected to our cultural and social experience. How can we cherish a song that has never been heard before and will never be heard again, which means it is deprived of social context that would give it meaning? One answer would be to create music that shares its roots with music that the listener already knows. This music could be enjoyable, but you can't exactly sing along to a melody you're hearing for the first and last time, so it has more limited engagement potential. This is an approach to composition that you learn when you study chord progressions and other elements in music theory, and it's what I'm sensing when I listen to the MusicGen outputs. To draw from greater cultural context, you can incorporate folk and popular melodies that are widely known. Musicians love this trick. "Immature artists copy, great artists steal." MusicGen seems capable of doing this, too. To promote a novel melody as something that listeners deeply cherish, or to innovate at the level of the theory, the social context has to be built up around the content after it's generated. E.g., when introducing a new song on the radio, a common trick is to play it between songs that are already popular; building up co-occurrences with songs that already have cultural significance. My challenge to Meta would be: can you use your platform to transform some of the model's novel outputs into familiar popular music? It would be an important cultural milestone if an AI-generated melody became a familiar tune that would be played in the café, recognized, and enjoyed.
- visarga 3y ago
  > It would be an important cultural milestone if an AI-generated melody became a familiar tune that would be played in the café, recognized, and enjoyed New generative music benchmark - popularity.
  - odyssey7 3y ago
    I can see why some might think this is silly, it seems unscientific. But what exactly is human-level performance in music, and how can we detect that? If music is an artifact appreciated by listeners, then any metric apart from whether the music is listened to would be a proxy — though I can appreciate the perspective of creating for the artist’s own sake, without a need to share the creations. Popularizing some of the model’s outputs would reveal their merit against human-produced music, by allowing them to succeed or fail in attaining the same quality of cultural significance. Then again, if the artist here is the AI research team and the audience is AI enthusiasts, then the music has succeeded in being heard and attaining cultural significance. It has been remarked that Schoenberg’s music was more often defended than listened to — music written by a theorist for an audience of theorists. I am a true fan of Schoenberg’s, though I can hear that the example outputs of this work are music that is meant to be accessible to the everyday listener.
oerpli 3y ago
Also pretty bad. Might get somewhere in a few years but currently this is only noise.
andrewstuart 3y ago
Can someone explain the significance of this and how I might use it?
pkaye 3y ago
That Riffusion output is causing me both laughter and pain. The pain is from laughing causing pressure of a surgical scar on my abdomen.
AxEy 3y ago
(This is not meant to be an anti-ai-generated-art rant. It's coming whether we like it or not. But some of the motives in this thread confuse me.) Music producer here with an honest question to those saying "this will provide me with a simple soundtrack/background music for $PROJECT" Have any of you checked out / made offers on music production subreddits? Or other music subreddits? various music production discords? Elsewhere on the internet? If so, could you say what your experience has been? I ask because the music production scene is like...ridiculously saturated, and it's almost a meme in the producer community how hard it is to make even a buck producing. I suspect that there are a significant number of producers who would be happy to take your "prompt" for a small fee. Yes, I understand 1) free and 2) immediate is convenient, but isn't 1) relatively inexpensive and 2) whatever advantage intent in construction gives good too? I'm willing to admit that I'm missing something here, but I'd love it if someone could enlighten me. While I'm asking follow ups, to all the folks who love digging for new music so much that they're considering turning to prompting AIs, I'd be seriously surprised if you've really checked out all the stuff that is coming out from new producers (again, reddit, soundcloud, etc). Another meme in the producer community is how one spends hundreds/thousands of hours perfecting ones craft, and dozens of hours working on a track, only for that track to get like 5 plays on soundcloud and negligible engagement elsewhere. Are music consumers really that desperate for new tunes? Frankly a lot of us just aren't seeing it....
- comfypotato 3y ago
  Free and immediate. You answered your own question. All things equal, people are happy to support local businesses. The value prop here is far from equal.
- yieldcrv 3y ago
  the simple answer is that your motivations for being an artist need to change to be exclusively personal fulfillment. because that was true for the pre-AI world, as you essentially described, and its true-er for this current-AI world. the real meme is about how artists have always been grasping for financial respect in every market condition ever, and yet nothing has changed. people were never going to commission you, they were never going to book you. While they do appreciate the content. But for the few that would ever actually try to commission something, they encountered friction after friction after friction and collectively artists have been disinterested in solving. Because they're starving and preoccupied with fighting for scraps and modicums of respect at all. The world’s has now solved many of these frictions. The frictions were: 1 hoping they found the right artist to begin with 2 hoping that artist is reliable and has any work ethic or structure in their life 3 not bruising that artists ego in however communication style is preferred 4 dealing with how completely segregated many artists are from contract negotiations and any aspect of the business world, but needing to secure rights properly 5 ego in securing rights properly without the artist overplaying their hand 6 waiting for the commission 7 revisions 8 circle back to 1 9 if you ever get past part 8, you have the issue of whether your new license can be used in an unforeseen way and medium in the future getting burned in altruistic commissions of living artists is simply over now. all these frictions are solved with the free and immediate way.
deleted 3y ago
[deleted]
speedgoose 3y ago
I installed it and everything went surprisingly fine and easily. It used about 8GB or VRAM max on a nvidia A30 and takes about 30s to generate 10s of audio. The max duration seems to be 30s in the frontend but the quality is a lot lower. Mixing genres do not really work and the model doesn’t seem to be trained on band names. However it does perform well to create music using existing styles. I generated some Eurovision crap and minimalist techno that were very much believable. But mixing death metal with lofi ambient isn’t the best, nor the epic progressive rock guitar solo I asked. I think the examples on the website are cherry picked but with some experience in prompt engineering and many tentatives, it should be possible to generate great samples. It’s also excellent at generating boards of Canada like music. The audio artéfacts, the low fidelity, the weird sounds, the detuned synths, this model does that very well and it does sound great to me. Thanks a lot to the authors.
- solumunus 3y ago
  > But mixing death metal with lofi ambient isn’t the best Would you expect that to be good if a human did it?
  - squeaky-clean 3y ago
    (Not the original commenter) I don't know of any death metal examples, but chill-lofi-beats+djent music already exists and it's pretty good. This isn't the only example but it's my personal favorite one: https://youtu.be/-Yeqp2-dF5M https://youtu.be/-Yeqp2-dF5M
    - solumunus 3y ago
      I don't think you can have death metal without double bass hammering. I don't think you can have lofi with double bass hammering.
  - speedgoose 3y ago
    Depends on the human. If it’s myself, not really. But if it’s well done, yes I would enjoy death metal sonorities in lofi ambiant once in a while. But I admit the prompt is perhaps a bit challenging.
deleted 3y ago
[deleted]
beefman 3y ago
> a piano and cello duet playing a sad chambers music No such thing as chambers music or a music. Maybe "a sad piano and cello duet" or "sad piano and cello chamber music" would be a better prompt.
rifty 3y ago
Style transfer is pretty cool tool as someone who likes to play around with sound design. It will be a lot of fun to drop this on parallel channels and blend it together into choruses and new instruments. Right now the generation still sounds a lot like loop packs smashed together anyone could technically make. But it is practical for anyone who really only cares for that style of sound but do not themselves have the familiarity to do it. Now they can just say what they want and hit regenerate, skipping the latent feedback cycle of iterating with humans or sifting through song snippets. My opinion on this style of content is that ai generation is simply accelerating us to the inevitable end of generic digital content, it isn't really changing it. It just happens to be also the optimal interface for discovering and not just generation.
moffkalast 3y ago
> Audiocraft requires Python 3.9, PyTorch 2.0.0, and a GPU with at least 16 GB of memory I sleep. And these are only 1-3.3B param models, that makes no sense.
cma 3y ago
They messed up and shipped Bolero with it in the repository, which is copyrighted until 2025 in the US > In the U.S., Boléro remains under copyright until 1 January 2025 https://en.wikipedia.org/wiki/Bol%C3%A9ro https://en.wikipedia.org/wiki/Bol%C3%A9ro
Kiro 3y ago
I'm curious how this handles bass music where the lower frequencies need to be tightly mixed to sound good. Is that really something that a model can learn?
iamsaitam 3y ago
The question that I think tells the kind of potential that ML has for music is, how good is written language in describing music? People here compare this to the development of Stable Diffusion, but the mediums are quite different. Describe your favorite shot in a movie and most likely you’ll enumerate and get pretty far in conveying it to someone who hasn’t seen it, at least the physical aspect of it. But try describing your favorite song and the interpretations will be wildly different (outside of hard defined genre music).
- siquick 3y ago
  “Writing about music is like dancing about architecture.”
- raincole 3y ago
  1. MusicGen accepts melody prompt. 2. It doesn't have an "interrogation" feature for now (unless I missed it), but I'm sure something like this is possible.
- zavertnik 3y ago
  > The question that I think tells the kind of potential that ML has for music is, how good is written language in describing music? If you took a DAW project file of a song.wav that was completely written and produced digitally using virtual instruments and compiled all of the parameters a user had to set to achieve their output.wav into a .csv file, you may be surprised to see (1) how few parameters were used (2) how often those parameters are unchanged from their defaults and (3) the amount of those parameters that would be expected in any other project file. When you break it down, you really only have 6 layers to parse, all of which are dynamic but within a relatively small and consistent sandbox, at least relative to image generation. 1. composition layer - the midi or notes of the song. 2. arrangement layer - the selection of instruments used in the song and the division of the song's midi to the song's respective instruments. 3. instrument layer - the parameters of each instrument, such as a synth path or a virtual piano's room setting. 4. post processing layer - the effects placed on the output of each instrument, such as reverb, compression, delay, ect. 5. mixing layer - the volume of each instrument + post processing channel 6. mastering layer - processing on the master track All of these things are more or less standardized. Developers always add their own flair (read: custom parameters) for their plugins, but they can be decompiled to be a composition of each of these layer's fundamental parameters. All these parameters + the midi of a song would be a few kb I feel like a LLM trained on the parameter sets which interacts with the software used to manipulate these layers could really produce amazing tools and open the door to writing high quality songs to everyone, just as other AI products have opened so many similar doors. The DALL-E app for music, in my mind, probably wont be a text based description -> .wav output. Instead, it would be the generation of the elements of each layer with options that can be listened to in real time using whatever VSTs were used in training. When you ask ChatGPT to write a complex python script, it starts with an outline of all the methods in the script as placeholders and then takes you step by step until you're done, then you troubleshoot it or flesh it out. The best part of a generative music like this is that it leaves the user really only with having to decide if something sounds good or not. As a mostly musically illiterate producer myself, I've produced hundreds of songs and a few albums without ever really learning how to do anything other than manipulate the parameters. When I started learning to produce music I was 15 years old and knew nothing about music production. But, I was really good at was using computers and software so I learned to play the DAW, the plugins, and the sample packs. The only layer that I couldn't learn through learning software was the composition, the writing of the midi. Fortunately, the midi of a song becomes very easy to brute force over time, so I learned to brute force midi. Once I became efficient with my workflow, producing music became a task of "make this idea sound good." And without ever really feeling like I was a musician or composer, this became an enormous passion and outlet for me that I did every day for a decade. I was able to do this because at its core, all the mechanical parts of a song are simple machines and a song's quality is the way those machines are used together. As an outside, this feels like a workflow that would be very machine learning friendly. But I could be wrong!
fintechie 3y ago
Do any of these generative music ML framework export as midi instead of .wav or .mp3? That would be 1000x times more useful as the quality we're reaching is good enough. I can imagine an VSTi that just takes a prompt and generates midi tracks. Something like this is surely coming in the next couple of years.
- mk_stjames 3y ago
  That was the way almost all ML research on music was done until recently- train for MIDi generation with input of other MIDI, or, at an even more fundamental level, just notes. If you go back, tons of ML music papers were written on generating believable sequences of Bach, Mozart, etc. because it was just note prediction. Since the advent of transformers, and this idea of using text models mapping the natural language space to tagged music samples, and the music tokenizer acting directly on sampled audio stream (the bits of a .wav file, essentially) all the cutting edge work is going that route. Because it is producing high quality, finished audio streams directly. And I think part of it is because there is way, way more training data for actual audio than there is for MIDI alone (there are tons of free midi sites out there but a lot of it is garbage and it pales in comparison to what is already sampled and tagged in real audio libraries). I imagine what will happen is... within two to three years these LM-transformer-music models will get so good that the audio will be damn near spotless sounding, and there will be additional methods developed to synthesize with more control directly with the models, to the point where wanting MIDI so you can use your own HQ synth isn't needed, because if you want "the lead synth to sound less digital and more like a classic Minimoog Model D" you just add that to another 'Music2Music" pass and out pops your sound. For those who still want MIDI there is still work being done on traditional audio-to-MIDI modeling and I think you'd wind up just using that in the chain.
IAmGraydon 3y ago
I’m a musician and have spent over 20 years making music. It’s my biggest passion. I still love this. Music isn’t a competition to me so it doesn’t bother me that someone with little experience can create great sounding music. I can still make music that’s an expression of myself, or I can generate some AI music to see if something catches my ear as a starting point. It’s just another tool, and I’m excited to see how it evolves.
bionhoward 3y ago
Typo “our a paper” if the devs are watching you can remove “a”
8bitsrule 3y ago
I'd get off any elevator playing anything on that page.
kennyloginz 3y ago
All the samples on that page sound pretty bland , or have nasty phase distortions. I guess that’s the point.
MicolashKyoka 3y ago
Amazing work by FAIR! With tools like these, AI-enhanced creativity is going to give us hyper specialized output and thankfully further us even more from mainstream, one-size-fits-all art. We are living in the future!
donbrae 3y ago
These are really impressive. Providing the ‘prompter-composer’ can generate high quality, stereo mixes — these all appear to be low bitrate, mono files — you could probably use them right now for various ‘wallpaper music’ purposes, where originality is not required or is even a detriment: ‘on hold’ music, low budget video productions, symphonic film scores etc. Producing this kind of anti-music is pretty soul-destroying for a musician, anyway, so the machines might as well do it. We can then spend our time working on stuff that means something, and if we’re lucky and it connects with enough people, make a living from it.
two_in_one 3y ago
Before using just listen the samples. It doesn't have any structure even within 30 sec. It's repetitive and boring. Can be used as a background in low budget game, but that's it. It's not a threat to human talents, not yet. It's like first text generators with tiny attention windows, took some time to evolve into GPT-4. Then it will have an impact. Bans and unions may delay that for some time.

444 points by famouswaffles 3y ago

IS it me or I just simply do not hear any real music in the "examples". I tried quite a few and would not want to use any for listening.

bulbosaur123 3y ago

How do you run AudioCraft on Apple Silicon?

xfour 3y ago

Pretty good at transcribing the text, but the outputted music feels for lack of a better word “safe”. For example the kicking beat is way to generic and soft.

deleted 3y ago
[deleted]

skilled 3y ago

Except for the 2 min lowfi demo, I found the examples to be pretty bad. Sounds like the music is being played in a cardboard box in your garage’s corner.

malux85 3y ago
Just like the first generated AI images were full of blurry artefacts, Then 2 months later, they weren’t. You’ve got to start somewhere.

obiefernandez 3y ago

Music is already one of the most extremely devalued art forms given how oversupplied it is. Boggles the mind to think of the consequences of technology like this reaching quality levels where the differences between it and professionally produced music are imperceptible.

Zetobal 3y ago
[flagged]
- istjohn 3y ago
  I downvoted because your comment is unnecessarily passive agressive and combative.
  - Zetobal 3y ago
    Is it? He stated something as a fact and I want him to take the viewpoint of other persons than himself and think about it. :)
    - amanaplanacanal 3y ago
      If there is a viewpoint you want people to think about, it would be much more productive to just state what it is.
      - Zetobal 3y ago
        No, I don't want to influence him it's way easier to see the prejudice of other people if you let them think for themselves.
rjh29 3y ago
One consequence is bespoke music that changes dynamically, e.g. in games. There's also a relative dearth of royalty free music for independent content creators to use. AI would enable them to produce better content on a limited budget. People who enjoy creating music from scratch will be unaffected - recognition and financial rewards are tiny already for most.
usaar333 3y ago
If anything, the consequences seem minimal because it is so oversupplied.

rubicon33 3y ago

Maybe I'm just getting older but I feel like the quality of both music and film has seriously declined over the last 5-10 years. Maybe the good stuff is still out there but lost in a sea of average garbage that has surfaced to the top. Something tells me AI isn't going to rescue us either. I just sampled a bunch of these generated tracks and they immediately remind me of the average, mediocre, soul-lacking content that most music and film is today.

chpatrick 3y ago
There was always garbage, you just don't remember it.
delusional 3y ago
There's plenty of really good creative music. If you only watch Marvel and Top 20 hits you won't know it, but there's plenty of good stuff out there. I've really enjoyed the last couple of Bon Iver releases and my favorite artist, The Tallest Man On Earth, just released his new album Henry st. Containing some super personal tracks. The music is fantastic if you just look a little.
rjh29 3y ago
You sound old.
brucethemoose2 3y ago
A flood of garbage film/music has always existed, we just don't remember it because its uninteresting. However, I think modern rec algorithms (like the Netflix home page) are recommending more mediocre stuff than the old system, and the streaming boom did produce an abnormal glut of junk. Anyway I think AI is going to spawn a music remixing/game modding/tv extending renaissance. They perform much better when pointing them at a good source (as you can see with the melody conditioning samples, and other stuff like sd img2img and finetuned llms).
- amanaplanacanal 3y ago
  It was a smaller flood though, when it required lots of money to record an album / make a movie. Gatekeepers kept most of it out. Now anybody can do it, so there is both a lot more chaff to sort through, and an outpouring of creativity.

rvz 3y ago

And here's the code: https://github.com/facebookresearch/audiocraft https://github.com/facebookresearch/audiocraft just so you know for the weights specifically: "The weights in this repository are released under the CC-BY-NC 4.0 license as found in the LICENSE_weights file." Combine it with AI voices and voice cloning and so begins the further devaluation of musicians and artists. Might as well accelerate it and see what happens. What could possibly go wrong?

seydor 3y ago

But how is GenA going to prompt this thing? how will they know what "80s music" is about and all. In other news, goodbye Youtube audio library, this is pretty good At 3.3B parameters this should be running locally, right Meta? (Yes it does, instructions on github) I 'm not sure I've seen any MIDI LLMs , wouldn't that be more fun to do ?

cwillu 3y ago

Would be interesting to see how well it can handle modulations: “play john lennon's "imagine" in a minor key”

ddmichael 3y ago

I may as a composer be biased but AI "generating" music is just sad. The hypocrisy is that musicians have been suing each other for intellectual property reasons, while this thing is being trained on everyone's music. The law should catch up on this. I get that it's going to improve but for now it's also just elevator/supermarket music.

seydor 3y ago
It can be sad, happy, energetic and all. It's no beethoven but there is a market for elevator music.lots of it
- layer8 3y ago
  Elevator music was indeed what came to my mind when listening to the examples.
  - speedgoose 3y ago
    As someone deep into elevator music, I am very excited to try this model.
esskay 3y ago
It's an artform so its natural for a composer to have the same reaction as a painter would to Stable Diffusion for example. That being said - it's happening, and nothings going to stop it regardless of which side of the fence people sit on.
- ddmichael 3y ago
  "That being said - it's happening, and nothings going to stop it" Well, the European Union is already working on a legal framework for AI. It happened with GDPR and it will happen again.
  - wilg 3y ago
    Sure, but that's not going to stop it.

marban 3y ago

Time to double down on my Roland synth

jimnotgym 3y ago

I just got excited at a mention of Summits On The Air and found a boring AI article. It's an amateur radio thing btw...You go up a mountain and make use of the good propogation to call other hams. You accrue points that are as valuable as HN points are!

Tenoke 3y ago

I was playing with it yesterday and it's not bad. I'd much rather use it for e.g. YouTube videos than risk getting copyright claimed for using something that already exists.

bratao 3y ago

Meta is truly on fire with the ML releases, outpacing Google and friends. Kudos to them! I'm genuinely thrilled about the potential release of LLaMA2.

seydor 3y ago
Lecun is rushing everything out the door before they are forced to say "Yes Senator" again
- blululu 3y ago
  Possibly. FAIR has always been doing great work and making it public though (PyTorch is so big that we forget about it sometimes). Sadly 'we sell ads' is going to remain the case unless product people ask users to pony up some cash to use this tech. To be fair, I would totally chuck some cash to play with something like this and I can easily imagine a world in which this technology is used to power some bizarre social experiences like an online drum circle or some such.
  - deleted 3y ago
    [deleted]
deleted 3y ago
[deleted]

lucis 3y ago

I wonder if any of those services can generate editable output for a software like Ableton or Logic Pro. Seems to be more useful as an "assistant" for music producers, similar to how Copilot operates.

NickC25 3y ago
Google has actually released Magenta [0] which is a plugin within Ableton that can do just that. It's pretty cool, it generates interesting melodic content at the click of a button. [0] https://magenta.tensorflow.org/studio https://magenta.tensorflow.org/studio

mmaunder 3y ago

This is incredible!! For all the “AI is stealing our music” naysayers here consider that all art is derivative or it lacks context and makes it nonsensical, and artists learn too.

ddmichael 3y ago
[flagged]
- deleted 3y ago
  [deleted]
- dang 3y ago
  Can you please make your substantive points without personal swipes? https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=share%20know%20learn%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor... https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
tgv 3y ago
I don't know why you're mentioning art being derivative, but that's not the thing that worries me. What worries me, is that a good enough model will take away the incentive to write music for many, and as a consequence it will also remove performers. This will reduce demand on music teaching and instruments, which will then both become nearly inaccessible. Since learning music isn't a question of following a few youtube videos, this will leave the world with just AI music. Jazz and classical music are probably exempt, since it relies on subsidies, their audiences care about the actual performance, and AI compositions will not draw enough of a crowd to make it financially interesting. But popular music will suffer, and that's what makes development of these models straight evil.

bottlepalm 3y ago

(and now for the rare take that isn't your typical cynical/jaded internet comment) Wow this is more than good enough to use for background music in video games, stores, commercials, etc.. You really could have super dynamic music in a video game for instance that changes based on the time of day, environment, situation, mood, etc.. all combined. Combine it with a LLM DJ and you could get some fun radio stations.

jsheard 3y ago
> You really could have super dynamic music in a video game for instance that changes based on the time of day, environment, situation, mood, etc.. all combined. Games can and do already do this, dynamic sequencing of music from a pool of stems has been common practice for a while. Maybe this could let you do it cheaper, and AI could go more granular by creating new stems on the fly, but the onus is still on the AI developers to show something which hits as hard as someone like Mick Gordons dynamic compositions. Infinite variety is of little value if the infinite space is full of infinitely boring, uninspired content.
- bottlepalm 3y ago
  Like I said, cynical/jaded internet commenter - I don't need Mick Gordon's dynamic compositions, I just need some background music for my game that's good enough. And no you can't do this already: const musicSpeed = inFightSequence ? 'intense' : 'chill'; const musicPrompt = `drum and bass beat with ${musicSpeed} percussions`; playMusic(musicPrompt);
  - jsheard 3y ago
    I wish you luck with your game which constantly oscillates between https://youtu.be/dLkFh9Kn8AU?t=60 https://youtu.be/dLkFh9Kn8AU?t=60 and https://youtu.be/gQktj-WkgEo?t=75 https://youtu.be/gQktj-WkgEo?t=75 because you didn't put any thought into the music beyond telling the computer to make "chill DnB" and "intense DnB".
    - 3y ago

erwincoumans 3y ago

I've tried MusicLM and other Google AI music tools and they sounded very low quality/lo-fi. Seems Facebook isn't much better?

tumult 3y ago

I don’t understand what’s motivating certain types of people to continue working on these types of AI practical implementation projects. There’s nothing good for the world this (type of AI in particular) will offer. Maybe someone will say this will let people who aren’t musicians express themselves by creating music. That’s not true. It’s as true as hiring a musician to make a song for you, given a description. And nobody would say that the person who hired the musician was expressing themself.

52-6F-62 3y ago
Exactly. What happened to automatic the boring things? Instead they seem to be all in on washing out any hope in creativity and pointing people to put all their hope in minting and munging “code”. It’s so myopic and short sighted it hurts my soul. I don’t understand at all. All that money, all that knowledge and talent… and this and stupid headsets strapped to peoples faces is the game? God dammit.
panosfilianos 3y ago
Let's say that ebooks now include metadata for soundtrack generation as you read them. Something like this model generates it real time based on the users reading speed etc. That'd be pretty cool for example.
- tumult 3y ago
  That does sound cool, but you don’t need a purely generative AI to do this. Dealing with a reader who jumps around, re-reads paragraphs, flips back a few pages for a moment, etc. in a coherent way seems like the more difficult and interesting problem.
arrosenberg 3y ago
> Maybe someone will say this will let people who aren’t musicians express themselves by creating music. I'd say it lets me, a non-musician, create generic tracks for other projects that need music, but don't need the Lord of the Rings soundtrack. If you want the insane soundtrack, you still need an artist, as this project demonstrates.

ben_w 3y ago

So, CC-BY-NC licensed model weights, and they've made sure to license the training data. And some jurisdictions are saying that copyright cannot be claimed on the output of such models. Oh, to be a fly on the wall in RIAA corporate offices… Sans schadenfreude, I think this (depending on inference speed) could be perfect for dynamic content in games (including IRL games: LARP, escape rooms, table top games, etc.)

seydor 3y ago
Well , a lot of artists have sued other artists for plagiarizing. Now MusicGen will be called to testify in court and show is composing method. And if it can't prove innocence, it will be put in jail
yyyk 3y ago
>Oh, to be a fly on the wall in RIAA corporate offices… In all likelihood, they're ok with events. Games were never anywhere near their main revenue stream. Now the labour costs on what they're actually selling are dropping to zero. RIAA's future: 1) Use AI to fake a band. 2) Use AI to write music (maybe even lyrics). Don't really care if the AI is any good. 3) Distribute output widely, note that copyright still applies to the output. 4) Use media to generate hype (the critical step). This depends only on platform control/relations, and they have that. 5) Yea, other people could technically generate same quality dreck with AI, but it won't be (and legally can't be) exactly like the hyped dreck. Others can replicate nearly everything except the hype. 6) Since the costs are near zero just about every sale is pure profit. Basically, since Music can be replicated, they'll sell hype and belonging to a fan group instead.
- LegitShady 3y ago
  Then they'll quickly be replaced because nothing about that is special - the RIAA exists because they were positioned to guard intellectual property that gave them a monopology on IP that was culturally significant. Making and marketing an AI band isn't even interesting. Someone will be doing it on twitch and youtube an anime vtuber ensemble before the RIAA even figure out any portion of it. The media hype is because of celebrity, and AI generated stuff can't be celebrity.

yantrams 3y ago

I'm just weirded about the fact that conversation about something AS EPIC AS THIS is so boring and rudderless here on hacker news of all places. I mean like YESTERDAY I did not have this superpower to summon something as majestic as say https://fb.watch/l4ssOD40M4/ https://fb.watch/l4ssOD40M4/ with a simple 'A quirky and skronky Aphex twin sample that just hits you' Edits: I woke up to this news delivered from Yann Lecun himself in the morning on facebook[1] and my gaped mouth can still be found for onlookers to witness I suppose! LIKE THIS IS IT FOLKS! Edit 2 All those back in my days muzzak folks lamenting about the quality of contemporary music can fuck right off because you clearly havent explored enough of the modern music landscape. Dont you dare blaspheme saying modern music has stagnated or some drivel like that. It is outright offensive to folks who are pushing the boundaries like say for example The Ex from Netherlands https://www.facebook.com/theexband https://www.facebook.com/theexband https://www.theex.nl/news.html https://www.theex.nl/news.html Just because you and the other soulless people you fraternize with are ignorant of all the innovative stuff thats going on, we have to suffer through your opinion on the state of pop culture?

fullshark 3y ago
Writing about music is like dancing about architecture
- gilmore606 3y ago
  This quote always makes me wish someone would give dancing about architecture a good try.
tomjakubowski 3y ago
that's cool but also sounds nothing like aphex twin. sort of four tet-ish maybe
- blensor 3y ago
  I may get down voted for this but it somehow reminds me of this: https://www.youtube.com/watch?v=fboNTcjJ8bo https://www.youtube.com/watch?v=fboNTcjJ8bo

emporas 3y ago

I used MusicGen yesterday to create 50 songs or so. Three of them sound pretty good [1][2][3]. MusicGen is definitely the best of four models of the presentation. I used the prompts differently than the article and i think i got better results. Suppose there is way to measure cardio beats or electricity spikes on the brain, and we configure the machine to generate music to increase cardio beats, or decrease them, or similarly increase electrical activity of the brain or decrease it. Then psychology might be deprecated, mood will be reduced to just a music channel. [1]https://soundcloud.com/kwstas-pramatias/lounge-owl https://soundcloud.com/kwstas-pramatias/lounge-owl [2]https://soundcloud.com/kwstas-pramatias/rock-glass-shattering https://soundcloud.com/kwstas-pramatias/rock-glass-shatterin... [3]https://soundcloud.com/kwstas-pramatias/rock-owl-howling https://soundcloud.com/kwstas-pramatias/rock-owl-howling

layer8 3y ago
Those three are pretty nice, but I’d say they are only a starting point, an inspiration, for creating a good song out of those themes. You are probably getting downvoted for your second paragraph which is a bit out there.
- emporas 3y ago
  Err, yeah maybe a little bit out there. Yes of course they are the starting point, a good musician may take some samples and transform a music generation to a better song for sure. Some artists state that a painting is never complete, or a song is never complete. There is always room for innovation. The prompts i used, referenced real songwriters, and the model seems to know their songs. The article does not prompt it that way. So i guess there may be a little bit of IP infringement, but we need that, only for the first bunch. Next models will be trained on the best generations of previous models.
  - pcthrowaway 3y ago
    > Some artists state that a painting is never complete, or a song is never complete Kind of like how the software is never complete :P

lelandfe 3y ago

I mean this is shockingly good. The longer “lofi” example at the bottom sounds like it could have been a Boards of Canada demo. I’m very impressed.

Jeff_Brown 3y ago
On sound quality, yes. But neither this nor any of the others offered for comparison feels creative in the slightest. By contrast OpenAI's JukeBox, which is maybe two years old now, really comes up with fascinating ideas.
- lelandfe 3y ago
  No, not just on sound quality. If that was an outro for a Boards of Canada or Ulrich Schnauss demo, I would not have batted an eye.
  - woah 3y ago
    No, it sounds like a car commercial.

finger 3y ago

I can’t find any requirements. Can you run it locally on a consumer GPU?

psycedelicAI 3y ago

But it’s meta……

muglug 3y ago

For the most part the new samples still sound like melodic nonsense — in all but one of the examples the melody doesn't fit properly with the chords underneath. It really does feel like the output of a music blender. The style transfer is the most interesting bit IMO, as you get a sense of how it hears the source examples. For example, when transferring the opening to the Bach Toccata all the new samples miss out the same passing note (the fifth note in the sequence). To a human ear that note is important, and could easily have been incorporated into the new samples, but it seemingly doesn't activate enough neurons for MusicGen to care.

justdep 3y ago
This is as bad as it'll ever be though.
- muglug 3y ago
  Sure, but how good can it get with transformers? Is there some upper bound imposed by this overall approach? We don't know that yet, either.

cubefox 3y ago

Surely cherrypicked, but holy cow. Where will this end? Can you imagine what the HN front page will spit out in a few years?

civilitty 3y ago
> Can you imagine what the HN front page will spit out in a few years? A profitable startup? That might be asking too much though

sureglymop 3y ago

The drum and bass is genuinely amazing! Wow.

toasternz 3y ago

Music is hard to describe well without using artist names or references to specific songs. There isn't an alternative way to really describe things - "Airy EDM with tropical feel" doesn't cut it. This space will belong to scrappy shadowy decentralised organisations who let you type "give me a filtered french disco song using mizell brothers era johnny hammond jazz funk samples, lil uzi rapping, with a thundercat bassline and crooning"

_sys49152 3y ago

[dead]

cutler 3y ago

As if the current music scene hasn't already plummed the depths of banality. At this rate the stars of the 70s and 80s will be in business until they expire.

mtlmtlmtlmtl 3y ago

This is still pretty deep in the uncanny valley. None of the rock examples for example even sound very much like guitar. One sounds more like trance, the others seem more like metal than rock(though interestingly trance is a lot more similar to metal than you might think. It's just hard to notice at the surface level due to very different instruments). Then again, in this case I don't mind. I'm sure someone like Simon Posford could do some really wacky sampling based off of this. Don't see myself using it to make music just for my own listening though(not much of a composer). That's still a long ways off.

cutler 3y ago

Music died in the late 80s when the DJ supplanted the musician and sampling replaced originality. This is just part of the same trend.

whynotmaybe 3y ago
Technology killed the video star that killed the radio star
- mkaic 3y ago
  which killed the stage star!
NickC25 3y ago
Sampling didn't replace originality. Look at the samples used by Daft Punk on their Interstellar 5555 album. Plenty of originality there.
- Kye 3y ago
  This is correct. I have about 2TB of (royalty-free) samples that I pick and chop and place and resample when making music. I actually get more original the more material I have to work with.

JakeAl 3y ago

Well it's good for unlicensed YouTube music I guess.

odyssey7 3y ago

Infinite music is interesting from the angle that the music that we value is connected to our cultural and social experience. How can we cherish a song that has never been heard before and will never be heard again, which means it is deprived of social context that would give it meaning? One answer would be to create music that shares its roots with music that the listener already knows. This music could be enjoyable, but you can't exactly sing along to a melody you're hearing for the first and last time, so it has more limited engagement potential. This is an approach to composition that you learn when you study chord progressions and other elements in music theory, and it's what I'm sensing when I listen to the MusicGen outputs. To draw from greater cultural context, you can incorporate folk and popular melodies that are widely known. Musicians love this trick. "Immature artists copy, great artists steal." MusicGen seems capable of doing this, too. To promote a novel melody as something that listeners deeply cherish, or to innovate at the level of the theory, the social context has to be built up around the content after it's generated. E.g., when introducing a new song on the radio, a common trick is to play it between songs that are already popular; building up co-occurrences with songs that already have cultural significance. My challenge to Meta would be: can you use your platform to transform some of the model's novel outputs into familiar popular music? It would be an important cultural milestone if an AI-generated melody became a familiar tune that would be played in the café, recognized, and enjoyed.

visarga 3y ago
> It would be an important cultural milestone if an AI-generated melody became a familiar tune that would be played in the café, recognized, and enjoyed New generative music benchmark - popularity.
- odyssey7 3y ago
  I can see why some might think this is silly, it seems unscientific. But what exactly is human-level performance in music, and how can we detect that? If music is an artifact appreciated by listeners, then any metric apart from whether the music is listened to would be a proxy — though I can appreciate the perspective of creating for the artist’s own sake, without a need to share the creations. Popularizing some of the model’s outputs would reveal their merit against human-produced music, by allowing them to succeed or fail in attaining the same quality of cultural significance. Then again, if the artist here is the AI research team and the audience is AI enthusiasts, then the music has succeeded in being heard and attaining cultural significance. It has been remarked that Schoenberg’s music was more often defended than listened to — music written by a theorist for an audience of theorists. I am a true fan of Schoenberg’s, though I can hear that the example outputs of this work are music that is meant to be accessible to the everyday listener.

oerpli 3y ago

Also pretty bad. Might get somewhere in a few years but currently this is only noise.

andrewstuart 3y ago

Can someone explain the significance of this and how I might use it?

pkaye 3y ago

That Riffusion output is causing me both laughter and pain. The pain is from laughing causing pressure of a surgical scar on my abdomen.

AxEy 3y ago

(This is not meant to be an anti-ai-generated-art rant. It's coming whether we like it or not. But some of the motives in this thread confuse me.) Music producer here with an honest question to those saying "this will provide me with a simple soundtrack/background music for $PROJECT" Have any of you checked out / made offers on music production subreddits? Or other music subreddits? various music production discords? Elsewhere on the internet? If so, could you say what your experience has been? I ask because the music production scene is like...ridiculously saturated, and it's almost a meme in the producer community how hard it is to make even a buck producing. I suspect that there are a significant number of producers who would be happy to take your "prompt" for a small fee. Yes, I understand 1) free and 2) immediate is convenient, but isn't 1) relatively inexpensive and 2) whatever advantage intent in construction gives good too? I'm willing to admit that I'm missing something here, but I'd love it if someone could enlighten me. While I'm asking follow ups, to all the folks who love digging for new music so much that they're considering turning to prompting AIs, I'd be seriously surprised if you've really checked out all the stuff that is coming out from new producers (again, reddit, soundcloud, etc). Another meme in the producer community is how one spends hundreds/thousands of hours perfecting ones craft, and dozens of hours working on a track, only for that track to get like 5 plays on soundcloud and negligible engagement elsewhere. Are music consumers really that desperate for new tunes? Frankly a lot of us just aren't seeing it....

comfypotato 3y ago
Free and immediate. You answered your own question. All things equal, people are happy to support local businesses. The value prop here is far from equal.
yieldcrv 3y ago
the simple answer is that your motivations for being an artist need to change to be exclusively personal fulfillment. because that was true for the pre-AI world, as you essentially described, and its true-er for this current-AI world. the real meme is about how artists have always been grasping for financial respect in every market condition ever, and yet nothing has changed. people were never going to commission you, they were never going to book you. While they do appreciate the content. But for the few that would ever actually try to commission something, they encountered friction after friction after friction and collectively artists have been disinterested in solving. Because they're starving and preoccupied with fighting for scraps and modicums of respect at all. The world’s has now solved many of these frictions. The frictions were: 1 hoping they found the right artist to begin with 2 hoping that artist is reliable and has any work ethic or structure in their life 3 not bruising that artists ego in however communication style is preferred 4 dealing with how completely segregated many artists are from contract negotiations and any aspect of the business world, but needing to secure rights properly 5 ego in securing rights properly without the artist overplaying their hand 6 waiting for the commission 7 revisions 8 circle back to 1 9 if you ever get past part 8, you have the issue of whether your new license can be used in an unforeseen way and medium in the future getting burned in altruistic commissions of living artists is simply over now. all these frictions are solved with the free and immediate way.

deleted 3y ago

[deleted]

speedgoose 3y ago

I installed it and everything went surprisingly fine and easily. It used about 8GB or VRAM max on a nvidia A30 and takes about 30s to generate 10s of audio. The max duration seems to be 30s in the frontend but the quality is a lot lower. Mixing genres do not really work and the model doesn’t seem to be trained on band names. However it does perform well to create music using existing styles. I generated some Eurovision crap and minimalist techno that were very much believable. But mixing death metal with lofi ambient isn’t the best, nor the epic progressive rock guitar solo I asked. I think the examples on the website are cherry picked but with some experience in prompt engineering and many tentatives, it should be possible to generate great samples. It’s also excellent at generating boards of Canada like music. The audio artéfacts, the low fidelity, the weird sounds, the detuned synths, this model does that very well and it does sound great to me. Thanks a lot to the authors.

solumunus 3y ago
> But mixing death metal with lofi ambient isn’t the best Would you expect that to be good if a human did it?
- squeaky-clean 3y ago
  (Not the original commenter) I don't know of any death metal examples, but chill-lofi-beats+djent music already exists and it's pretty good. This isn't the only example but it's my personal favorite one: https://youtu.be/-Yeqp2-dF5M https://youtu.be/-Yeqp2-dF5M
  - solumunus 3y ago
    I don't think you can have death metal without double bass hammering. I don't think you can have lofi with double bass hammering.
- speedgoose 3y ago
  Depends on the human. If it’s myself, not really. But if it’s well done, yes I would enjoy death metal sonorities in lofi ambiant once in a while. But I admit the prompt is perhaps a bit challenging.

deleted 3y ago

[deleted]

beefman 3y ago

> a piano and cello duet playing a sad chambers music No such thing as chambers music or a music. Maybe "a sad piano and cello duet" or "sad piano and cello chamber music" would be a better prompt.

rifty 3y ago

Style transfer is pretty cool tool as someone who likes to play around with sound design. It will be a lot of fun to drop this on parallel channels and blend it together into choruses and new instruments. Right now the generation still sounds a lot like loop packs smashed together anyone could technically make. But it is practical for anyone who really only cares for that style of sound but do not themselves have the familiarity to do it. Now they can just say what they want and hit regenerate, skipping the latent feedback cycle of iterating with humans or sifting through song snippets. My opinion on this style of content is that ai generation is simply accelerating us to the inevitable end of generic digital content, it isn't really changing it. It just happens to be also the optimal interface for discovering and not just generation.

moffkalast 3y ago

> Audiocraft requires Python 3.9, PyTorch 2.0.0, and a GPU with at least 16 GB of memory I sleep. And these are only 1-3.3B param models, that makes no sense.

cma 3y ago

They messed up and shipped Bolero with it in the repository, which is copyrighted until 2025 in the US > In the U.S., Boléro remains under copyright until 1 January 2025 https://en.wikipedia.org/wiki/Bol%C3%A9ro https://en.wikipedia.org/wiki/Bol%C3%A9ro

Kiro 3y ago

I'm curious how this handles bass music where the lower frequencies need to be tightly mixed to sound good. Is that really something that a model can learn?

iamsaitam 3y ago

The question that I think tells the kind of potential that ML has for music is, how good is written language in describing music? People here compare this to the development of Stable Diffusion, but the mediums are quite different. Describe your favorite shot in a movie and most likely you’ll enumerate and get pretty far in conveying it to someone who hasn’t seen it, at least the physical aspect of it. But try describing your favorite song and the interpretations will be wildly different (outside of hard defined genre music).

siquick 3y ago
“Writing about music is like dancing about architecture.”
raincole 3y ago
1. MusicGen accepts melody prompt. 2. It doesn't have an "interrogation" feature for now (unless I missed it), but I'm sure something like this is possible.
zavertnik 3y ago
> The question that I think tells the kind of potential that ML has for music is, how good is written language in describing music? If you took a DAW project file of a song.wav that was completely written and produced digitally using virtual instruments and compiled all of the parameters a user had to set to achieve their output.wav into a .csv file, you may be surprised to see (1) how few parameters were used (2) how often those parameters are unchanged from their defaults and (3) the amount of those parameters that would be expected in any other project file. When you break it down, you really only have 6 layers to parse, all of which are dynamic but within a relatively small and consistent sandbox, at least relative to image generation. 1. composition layer - the midi or notes of the song. 2. arrangement layer - the selection of instruments used in the song and the division of the song's midi to the song's respective instruments. 3. instrument layer - the parameters of each instrument, such as a synth path or a virtual piano's room setting. 4. post processing layer - the effects placed on the output of each instrument, such as reverb, compression, delay, ect. 5. mixing layer - the volume of each instrument + post processing channel 6. mastering layer - processing on the master track All of these things are more or less standardized. Developers always add their own flair (read: custom parameters) for their plugins, but they can be decompiled to be a composition of each of these layer's fundamental parameters. All these parameters + the midi of a song would be a few kb I feel like a LLM trained on the parameter sets which interacts with the software used to manipulate these layers could really produce amazing tools and open the door to writing high quality songs to everyone, just as other AI products have opened so many similar doors. The DALL-E app for music, in my mind, probably wont be a text based description -> .wav output. Instead, it would be the generation of the elements of each layer with options that can be listened to in real time using whatever VSTs were used in training. When you ask ChatGPT to write a complex python script, it starts with an outline of all the methods in the script as placeholders and then takes you step by step until you're done, then you troubleshoot it or flesh it out. The best part of a generative music like this is that it leaves the user really only with having to decide if something sounds good or not. As a mostly musically illiterate producer myself, I've produced hundreds of songs and a few albums without ever really learning how to do anything other than manipulate the parameters. When I started learning to produce music I was 15 years old and knew nothing about music production. But, I was really good at was using computers and software so I learned to play the DAW, the plugins, and the sample packs. The only layer that I couldn't learn through learning software was the composition, the writing of the midi. Fortunately, the midi of a song becomes very easy to brute force over time, so I learned to brute force midi. Once I became efficient with my workflow, producing music became a task of "make this idea sound good." And without ever really feeling like I was a musician or composer, this became an enormous passion and outlet for me that I did every day for a decade. I was able to do this because at its core, all the mechanical parts of a song are simple machines and a song's quality is the way those machines are used together. As an outside, this feels like a workflow that would be very machine learning friendly. But I could be wrong!

fintechie 3y ago

Do any of these generative music ML framework export as midi instead of .wav or .mp3? That would be 1000x times more useful as the quality we're reaching is good enough. I can imagine an VSTi that just takes a prompt and generates midi tracks. Something like this is surely coming in the next couple of years.

mk_stjames 3y ago
That was the way almost all ML research on music was done until recently- train for MIDi generation with input of other MIDI, or, at an even more fundamental level, just notes. If you go back, tons of ML music papers were written on generating believable sequences of Bach, Mozart, etc. because it was just note prediction. Since the advent of transformers, and this idea of using text models mapping the natural language space to tagged music samples, and the music tokenizer acting directly on sampled audio stream (the bits of a .wav file, essentially) all the cutting edge work is going that route. Because it is producing high quality, finished audio streams directly. And I think part of it is because there is way, way more training data for actual audio than there is for MIDI alone (there are tons of free midi sites out there but a lot of it is garbage and it pales in comparison to what is already sampled and tagged in real audio libraries). I imagine what will happen is... within two to three years these LM-transformer-music models will get so good that the audio will be damn near spotless sounding, and there will be additional methods developed to synthesize with more control directly with the models, to the point where wanting MIDI so you can use your own HQ synth isn't needed, because if you want "the lead synth to sound less digital and more like a classic Minimoog Model D" you just add that to another 'Music2Music" pass and out pops your sound. For those who still want MIDI there is still work being done on traditional audio-to-MIDI modeling and I think you'd wind up just using that in the chain.

IAmGraydon 3y ago

I’m a musician and have spent over 20 years making music. It’s my biggest passion. I still love this. Music isn’t a competition to me so it doesn’t bother me that someone with little experience can create great sounding music. I can still make music that’s an expression of myself, or I can generate some AI music to see if something catches my ear as a starting point. It’s just another tool, and I’m excited to see how it evolves.

bionhoward 3y ago

Typo “our a paper” if the devs are watching you can remove “a”

8bitsrule 3y ago

I'd get off any elevator playing anything on that page.

kennyloginz 3y ago

All the samples on that page sound pretty bland , or have nasty phase distortions. I guess that’s the point.

MicolashKyoka 3y ago

Amazing work by FAIR! With tools like these, AI-enhanced creativity is going to give us hyper specialized output and thankfully further us even more from mainstream, one-size-fits-all art. We are living in the future!

donbrae 3y ago

These are really impressive. Providing the ‘prompter-composer’ can generate high quality, stereo mixes — these all appear to be low bitrate, mono files — you could probably use them right now for various ‘wallpaper music’ purposes, where originality is not required or is even a detriment: ‘on hold’ music, low budget video productions, symphonic film scores etc. Producing this kind of anti-music is pretty soul-destroying for a musician, anyway, so the machines might as well do it. We can then spend our time working on stuff that means something, and if we’re lucky and it connects with enough people, make a living from it.

two_in_one 3y ago

Before using just listen the samples. It doesn't have any structure even within 30 sec. It's repetitive and boring. Can be used as a background in low budget game, but that's it. It's not a threat to human talents, not yet. It's like first text generators with tiny attention windows, took some time to evolve into GPT-4. Then it will have an impact. Bans and unions may delay that for some time.