8 ms·

Transformers in music recommendation

211 points by panarky 2y ago

naltroc 2y ago
when did google get a TLD
- Zambyte 2y ago
  A decade ago https://en.wikipedia.org/wiki/.google https://en.wikipedia.org/wiki/.google
- incognito124 2y ago
  dns.google has been with us for a long time
  - warkdarrior 2y ago
    Shouldn't that be dns.squarespace now?
janalsncm 2y ago
Other than stating there was one, they didn’t show a benefit of this over something like a Wide and Deep model, DCNv2 model, or even a vanilla NN. Transformers make sense if you need to use something N items ago as context (as in text) where N is large. But in their example, any model which takes the last ~5 or so interactions should be able to quickly understand contextual user preferences. A transformer may also be larger than their baseline, but you still need to justify how those parameters are allocated.
- hatthew 2y ago
  Empirically, I have found that user action sequences are a good way to model user behavior since it can look at several different scales, and specific behaviors. Interest tracking can see what a user generally likes, and the last few actions can help the model see what the user is listening to right now. But with a full sequence, you can start to model things like what the user is listening to right now, what they've been listening to recently, what they tend to listen to at this time of day, how much of a change in genre they could enjoy, etc.
- Fripplebubby 2y ago
  To be fair, this is just a blog post, it's not a peer-reviewed scientific paper. You don't really _need_ to do anything.
disposition2 2y ago
It’s interesting the amount of research listed in the article and IMHO the recommendation engine/ algorithm used by Rdio in the late aughts and early 2010s eclipses anything I’ve encountered to date. Seems like folks are reinventing the wheel, and trying to deduce what folks want to engage in with data and “AI”, rather than providing sufficient tools to allow the user to drive the narrative.
- motoxpro 2y ago
  Exploit is easy. It’s the explore part that’s hard. I.e. recommending me something i never knew i liked. Pandora and Rdio and others solved the exploit problems years and years ago.
- TJSomething 2y ago
  The problem is that Rdio's was based on Echo Nest's similarity algorithm, which went private after Spotify bought Echo Nest. Doing music similarity with Echo Nest was great back when it was public. I did a project in grad school with it.
- parpfish 2y ago
  any time there's a music recommender thread, there will be comments lamenting old algos like Rdio or play.fm it's interesting how this continues a trend in across music/audio tech, such as hipsters insisting "i liked the earlier work better" or audiophiles obsessing over amps from the 1970s.
tulsidas 2y ago
It's all very nice but if they end up "altering" the results heavily to play you the music they want you to listen for X or Y reason then it's pointless. I would like to be able to run this model myself and have a pristine and unbiased output of suggestions
- vagabund 2y ago
  It may just be my perception, but I seem to have noticed this steering becoming a lot more heavy handed on Spotify. If I try to play any music from a historical genre, it's only about 3 or 4 autoplays before it's queued exclusively contemporary artists, usually performing a cheap pastiche of the original style. It's honestly made the algorithm unusable, to the point that I built a CLI tool that lets me get recommendations from Claude conversationally, and adds them to my queue via api. It's limited by Claude's relatively shallow ability to retrieve from the vast library on these streaming services, but it's still better than the alternative. Hoping someone makes a model specifically for conversational music DJing, it's really pretty magical when it's working well.
  - ThrowawayTestr 2y ago
    Spotify's recommendations are biased towards what you've listened to recently. Do you share the account with someone else?
    - vagabund 2y ago
      No, but it's also biased toward their commercial partners. From this page [0], detailing their recommendation process: > How do commercial considerations impact recommendations? > [...] In some cases, commercial considerations, such as the cost of content or whether we can monetize it, may influence our recommendations. For example, Discovery Mode gives artists and labels the opportunity to identify songs that are a priority for them, and our system will add that signal to the algorithms that determine the content of personalized listening sessions. When an artist or label turns on Discovery Mode for a song, Spotify charges a commission on streams of that song in areas of the platform where Discovery Mode is active. So Spotify's incentivized to coerce listening behavior towards contemporary artists that vaguely match your tastes, so they can collect the commission. This explains why it's essentially impossible to keep the algorithm in a historical era or genre -- even if well defined, and seeded with a playlist full of songs that fit the definition. It also explains why the "shuffle" button now defaults to "smart shuffle" so they can insert "recommended" (read: commission-generating) songs into your playlist. [0]: https://www.spotify.com/ca-en/safetyandprivacy/understanding-recommendations https://www.spotify.com/ca-en/safetyandprivacy/understanding...
atum47 2y ago
All this research to create an apparently awesome recommendation system only for the sales department forces the recommendation of what they want to promote.
drdaeman 2y ago
It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced, but it mentions that user-action signals are of the same length, which makes me doubt track embeddings have any content-derived (rather than metadata-derived) information. Maybe I'm wrong, of course. I doubt that any recommendation system is capable of providing meaningful results in absence of the "awareness" about the actual content (be it music, books, movies or anything else) of what it's meant to recommend. It's like a deaf DJ that uses the charts data to decide what to play, guessing and incorporating listeners' profiles/wishes. It's better than a deaf DJ who just picks whatever's popular without any context (or going by genre only), but it's not exactly what one looks forward to when looking for a recommendation.
- acchow 2y ago
  > It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced That's the thing with transformers, right? It doesn't actually "know" anything about its inputs. The embeddings are learned (initialized to random).
- TJSomething 2y ago
  This is a shortcoming of every music recommendation algorithm except Spotify and Pandora's. Spotify holds holds a pretty hefty patent portfolio of music classification algorithms and Pandora employs hundreds of music experts that spend an hour tagging each song.
  - antifa 2y ago
    Spotify's Discover Weekly seems to be a healthy mix of close enough guesses and random but not too random suggestions. Song radio is okay. The pop up recommendation for specific new songs/ albums feels so unrelated to my likes that it must be a sponsored recommendation. """Smart""" shuffle exclusively sends me whatever was popular on the radio in the last 5-30 years despite my listening habits being the opposite. Pandora was much smarter, but seemed to run out of songs instantly.
- 2y ago
dr0p 2y ago
[flagged]
- deleted 2y ago
  [deleted]
next_xibalba 2y ago
I really wish someone would do this for books.
- KetoManx64 2y ago
  Ask chatGPT to recommend you a few books based on the books you've enjoyed, I've found some great books this way
  - next_xibalba 2y ago
    Tried that. It's been a while, but when I last did it, ChatGPT hallucinated non-existent books.
- darthShadow 2y ago
  Hardcover may be worth a try: https://hardcover.app/askjules https://hardcover.app/askjules
jcgr 2y ago
it seems that suggesting media will always be limited unless there's more context given to the user's situation via device or external apis..
hkon 2y ago
If it gets me off the never ending Finnish rap cycle, I will take it.
m3kw9 2y ago
Music recommendations algorithms are a fools errand. If you look at Spotify, they do a good job initially but it gets really boring after a while. All algorithms or AI tries and usually fails to guess how you are feeling at that moment, maybe there was a trigger for me to hear a specific type of music, or maybe I suddenly feel nostalgic and want to hear 90s music instead of my favourite modern electronic which I skipped a whole tone if it comes up.
- DonaldFisk 2y ago
  You could still use collaborative filtering, except with users giving different ratings to the same songs depending on their mood. This approach was used in a movie recommendation engine (I forget which one).
  - m3kw9 2y ago
    Yeah, this would get what songs a user that has similar tastes like, but many times is based on an individuals exact mood at the moment.
tunesmith 2y ago
I look at my own music tastes as roughly two levels. One level is whether I objectively like the music. The other level is what I'm in the mood to listen to. I can definitely not be in the mood to listen to some music I really enjoy. But there is also music I will never enjoy no matter the mood. I don't think a recommendation system will work very well if it conflates those two levels.
- eleveriven 2y ago
  Wouldn't it be nice to have recommendations that effectively address both levels and understanding not just your general preferences but also your current emotional state
- 2muchcoffeeman 2y ago
  Or if your tastes change. If it recommended music to me based on patterns, it would periodical be completely wrong.
markl42 2y ago
Has any streaming service trained a model to actually understand the music itself to work out what other songs would be of a similar vibe/genre? My favorite band (vulfpeck, and more recently jack's solo stuff) often branch out into different genres, and it's a bit of whiplash when it goes to another song just because the artists are similar / appear together in other places.
- sundalia 2y ago
  AFAIK all efforts in that direction were way too costly a few years back and degraded models considerably. Spotify, for the longest time, only trained on an equivalent of manually curated playlists by experts and users to understand similarity.
- dathery 2y ago
  Not a traditional streaming service, but Plex offers sonic analysis: https://support.plex.tv/articles/sonic-analysis-music/ https://support.plex.tv/articles/sonic-analysis-music/ > Plex Media Server uses a sophisticated neural network to analyze each track in the music library, cataloging a wide variety of characteristics of the track. Think of it as things like female vs male, vocals vs not, sad, happy, rock, rap, etc. All these various characteristic constitute a “Musical Universe” and the server is determining where that particular track exists within it. > For the math-savvy, the Musical Universe consists of points in N-dimensional space. But what’s important is that this allows us to see how “close” anything in your library is from anything else, where distance is based on a large number of sonic elements in the audio. I haven't tried it so can't speak to its effectiveness.
- nevernothing 2y ago
  ChatGPT is pretty good with this, you can try it yourself. I created a playlist generator for YouTube a while ago. It is powered by GPT-3.5 Turbo and can create playlists based on text descriptions: https://playlists.at/youtube/generate/ https://playlists.at/youtube/generate/
  - visarga 2y ago
    I used gpt-4 to generate ideas what to listen to. I just say "I like X, Y and Z" and it gives me interesting, motivated choices. No special recommender transformer, just the plain text one.
mianos 2y ago
It would be good if they started using this. I moved to spotify because youtube music kept adding songs I thumbs-downed to my playlists. They even added songs I hated to my seasonal mixtapes. If I didn't pay youtube for no ads and creators I'd never give these idiots a cent.
- eleveriven 2y ago
  Wow, I think it's really annoying to see music that you disliked on your recommendation list
afro88 2y ago
> Using transformers to incorporate different user actions based on the current user context helps steer music recommendations directly towards the user’s current need What if the user's current need is to not play music? To not consume yet more content? To not make them addicted to the content application? How can we optimize for user wellbeing, and still make money? That's the question we should be pouring resources into
- chrismartin 2y ago
  I often make the same criticism about services like this, but personally, discovering good new music greatly enriches my life. Sometimes I'll bounce off of a given artist several times, over years or decades, until the right track catches me when I'm "ready" for it, and then I'll enjoy discovering their whole catalog. At that point, the affinity is durable. I would love to find whoever is my next (e.g.) Steely Dan.
  - afro88 2y ago
    Yeah, me too. But this being google, and them talking about youtube... you know where this tech is going next.
- vasco 2y ago
  > What if the user's current need is to not play music? Then don't play music. You are asking for something that only works in a dystopia. Either the machine tries to understand "what I need now" from partial information and will not play me music when I want to listen because it thinks it "knows what I need". Or the machine actually is hooked up to my real time health data and possibly brain activity to actually know what I need. I definitely don't want to have a personal computing machine thinking it knows what I want and deciding for me in such a way, or to have such access to my internal state.
  - afro88 2y ago
    You missed my point a little. We keep optimizing for more addictive services. It's just as dystopian for a service to always present exactly what will keep me using the service and giving thumbs up.
deleted 2y ago
[deleted]
cokeandpepsi 2y ago
[dead]
visarga 2y ago
Hopefully it sees the light of day, because too many interesting Google papers remain just that, papers.
sova 2y ago
YouTube just "infers the context" meaning it uses some sort of tracking to realize you're at the gym versus "just chilling." But what's really chilling is that they don't mention location data or how they get it anywhere in this post.
amjoshuamichael 2y ago
I sometimes wonder if we're looking in the wrong place with music recommendation systems. I've tried both Apple Music and Spotify; it's rare that I hear a song come through on the linear recommendation stream and think to myself "oh my gosh! that's exactly what I wanted!" For me, discovering new music is a branching experience, where I'm constantly listening to little bits of different things, figuring out what I like, and then looking online on forums and blogs to see what's similar to that. It's surprising that the company that owns YouTube, a platform driven by user choice and 'rabbit-hole discovery', would be looking for a new way to feed users linear song recommendations. I would much rather be able to see several 'similar songs' while listening to something, similar to YouTube's recommendation tab. Alas, no streaming service seems to have implemented this (not even YouTube music, afaik). https://cosine.club/ https://cosine.club/ is the closest I've seen to the ideal branching system. My understanding is that it uses vector embeddings to search for songs that are similar in sound, and it works shockingly well for that purpose. However, it has a limited song database. Also see https://everynoise.com/ https://everynoise.com/, which is no longer updated. These use vector embeddings in similar ways, but the exploration experience is controlled by the user, not by a list-generating ranking model. I definitely think that AI-tech is the future of music recommendation, but I would prefer to see more research by large companies in to these user-driven systems, instead of the 'similar autosuggested list', which is, by its very nature, only ever 'good enough.'
- jillesvangurp 2y ago
  The best recommendations I get on Spotify are usually via their users that listened to artist X, also listened to artist Y type recommendations. That combined with their list of most popular tracks per artist gives me a rich source of new things to listen to. Their regular recommendations aren't great; it falls into the same "more of the same shit" trap that most other recommendation systems fall into. The reason this simple mechanism works so well is that it gets rid of personal biases and instead taps into a community of listeners listening to the same stuff. Confirmation bias is the core issue here. I don't want confirmation bias. I want my biases challenged with new things. Not randomly new but based on what others are listening to that listen to similar things. And not just randomly based on everything I listen to but on specific things that I'm playing. Vector similarity of artists could be an interesting angle. But it would probably risk pulling out a lot of cover bands and imitators. You want stuff that is close but not too close.
ferbncode 2y ago
I think the only thing that works for me properly is listenbrainz.org for recommendations. They give similar users, good playlists. Plus they have bunch of other interesting projects like acousticbrainz.org which do even low-level analysis and high-level classification.
- ferbncode 2y ago
  Fixing links: https://listenbrainz.org https://listenbrainz.org and https://acousticbrainz.org https://acousticbrainz.org. Also, all code is open-source in https://github.com/metabrainz https://github.com/metabrainz!
laserbeam 2y ago
Spotify's music recommendation is great not because it's doing smart AI things, but because it gives you options for discovery. You have: - Playlists provided by spotify and tagged with various terms. - User-made playlists (harder to find, but they exist, and many of them are fun!) - Discover Weekly, Release Radar, Daily mixes - generated playlists for your account with 6-7 variations on how they are biased. - Social features (see what your friends are listening to) - Radios for songs/albums/artists - Artist bios and "Fans also like" sections for each artist - Smart shuffle on playlists (every 3rd song is a recommendation) - A very permissive search box that lets you make mistakes (I hate competitors that punish me for writing things like "and" instead of "&" for an artist's name... I'm looking at you Tidal) - Configurable search APIs to build your own funky queries or extensions ... and probably a few more tools to use when discovering music. An AI recommendation system will NEVER beat flexibility and giving users agency. I don't care that an AI takes into consideration that I went to the gym automatically. I care about having lots of options for discovery, each that is decent and biased towards a different style of exploration.
eimrine 2y ago
All megacorp recommendation algos are shit just because they always binded to some media library. Get yourself on any torrent website with lots of music releases and you will unleash what is the meaning of "musical taste". I recommend to have at least 10Tb disk drive to have no reasons to delete any releases.
- layer8 2y ago
  YouTube Music is an exception regarding “some media library”, because basically everything is on YouTube thanks to user uploads.
  - eimrine 2y ago
    Youtube Music have no lossless, its "tracks" have no metadata and are stored in pesky format with a lot of stupid limitations about receiving them. Seems you just do not know what is a good music and how it even looks on your filesystem. And good luck about acknowledging (not even receiving) every existing recording of your favorite artist; discography.
    - layer8 2y ago
      It's perfectly fine for streaming and getting music recommendations, which is the topic of this thread. For the local music library I then purchase lossless versions of the tracks I like.
l72 2y ago
Does anyone know if there is something similar to pandora but for your local music collection? I typically listen to full albums, but when I am working out or doing a road trip, having playlists for specific moods would be really nice.
jiwidi 2y ago
This research isnt very novel either. Transformers for sequential event handling and recommendations is a known thing. Pretty sure this is what spotify uses on their autocomplete shuffle playlist (fill in songs between your songs, similar to how autocompleting text would be)
dr_dshiv 2y ago
We had an informal college group “music appreciation” that would meet every Wednesday to listen to 2 full albums—-each related to the other in some manner. The only rule was “no talking while the music played.” It lasted four years and I miss it!
ruuda 2y ago
I tried to make embeddings from my personal listening history a few years ago (https://github.com/ruuda/deepnote https://github.com/ruuda/deepnote) with the idea that listens of music that matches well would be played in the same session, basically word2vec applied to listening history. It didn't work very well, it mostly found that tracks on albums are similar because I tend to listen to full albums. Maybe I didn't stir the parameters enough, or maybe it needs far more data. (All of Listenbrainz?) I also still want to experiment with generating embeddings for tracks just from the time of the day, week, and year. (I tend to listen to different things on a Friday night vs. Sunday morning, and I listen to very different music in summer vs. winter.) But that's for locally recommending relevant tracks that you already have in your library (for https://github.com/ruuda/musium https://github.com/ruuda/musium), not for discovering new music. I already implemented a sorting mode to "rediscover" albums (albums that have a lot of listens in the past but few recently), and it works reasonably well. I expect that adding time of day/week/year will improve this a lot, but I haven't implemented it yet. I wonder how much of an improvement a transformer like in the article adds on top of that.
layer8 2y ago
IMO it’s a mistake to try to draw any conclusions from skip actions or from what a user is listening to, as opposed to from explicit like/dislike actions. There are just too many reasons why someone may be skipping a track or (appear to) be listening to tracks. As a user, I don’t want to be in the position of having to fear that the algorithm will misinterpret my skipping or my letting some playlist play.
gavin_gee 2y ago
I suspect there is no available dataset that can teach the models to learn this. The enjoyment of music is entirely internalized. Skipping a track is an incredibly low fidelity data point. I've often wondered if Spotify could capture volume control data as part of a track to see if that produces better training data. But again its still too low fidelity.
comprev 2y ago
Virtual crate digging on Discogs.com is still by far the best way to discover music. Artists, record labels, producers, technicians (mastering, vinyl cutting), distribution channels, etc. are all there just a click away. Who did/does you favourite artist share a label with? Who else used the same recording studio? What other aliases does an artist use? What other bands/groups have they been in? Combined with the embedded YouTube player on the release page it's a gold mine. Every week I'll spend hours down the Discogs rabbit hole - often adding missing YouTube videos to contribute something for future visitors.
taosx 2y ago
I haven't found any music recommendations services that really find what I'm looking for. What I'm looking for is usually something without high pitched noises, rythmic and a bit of novelty based on old patterns. I've tried my playlist on both Spotify, apple and YouTube music but they only find a new song I like 1 in 10 Times.
fallinditch 2y ago
We are living through the best ever time to be a music lover. With so much good music being made and added to the existing body of recordings, together with the great (albeit commercially compromised) recommendation systems, it's just fantastic. Algos are useful for finding and playing music but they are just a tool. The human touch is needed to create the best musical experiences. That's why we need DJs now more than ever. To experience the best music, for you, you're going to have to do a bit of work to search out new sounds and old sounds. So go digging and create awesome playlists, for you, and share them: they'll probably be better than algo-created playlists. Algorithmic playlist systems are great but I believe we should counteract their prevalence by also supporting human efforts: by also listening to online radio stations and DJ mixes, supporting music journalism and publications (as well as buying physical media and merch, going to gigs, of course!) I don't understand exactly what this Google paper is describing but it sounds like it's going to anticipate whether I want to listen to up beat or down beat music. What a very googleish thing to do! - this doesn't help me at all. Also I'm turned off by it's opening statement: >We present a music recommendation ranking system that uses Transformer models to better understand the sequential nature of user actions based on the current user context. This comes across as nonsensical gobbledygook, but also a bit dystopian.

211 points by panarky 2y ago

naltroc 2y ago

when did google get a TLD

Zambyte 2y ago
A decade ago https://en.wikipedia.org/wiki/.google https://en.wikipedia.org/wiki/.google
incognito124 2y ago
dns.google has been with us for a long time
- warkdarrior 2y ago
  Shouldn't that be dns.squarespace now?

janalsncm 2y ago

Other than stating there was one, they didn’t show a benefit of this over something like a Wide and Deep model, DCNv2 model, or even a vanilla NN. Transformers make sense if you need to use something N items ago as context (as in text) where N is large. But in their example, any model which takes the last ~5 or so interactions should be able to quickly understand contextual user preferences. A transformer may also be larger than their baseline, but you still need to justify how those parameters are allocated.

hatthew 2y ago
Empirically, I have found that user action sequences are a good way to model user behavior since it can look at several different scales, and specific behaviors. Interest tracking can see what a user generally likes, and the last few actions can help the model see what the user is listening to right now. But with a full sequence, you can start to model things like what the user is listening to right now, what they've been listening to recently, what they tend to listen to at this time of day, how much of a change in genre they could enjoy, etc.
Fripplebubby 2y ago
To be fair, this is just a blog post, it's not a peer-reviewed scientific paper. You don't really _need_ to do anything.

disposition2 2y ago

It’s interesting the amount of research listed in the article and IMHO the recommendation engine/ algorithm used by Rdio in the late aughts and early 2010s eclipses anything I’ve encountered to date. Seems like folks are reinventing the wheel, and trying to deduce what folks want to engage in with data and “AI”, rather than providing sufficient tools to allow the user to drive the narrative.

motoxpro 2y ago
Exploit is easy. It’s the explore part that’s hard. I.e. recommending me something i never knew i liked. Pandora and Rdio and others solved the exploit problems years and years ago.
TJSomething 2y ago
The problem is that Rdio's was based on Echo Nest's similarity algorithm, which went private after Spotify bought Echo Nest. Doing music similarity with Echo Nest was great back when it was public. I did a project in grad school with it.
parpfish 2y ago
any time there's a music recommender thread, there will be comments lamenting old algos like Rdio or play.fm it's interesting how this continues a trend in across music/audio tech, such as hipsters insisting "i liked the earlier work better" or audiophiles obsessing over amps from the 1970s.

tulsidas 2y ago

It's all very nice but if they end up "altering" the results heavily to play you the music they want you to listen for X or Y reason then it's pointless. I would like to be able to run this model myself and have a pristine and unbiased output of suggestions

vagabund 2y ago
It may just be my perception, but I seem to have noticed this steering becoming a lot more heavy handed on Spotify. If I try to play any music from a historical genre, it's only about 3 or 4 autoplays before it's queued exclusively contemporary artists, usually performing a cheap pastiche of the original style. It's honestly made the algorithm unusable, to the point that I built a CLI tool that lets me get recommendations from Claude conversationally, and adds them to my queue via api. It's limited by Claude's relatively shallow ability to retrieve from the vast library on these streaming services, but it's still better than the alternative. Hoping someone makes a model specifically for conversational music DJing, it's really pretty magical when it's working well.
- ThrowawayTestr 2y ago
  Spotify's recommendations are biased towards what you've listened to recently. Do you share the account with someone else?
  - vagabund 2y ago
    No, but it's also biased toward their commercial partners. From this page [0], detailing their recommendation process: > How do commercial considerations impact recommendations? > [...] In some cases, commercial considerations, such as the cost of content or whether we can monetize it, may influence our recommendations. For example, Discovery Mode gives artists and labels the opportunity to identify songs that are a priority for them, and our system will add that signal to the algorithms that determine the content of personalized listening sessions. When an artist or label turns on Discovery Mode for a song, Spotify charges a commission on streams of that song in areas of the platform where Discovery Mode is active. So Spotify's incentivized to coerce listening behavior towards contemporary artists that vaguely match your tastes, so they can collect the commission. This explains why it's essentially impossible to keep the algorithm in a historical era or genre -- even if well defined, and seeded with a playlist full of songs that fit the definition. It also explains why the "shuffle" button now defaults to "smart shuffle" so they can insert "recommended" (read: commission-generating) songs into your playlist. [0]: https://www.spotify.com/ca-en/safetyandprivacy/understanding-recommendations https://www.spotify.com/ca-en/safetyandprivacy/understanding...

atum47 2y ago

All this research to create an apparently awesome recommendation system only for the sales department forces the recommendation of what they want to promote.

drdaeman 2y ago

It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced, but it mentions that user-action signals are of the same length, which makes me doubt track embeddings have any content-derived (rather than metadata-derived) information. Maybe I'm wrong, of course. I doubt that any recommendation system is capable of providing meaningful results in absence of the "awareness" about the actual content (be it music, books, movies or anything else) of what it's meant to recommend. It's like a deaf DJ that uses the charts data to decide what to play, guessing and incorporating listeners' profiles/wishes. It's better than a deaf DJ who just picks whatever's popular without any context (or going by genre only), but it's not exactly what one looks forward to when looking for a recommendation.

acchow 2y ago
> It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced That's the thing with transformers, right? It doesn't actually "know" anything about its inputs. The embeddings are learned (initialized to random).
TJSomething 2y ago
This is a shortcoming of every music recommendation algorithm except Spotify and Pandora's. Spotify holds holds a pretty hefty patent portfolio of music classification algorithms and Pandora employs hundreds of music experts that spend an hour tagging each song.
- antifa 2y ago
  Spotify's Discover Weekly seems to be a healthy mix of close enough guesses and random but not too random suggestions. Song radio is okay. The pop up recommendation for specific new songs/ albums feels so unrelated to my likes that it must be a sponsored recommendation. """Smart""" shuffle exclusively sends me whatever was popular on the radio in the last 5-30 years despite my listening habits being the opposite. Pandora was much smarter, but seemed to run out of songs instantly.
2y ago

dr0p 2y ago

[flagged]

deleted 2y ago
[deleted]

next_xibalba 2y ago

I really wish someone would do this for books.

KetoManx64 2y ago
Ask chatGPT to recommend you a few books based on the books you've enjoyed, I've found some great books this way
- next_xibalba 2y ago
  Tried that. It's been a while, but when I last did it, ChatGPT hallucinated non-existent books.
darthShadow 2y ago
Hardcover may be worth a try: https://hardcover.app/askjules https://hardcover.app/askjules

jcgr 2y ago

it seems that suggesting media will always be limited unless there's more context given to the user's situation via device or external apis..

hkon 2y ago

If it gets me off the never ending Finnish rap cycle, I will take it.

m3kw9 2y ago

Music recommendations algorithms are a fools errand. If you look at Spotify, they do a good job initially but it gets really boring after a while. All algorithms or AI tries and usually fails to guess how you are feeling at that moment, maybe there was a trigger for me to hear a specific type of music, or maybe I suddenly feel nostalgic and want to hear 90s music instead of my favourite modern electronic which I skipped a whole tone if it comes up.

DonaldFisk 2y ago
You could still use collaborative filtering, except with users giving different ratings to the same songs depending on their mood. This approach was used in a movie recommendation engine (I forget which one).
- m3kw9 2y ago
  Yeah, this would get what songs a user that has similar tastes like, but many times is based on an individuals exact mood at the moment.

tunesmith 2y ago

I look at my own music tastes as roughly two levels. One level is whether I objectively like the music. The other level is what I'm in the mood to listen to. I can definitely not be in the mood to listen to some music I really enjoy. But there is also music I will never enjoy no matter the mood. I don't think a recommendation system will work very well if it conflates those two levels.

eleveriven 2y ago
Wouldn't it be nice to have recommendations that effectively address both levels and understanding not just your general preferences but also your current emotional state
2muchcoffeeman 2y ago
Or if your tastes change. If it recommended music to me based on patterns, it would periodical be completely wrong.

markl42 2y ago

Has any streaming service trained a model to actually understand the music itself to work out what other songs would be of a similar vibe/genre? My favorite band (vulfpeck, and more recently jack's solo stuff) often branch out into different genres, and it's a bit of whiplash when it goes to another song just because the artists are similar / appear together in other places.

sundalia 2y ago
AFAIK all efforts in that direction were way too costly a few years back and degraded models considerably. Spotify, for the longest time, only trained on an equivalent of manually curated playlists by experts and users to understand similarity.
dathery 2y ago
Not a traditional streaming service, but Plex offers sonic analysis: https://support.plex.tv/articles/sonic-analysis-music/ https://support.plex.tv/articles/sonic-analysis-music/ > Plex Media Server uses a sophisticated neural network to analyze each track in the music library, cataloging a wide variety of characteristics of the track. Think of it as things like female vs male, vocals vs not, sad, happy, rock, rap, etc. All these various characteristic constitute a “Musical Universe” and the server is determining where that particular track exists within it. > For the math-savvy, the Musical Universe consists of points in N-dimensional space. But what’s important is that this allows us to see how “close” anything in your library is from anything else, where distance is based on a large number of sonic elements in the audio. I haven't tried it so can't speak to its effectiveness.
nevernothing 2y ago
ChatGPT is pretty good with this, you can try it yourself. I created a playlist generator for YouTube a while ago. It is powered by GPT-3.5 Turbo and can create playlists based on text descriptions: https://playlists.at/youtube/generate/ https://playlists.at/youtube/generate/
- visarga 2y ago
  I used gpt-4 to generate ideas what to listen to. I just say "I like X, Y and Z" and it gives me interesting, motivated choices. No special recommender transformer, just the plain text one.

mianos 2y ago

It would be good if they started using this. I moved to spotify because youtube music kept adding songs I thumbs-downed to my playlists. They even added songs I hated to my seasonal mixtapes. If I didn't pay youtube for no ads and creators I'd never give these idiots a cent.

eleveriven 2y ago
Wow, I think it's really annoying to see music that you disliked on your recommendation list

afro88 2y ago

> Using transformers to incorporate different user actions based on the current user context helps steer music recommendations directly towards the user’s current need What if the user's current need is to not play music? To not consume yet more content? To not make them addicted to the content application? How can we optimize for user wellbeing, and still make money? That's the question we should be pouring resources into

chrismartin 2y ago
I often make the same criticism about services like this, but personally, discovering good new music greatly enriches my life. Sometimes I'll bounce off of a given artist several times, over years or decades, until the right track catches me when I'm "ready" for it, and then I'll enjoy discovering their whole catalog. At that point, the affinity is durable. I would love to find whoever is my next (e.g.) Steely Dan.
- afro88 2y ago
  Yeah, me too. But this being google, and them talking about youtube... you know where this tech is going next.
vasco 2y ago
> What if the user's current need is to not play music? Then don't play music. You are asking for something that only works in a dystopia. Either the machine tries to understand "what I need now" from partial information and will not play me music when I want to listen because it thinks it "knows what I need". Or the machine actually is hooked up to my real time health data and possibly brain activity to actually know what I need. I definitely don't want to have a personal computing machine thinking it knows what I want and deciding for me in such a way, or to have such access to my internal state.
- afro88 2y ago
  You missed my point a little. We keep optimizing for more addictive services. It's just as dystopian for a service to always present exactly what will keep me using the service and giving thumbs up.

deleted 2y ago

[deleted]

cokeandpepsi 2y ago

[dead]

visarga 2y ago

Hopefully it sees the light of day, because too many interesting Google papers remain just that, papers.

sova 2y ago

YouTube just "infers the context" meaning it uses some sort of tracking to realize you're at the gym versus "just chilling." But what's really chilling is that they don't mention location data or how they get it anywhere in this post.

amjoshuamichael 2y ago

I sometimes wonder if we're looking in the wrong place with music recommendation systems. I've tried both Apple Music and Spotify; it's rare that I hear a song come through on the linear recommendation stream and think to myself "oh my gosh! that's exactly what I wanted!" For me, discovering new music is a branching experience, where I'm constantly listening to little bits of different things, figuring out what I like, and then looking online on forums and blogs to see what's similar to that. It's surprising that the company that owns YouTube, a platform driven by user choice and 'rabbit-hole discovery', would be looking for a new way to feed users linear song recommendations. I would much rather be able to see several 'similar songs' while listening to something, similar to YouTube's recommendation tab. Alas, no streaming service seems to have implemented this (not even YouTube music, afaik). https://cosine.club/ https://cosine.club/ is the closest I've seen to the ideal branching system. My understanding is that it uses vector embeddings to search for songs that are similar in sound, and it works shockingly well for that purpose. However, it has a limited song database. Also see https://everynoise.com/ https://everynoise.com/, which is no longer updated. These use vector embeddings in similar ways, but the exploration experience is controlled by the user, not by a list-generating ranking model. I definitely think that AI-tech is the future of music recommendation, but I would prefer to see more research by large companies in to these user-driven systems, instead of the 'similar autosuggested list', which is, by its very nature, only ever 'good enough.'

jillesvangurp 2y ago
The best recommendations I get on Spotify are usually via their users that listened to artist X, also listened to artist Y type recommendations. That combined with their list of most popular tracks per artist gives me a rich source of new things to listen to. Their regular recommendations aren't great; it falls into the same "more of the same shit" trap that most other recommendation systems fall into. The reason this simple mechanism works so well is that it gets rid of personal biases and instead taps into a community of listeners listening to the same stuff. Confirmation bias is the core issue here. I don't want confirmation bias. I want my biases challenged with new things. Not randomly new but based on what others are listening to that listen to similar things. And not just randomly based on everything I listen to but on specific things that I'm playing. Vector similarity of artists could be an interesting angle. But it would probably risk pulling out a lot of cover bands and imitators. You want stuff that is close but not too close.

ferbncode 2y ago

I think the only thing that works for me properly is listenbrainz.org for recommendations. They give similar users, good playlists. Plus they have bunch of other interesting projects like acousticbrainz.org which do even low-level analysis and high-level classification.

ferbncode 2y ago
Fixing links: https://listenbrainz.org https://listenbrainz.org and https://acousticbrainz.org https://acousticbrainz.org. Also, all code is open-source in https://github.com/metabrainz https://github.com/metabrainz!

laserbeam 2y ago

Spotify's music recommendation is great not because it's doing smart AI things, but because it gives you options for discovery. You have: - Playlists provided by spotify and tagged with various terms. - User-made playlists (harder to find, but they exist, and many of them are fun!) - Discover Weekly, Release Radar, Daily mixes - generated playlists for your account with 6-7 variations on how they are biased. - Social features (see what your friends are listening to) - Radios for songs/albums/artists - Artist bios and "Fans also like" sections for each artist - Smart shuffle on playlists (every 3rd song is a recommendation) - A very permissive search box that lets you make mistakes (I hate competitors that punish me for writing things like "and" instead of "&" for an artist's name... I'm looking at you Tidal) - Configurable search APIs to build your own funky queries or extensions ... and probably a few more tools to use when discovering music. An AI recommendation system will NEVER beat flexibility and giving users agency. I don't care that an AI takes into consideration that I went to the gym automatically. I care about having lots of options for discovery, each that is decent and biased towards a different style of exploration.

eimrine 2y ago

All megacorp recommendation algos are shit just because they always binded to some media library. Get yourself on any torrent website with lots of music releases and you will unleash what is the meaning of "musical taste". I recommend to have at least 10Tb disk drive to have no reasons to delete any releases.

layer8 2y ago
YouTube Music is an exception regarding “some media library”, because basically everything is on YouTube thanks to user uploads.
- eimrine 2y ago
  Youtube Music have no lossless, its "tracks" have no metadata and are stored in pesky format with a lot of stupid limitations about receiving them. Seems you just do not know what is a good music and how it even looks on your filesystem. And good luck about acknowledging (not even receiving) every existing recording of your favorite artist; discography.
  - layer8 2y ago
    It's perfectly fine for streaming and getting music recommendations, which is the topic of this thread. For the local music library I then purchase lossless versions of the tracks I like.

l72 2y ago

Does anyone know if there is something similar to pandora but for your local music collection? I typically listen to full albums, but when I am working out or doing a road trip, having playlists for specific moods would be really nice.

jiwidi 2y ago

This research isnt very novel either. Transformers for sequential event handling and recommendations is a known thing. Pretty sure this is what spotify uses on their autocomplete shuffle playlist (fill in songs between your songs, similar to how autocompleting text would be)

dr_dshiv 2y ago

We had an informal college group “music appreciation” that would meet every Wednesday to listen to 2 full albums—-each related to the other in some manner. The only rule was “no talking while the music played.” It lasted four years and I miss it!

ruuda 2y ago

I tried to make embeddings from my personal listening history a few years ago (https://github.com/ruuda/deepnote https://github.com/ruuda/deepnote) with the idea that listens of music that matches well would be played in the same session, basically word2vec applied to listening history. It didn't work very well, it mostly found that tracks on albums are similar because I tend to listen to full albums. Maybe I didn't stir the parameters enough, or maybe it needs far more data. (All of Listenbrainz?) I also still want to experiment with generating embeddings for tracks just from the time of the day, week, and year. (I tend to listen to different things on a Friday night vs. Sunday morning, and I listen to very different music in summer vs. winter.) But that's for locally recommending relevant tracks that you already have in your library (for https://github.com/ruuda/musium https://github.com/ruuda/musium), not for discovering new music. I already implemented a sorting mode to "rediscover" albums (albums that have a lot of listens in the past but few recently), and it works reasonably well. I expect that adding time of day/week/year will improve this a lot, but I haven't implemented it yet. I wonder how much of an improvement a transformer like in the article adds on top of that.

layer8 2y ago

IMO it’s a mistake to try to draw any conclusions from skip actions or from what a user is listening to, as opposed to from explicit like/dislike actions. There are just too many reasons why someone may be skipping a track or (appear to) be listening to tracks. As a user, I don’t want to be in the position of having to fear that the algorithm will misinterpret my skipping or my letting some playlist play.

gavin_gee 2y ago

I suspect there is no available dataset that can teach the models to learn this. The enjoyment of music is entirely internalized. Skipping a track is an incredibly low fidelity data point. I've often wondered if Spotify could capture volume control data as part of a track to see if that produces better training data. But again its still too low fidelity.

comprev 2y ago

Virtual crate digging on Discogs.com is still by far the best way to discover music. Artists, record labels, producers, technicians (mastering, vinyl cutting), distribution channels, etc. are all there just a click away. Who did/does you favourite artist share a label with? Who else used the same recording studio? What other aliases does an artist use? What other bands/groups have they been in? Combined with the embedded YouTube player on the release page it's a gold mine. Every week I'll spend hours down the Discogs rabbit hole - often adding missing YouTube videos to contribute something for future visitors.

taosx 2y ago

I haven't found any music recommendations services that really find what I'm looking for. What I'm looking for is usually something without high pitched noises, rythmic and a bit of novelty based on old patterns. I've tried my playlist on both Spotify, apple and YouTube music but they only find a new song I like 1 in 10 Times.

fallinditch 2y ago

We are living through the best ever time to be a music lover. With so much good music being made and added to the existing body of recordings, together with the great (albeit commercially compromised) recommendation systems, it's just fantastic. Algos are useful for finding and playing music but they are just a tool. The human touch is needed to create the best musical experiences. That's why we need DJs now more than ever. To experience the best music, for you, you're going to have to do a bit of work to search out new sounds and old sounds. So go digging and create awesome playlists, for you, and share them: they'll probably be better than algo-created playlists. Algorithmic playlist systems are great but I believe we should counteract their prevalence by also supporting human efforts: by also listening to online radio stations and DJ mixes, supporting music journalism and publications (as well as buying physical media and merch, going to gigs, of course!) I don't understand exactly what this Google paper is describing but it sounds like it's going to anticipate whether I want to listen to up beat or down beat music. What a very googleish thing to do! - this doesn't help me at all. Also I'm turned off by it's opening statement: >We present a music recommendation ranking system that uses Transformer models to better understand the sequential nature of user actions based on the current user context. This comes across as nonsensical gobbledygook, but also a bit dystopian.