12 ms·
Continuous low-power music recognition
- bhouston 9y agoIs this why Shazam sold? Its basic functionality was going to be built into Andriod/Google Assistant?
- lozenge 9y agoI think it was inevitable that this feature would get built into phone OSes and at that point nobody would want a separate app.
- jclardy 9y agoIt already is available on both the iPhone and Android through Siri/Google Assistant. Just ask them "What song is this?" This just seems to be an extension of it so you don't need to ask, the phone can just display the album info passively when you pick up the phone. Of course Apple's integration is already using Shazam for the actual identification, but you don't need the app installed. Hard to compete with that if you aren't integrated at an OS level.
- em3rgent0rdr 9y agoWill they use this tech for non-music purposes? Such as for enhancing google ads or for sending keywords to the NSA?
- lawlessone 9y agoProbably already have.
- adrianN 9y agoAutomatically identifying whether you're at an illegal public performance and sending lawyers your way to extract the songs from your ears.
- xr4ti 9y agoWell change the words "music" to "conversation" and song to "person" (or phrase), and things start to get pretty scary.
- hugecannon 9y ago> author = {Beat Gfeller and ... A case of nominative determinism? https://en.wikipedia.org/wiki/Nominative_determinism https://en.wikipedia.org/wiki/Nominative_determinism
- piquadrat 9y agoBeat is a _very_ common name in Switzerland. I know about 10, none of them work in music :)
- hugecannon 9y agoI wasn't aware, interesting!
- StavrosK 9y agoIs it pronounced bay-at?
- piquadrat 9y agoFrom https://en.wikipedia.org/wiki/Beat_(name) https://en.wikipedia.org/wiki/Beat_(name): > pronounced "BEH-awe-t"
- homero 9y agoWow that explains the many meteorologists in my area with names like Rains
- qualitytime 9y agoThese are one kind of HN posts that drive me up the wall. You say less power to get us to buy more? How about saving all that energy and using it for real world building.
- taneq 9y agoTo be fair, they say: > Since everything runs locally on the device without sending either audio or fingerprints to a server, the privacy of the user is respected and the whole system can run in airplane mode. This is far more respectful of user privacy than we usually see from Google. I, for one, am impressed.
- Harvey-Specter 9y agoWhat does this have to do with the comment you replied to?
- taneq 9y agoA complaint about marketing ("get us to buy more"), on an article about always-on audio processing in a phone, implies a concern about privacy-invading user tracking. I was pointing out that (on the face of it, at least) this doesn't seem to do that.
- deleted 9y ago[deleted]
- manol74 9y agoGreat work, but super scary for me. The "service" of music recognition will probably be extended very quickly to a constantly running "environment recognition", just fingerprinting the audio around you, places, speakers etc..
- obelos 9y agoYes, this has a “proof of concept” feel to it.
- xr4ti 9y ago> Now Playing, has a daily battery usage of less than 1% on average, respects user privacy by running entirely on-device and can passively recognize a wide range of music. Maybe I am missing something, but this seems to be much less respectful of my privacy than an app that only listens to the ambient sound around me when I explicitly give my consent (by opening/activating the app).
- CapacitorSet 9y agoOf course their baseline for privacy is not "leaks as little user data as possible", but "does everything on the cloud". Hence it seems to respect user privacy from their point of view. Not that they have a reasonable point of view, of course.
- yathern 9y agoWhile I agree with you that "cloud-private" is a new baseline that may not be a great thing - that's a strange point to make, given the text explicitly calls out that it's not doing anything on the cloud. It's all on-device.
- keytarsolo 9y agoThis option has to be explicitly turned on. It's off by default (at least it was on my German Pixel 2).
- pavel_lishin 9y agoIt was enabled on my phone - which I discovered at the dentist's office. Now, it's possible that I idly clicked "ok" while setting up my Pixel 2 - it's not out of the realm of possibility - but I'm not very likely to turn that on at all. Germany has much more stringent privacy regulations than the US does - maybe that's the difference.
- itp 9y agoI've just recently set up a Pixel 2 XL straight from factory a few times over the last couple weeks, all in the US. Every time I had to opt in to the background music detection feature, even in the US.
- ufmace 9y agoI have the Nexus 2 with this feature, and it works remarkably well. No noticeable power drain, and it identifies a lot of songs that I never would have thought to ask about, like TV show intros and such. It's handy that if I ever wonder what a song that's playing is, I can pull out my phone and it's usually already there instead of having to unlock it, ask it what's playing, and wait a bit, if the internet is even working there. Pretty cool that they can pull this off with no network communication.
- svantana 9y agoUsing neural nets for this purpose makes it likely to be susceptible to adversarial examples. Fairly harmless in this use case, but if a similar system would be used for ContentID on YouTube, that could be exploited to subvert it (i.e. spam it with false positives and contest the verdicts en masse).
- spdustin 9y agoSeems to me that it’s another part of a fingerprint for a user that can locate them, and possibly place you with others in the vicinity who have ID’d the same audio. Retail outlets have playlists that could be ID’d, possibly with enough granularity to indicate your path through a shopping mall or your dwell time in a particular department. I know the recognition is done on-device for some percentage of tracks, but it’s been unclear to me what happens to the running tally of times/locations/tracks identified afterward.
- itp 9y agoIt's not sent anywhere or even stored locally on the device. Some people view that as a missing feature, actually, so there are apps that add this functionality[0], so you can review the songs you heard and find it later. [0] http://www.androidpolice.com/2017/10/26/add-track-id-history-pixel-2-now-playing-feature-app/ http://www.androidpolice.com/2017/10/26/add-track-id-history...
- Omniusaspirer 9y agoOnly being able to compare to 70k songs isn't really going to be adequate the second you step off a mainstream playlist.
- awalton 9y ago...this doesn't seem like it is at all worth giving up 1% of battery life. I don't think it's even worth giving away 0.25% of battery life to detect whatever the 70k songs stored in the database is, nor is it worth the space the database is taking up on the device. The question of "what song is this" comes up maybe once a month at best, and apps like Shazam already exist and have much deeper databases to search through. In other words, it does a worse job than existing solutions and uses power constantly. It feels more like yet another solution looking for a problem. Worse, it screams like a foot in the door for telling users it's okay for microphones around you to always be listening, ala Amazon Echo. It also weakly smells like it'll immediately be used to send packets of what songs were listened to and similar frequency info off-device to be collected by the Google Big Data machine to be sold to RIAA members, as yet another way of extracting ad dollars from Android.
- mankyd 9y ago> ...this doesn't seem like it is at all worth giving up 1% of battery life. I view it differently. Percent of battery life isn't an interesting metric in a vacuum. It's whether you easily/regularly get the phone back to a charger without it feeling burdensome. That is to say, does the phone last all your waking hours without threatening to turn off. I own one of these fancy new Pixel 2's. I find that the phone is usually above 30% when I'm returning to bed at night. The music detection has proved useful at times, and a single percent of battery isn't going to make me sweat. However, in a year or two, when the battery starts losing performance, you may be right. My old phone, a 5X, often dipped close to 5% by the time I got to bed. I had to go out of my way to find a charger, which was frustrating. Sacrificing yet another bit of battery for this feature would only make that worse.
- khr 9y agoI have found it to be a useful feature that is worth the 1% daily battery life. I prefer it to Shazam/Soundhound because it's passive. I don't have to unlock my phone, open an app, press a button, and wait for the app to recognize the music. That whole process is disruptive to conversation if I'm with someone. How often the question of "what song is this" comes up depends on who you are, and it has occurred to me often enough such that I've recognized that I much prefer using Now Playing. I agree that the database is a major limitation, although for me it acts as a first pass to see if the song I'm listening to is popular enough to be recognized.
- iforgotpassword 9y agoIs this pixel 2 exclusive or can I install this on my xiaomi? I skimmed the paper but didn't find any name for that app.
- e12e 9y agoI wonder how well this performs. I see I have roughly 2000 songs on mys sd card. I find it a little hard to believe that 40x that is enough to cover a big enough slice to be able to actually answer the question of "what's this" when it comes up? [ed: i actually think I'd have plenty of room for 70k songs in high quality ogg vorbis vbr or equivalent - that'd frankly be more interesting... ]