9 ms·

Breaking Spotify DRM with PANDA

278 points by ivank 12y ago

moyix 12y ago
Post author here! I wanted to make it clear that the original technique for breaking Spotify DRM is not mine - it was developed by Wang et al. in their excellent paper, Steal This Movie: Automatically Bypassing DRM Protection in Streaming Media Services. I just thought it would be a nice showcase of PANDA's capabilities. In particular, we can avoid the crazy optimizations in that paper because we can operate on a replayed execution rather than doing it live.
- rational-future 12y ago
  Hello OP! Can you give a short not so technical description of this attack please?
  - gcv 12y ago
    Not the OP here, but briefly, the technique uses a little statistical cleverness and the ability to examine a running program to help identify which part of the Spotify program produces the unencrypted bytes of the track it is playing. It then taps into that part of the program, and copies these bytes to a file. That file then contains playable audio. (This data must be unencrypted in order to meaningfully feed the digital-to-analog converter of the playback computer.)
  - AlyssaRowan 12y ago
    - Runs it in a virtual machine/emulator (QEMU). - Records (with PANDA) every instruction run and every piece of data read or written by that virtual machine for half a minute while it's playing audio. (!) - Analyses that recording and uses some very clever statistics to identify functions that read chunks of data that looks encrypted, and write chunks of data that looks compressed (yes, you can tell the difference, compression is imperfect). - Out pops one likely candidate, which sure enough is the decrypter.
    - moyix 12y ago
      Small correction: the magic of record-replay is that you don't have to record every piece of data read/written, only the non-deterministic events (interrupts, accesses to devices, and a couple instructions like `rdtsc`). This lets the overhead during recording stay low, while still getting all the benefits of a full execution trace. This is why we're able to store the Spotify trace [1] in only 263.1 MB. [1] http://www.rrshare.org/detail/28/ http://www.rrshare.org/detail/28/
- thrownaway2424 12y ago
  Hello, author. I read your whole post and still didn't know what PANDA might be. Please add a description or a link.
  - rememberlenny 12y ago
    His other post explains it: http://moyix.blogspot.com/2013/09/announcing-panda-platform-for.html http://moyix.blogspot.com/2013/09/announcing-panda-platform-...
    - test_account_1 12y ago
      Thanks, saved me time for searching.
- msane 12y ago
  On the subject of cracking streaming music DRMs, a realization I have been sitting on for a while about what people can do with it... Considering the common wisdom that: - storage will get exponentially cheaper - data transfer speeds will get higher It makes me think that eventually there will be illicit torrents of all the worlds music, plus the index, plus the metadata, and plus the interface/app for browsing it. In other words, people would not only pirate individual songs or movies, they would download their own complete copies of Spotify / Netflix. It isn't feasible now but it could be sometime in the next 5-15 years, depending bandwidth speeds. I'm not sure how many people see this coming or take it seriously but I wonder what the effect could be and what the remedy attempts would be.
  - null_ptr 12y ago
    You can see the remedy attempts already * push for cloud storage over local storage * push for locked down devices over general purpose computers * push for DRM on the open web * big ISP companies fighting against net neutrality
    - msane 12y ago
      I very much agree. I hope that, capitalism, ironically enough for DRM, is the probably always the solution to overstepped DRM remedies. Regardless of the issue of piracy, which I am not advocating. For instance there will always be a market for custom computing and "PCs", and so non-locked PCs will (hopefully) always exist in a capitalist environment. That market I think ultimately circumvents any attempt at ubiquitous control of hardware. The same thing is at play with software. And hypothetical new methods of connectivity may be able to circumvent many attempts at central control of the net.
      - userbinator 12y ago
        For instance there will always be a market for custom computing and "PCs", and so non-locked PCs will (hopefully) always exist in a capitalist environment. Unfortunately, that market is slowly becoming the minority, and because those "more free" devices may have limitations that make them incompatible with a lot of proprietary content (which is the majority) and circumventing those limitations could be illegal and difficult, there will be fewer users of them. Reminds me of this https://www.gnu.org/philosophy/right-to-read.html https://www.gnu.org/philosophy/right-to-read.html and the old saying "If you outlaw freedom, only outlaws will have freedom."
- AlyssaRowan 12y ago
  That's a lovely piece of work. Congratulations. It's always fascinating to me how far the tools of the trade have come on since my heyday. They surpassed my own custom tools of yore a little while back, and things like this were things I could obviously never do at the time - nowhere near enough storage, for a start, but I also didn't know about the compressed/encrypted distinction (that would have saved me a lot of time!). That you can do it in such an automated fashion now, where I just paged through the disassembly and hexdumps until I saw something that leaped out at me, is stunning. I'd coded a few helper routines, and my debugger was completely stealth and I could rewind a couple thousand instructions (the secret of my success! <g>), but never anything like that back then. Perhaps I should look into this PANDA.
MichaelGG 12y ago
If you're wondering, PANDA is this:https://github.com/moyix/panda https://github.com/moyix/panda QEMU based recorder. Sounds pretty neat. Although if you wanted realtime audio decryption, I'd imagine just writing a sound driver or hooking the OS's sound functions would be far more direct.
- nzmsv 12y ago
  That would miss the point. The approach in the article does not hook the audio driver, but rather the decryptor. This gets you the original compressed audio file, not a stream that has been uncompressed (and would have artifacts when re-compressed).
  - csense 12y ago
    Not if you recompressed with a lossless codec like FLAC.
    - KMag 12y ago
      That's true, unless the original audio is 24-bit and your audio driver is sending 16-bit audio to your sound card, or similar sorts of loss are happening post-decompression. Many devices have hardware acceleration for mp3 decoding, which also helps with battery life. I'm not sure how many devices have hardware accelerators that can be used/repurposed for Vorbis decoding in the Spotify case. I believe it's less common for hardware decoders to be useful for FLAC decoding, so there may be a battery life penalty for re-encoding mp3 or Vorbis audio as FLAC. (I also presume everyone realizes re-encoding lossily-compressed audio using a lossless codec likely pays a size penalty without any quality increase.)
  - MichaelGG 12y ago
    That's fair. Although a lot of BluRay rips are re-encoded and seem quite usable. Even the YiFY stuff at 1GB/hour or so. Edit: Why can't a compressor perfectly re-compress the decompressed audio? It's obviously possible since the compressed data exists producing that specific decompressed data.
spindritf 12y ago
I'm pretty sure you used to be able to just copy .ogg files from Spotify's local cache at one point. Am I remembering right? It's a soup of 10-2000kB files now.
- AlyssaRowan 12y ago
  At one very early point in its history, yes, I understand that was accurate. Then they encrypted them, but the key was easy to find, and then they started doing more complex stuff. It's of idle academic interest to me. Never used Spotify, but I don't wish them harm either.
Steer 12y ago
I don't know, perhaps I'm totally wrong, but this just depresses me. I can see the point from an engineering point-of-view, this is a riddle to be solved, but why would you want to help people circumvent one of the (to me) reasonable ways of enjoying and paying for music? I know the author stopped short of giving the full solution to getting it to work, but still. Is this what we've come to? No one should get paid for anything if we can enjoy it for free regardless of the hoops we will have to jump through to not pay? Sorry if I am too dramatic. I can often see the point of pirating things, but in this case I just don't get it. Edit: I would appreciate an explanation of the downvotes.
- tptacek 12y ago
  I understand where you're coming from (also, piracy repulses me). However: it's worth knowing whether content protection is implemented soundly. Contrary to overwhelming popular opinion, there are content protection schemes that work. Generalist engineers mistakenly believe that content protection schemes must be unbreakable to provide value. They don't: all they have to do is cost more to break than the value of the content they protect (across all the users who might subsequently get access to it). For an example of a content protection scheme that worked extremely well, see modern satellite TV smart cards. Given that there are ways to implement content protection soundly, there's validity to research that determines whether a given content protection scheme is sound.
  - Steer 12y ago
    First, I'm a bit starstruck by getting a reply from tptacek, I feel validated that I wasn't totally off base. Silly, I know. Thank you for your reply and you are of course right, I appreciate you also putting this into context (tv smart cards). As I said, I was probably overly dramatic. My point was that normally we explain piracy by "it is an easier way of enjoying ..." and I can agree with that, but few things are easier to use (and pay for) than Spotify. I can see your point about researching their content protection. Edit: Wow, learned my lesson, don't express admiration. I guess I just don't get the downvoting without leaving a comment.
  - bri3d 12y ago
MichaelAza 12y ago
This is really cool. Both referenced tools (PANDA and Volatility) seem awesome. I have some experience with IDA and Olly, but learning about new tools is always nice. Any resources you guys recommend on the subject of reversing?
shmerl 12y ago
Isn't music available through DRM-free sources most of the time anyway? Just don't use Spotify if you are against DRM. It will also be a vote with your wallet against it. By using it you implicitly support DRM proliferation.
- jdong 12y ago
  Voting with your wallet is much less effective than publicly complaining, unless of course your wallet is worth a lot.
  - shmerl 12y ago
    My comment wasn't really to the authors of this tool. Their tool is a form of the public protest and is appropriate. It was more directed to those who actually use Spotify. For them, complaining while actually using these kind of services (and helping them to spread more DRM in the process) is strange.
    - jdong 12y ago
      What if they like the product but would rather have it without the DRM?
      - shmerl 12y ago
        I'd say they should stop using it until the product drops the DRM. Otherwise their complaints don't sound sincere. Or at least not convincing. It's like as if smokers would complain that cigarets industry ruins public health.
        MichaelGG 12y ago
        Smokers complain, and now we have e-cigarettes which may be less harmful while still allowing the enjoyment of nicotine and the fun of smoking. I'm not sure why it's insincere. I keep buying ThinkPads, but I hate the new designs with a passion. (And they hate me; they literally cause me RSI where the earlier ones didn't.) It doesn't make my arguments against the new ThinkPads any weaker.
timsally 12y ago
Shameless plug to follow... Much of the technology here was invented by Brendan and others at MIT Lincoln Laboratory, which is where I work. We have been very lucky to have Brendan join us for a few summers while he was completing his PhD at Georgia Tech and he gave a great showing at RECON. Brilliant guy. If you're interested in reverse engineering his most recent papers are essential reading: http://www.cc.gatech.edu/grads/b/brendan http://www.cc.gatech.edu/grads/b/brendan. In addition to some of the automated RE work, we've also got multi-million dollar research efforts hacking the Linux kernel and reverse engineering/analyzing embedded systems. Lot's of fun stuff. You get to work on really exciting problems and you'll have the funding and the skilled coworkers you need to execute successfully. If you find this type of stuff exciting, you should drop me a line at sally@ll.mit.edu. We're always hiring^. We've got great benefits too, like a pension, unlimited sick leave, 13 holidays, 20 vacation days, and free classes at MIT. ^One caveat is that because of how we are funded, we are only able to employ US citizens.
- reitanqild 12y ago
  > One caveat is that because of how we are funded, we are only able to employ US citizens. Just wanted to say thank you: just being aware of limitations like this and letting people know up front makes it a whole less annoying. Also the circumstances makes it understandable.
  - test_account_1 12y ago
    I don't understand why being funded has anything to do of only hiring US citizens.
    - lsaferite 12y ago
      I think maybe you misunderstood the statement. Not funded as in "my startup just got funded." More like, "we are funded by US government grants that preclude us from hiring aliens." Just my guess.
    - KMag 12y ago
      MIT Lincoln Labs is funded by the department of defense. They do a lot of classified work, and may have large areas that are off-limits to non-US citizens. I had a Summer internship at a similar institution, where my work wasn't classified, but was sensitive enough and close enough to classified work that only US citizens were allowed past the keypad lock to my desk area.
sigzero 12y ago
I guess I could just google it but isn't breaking DRM illegal?
- DanBC 12y ago
  Yes, under US DMCA and EU something or other. EDIT: here's the relevant wikipedia article http://en.m.wikipedia.org/wiki/Anti-circumvention http://en.m.wikipedia.org/wiki/Anti-circumvention (sorry for mobile link)
  - eli 12y ago
    "Sec. 103(f) of the DMCA (17 U.S.C. § 1201 (f)) says that if you legally obtain a program that is protected, you are allowed to reverse-engineer and circumvent the protection to achieve the ability the interoperability of computer programs (i.e., the ability to exchange and make use of information)." Though I wouldn't want to have to test that in court. IANAL.
- KMag 12y ago
  Here in China? Perhaps.
desarun 12y ago
I read this and thought "I am not a true developer, I am merely a journalist-style hack". Disclaimer: Contractor, native app dev.
the_cat_kittles 12y ago
this is cool, but for someone who actually wants to rip songs off spotify, why would you do this instead of use audio hijack? is there some advantage?
- MichaelGG 12y ago
  As mentioned in another comment, if you do this you get the original compressed file. If you capture the audio and then want to recompress it, you'll introduce artifacts. Although, it seems to me that an intelligent compressor could perfectly recompress the audio back to the original form.
  - KMag 12y ago
    This presumes that downsampling in either bit depth or sample rate isn't happening somewhere (such as your audio drivers) between decompressing the audio and where you're capturing the audio.
- lucb1e 12y ago
  At the very least it makes a good demo for a technique that could be applied in other situations. Or even just the hack value (see the jargon file). Many things have value in some indirect way.
n0body 12y ago
A very interesting read.
ilyagr 12y ago
I am wondering if someone who understands statistics better than I do could explain conceptually how encrypted data is distinguishable from compressed data. I always assumed that Shannon's paper says that perfectly compressed data should be indistinguishable from random data (which is indistinguishable from encrypted data). Is mp3 compression not sufficiently perfect? Is my understanding wrong? Thank you! P.S. This is a really clever trick!
- MichaelGG 12y ago
  Pretty sure you explained it: it's not perfectly compressed data. Even a small bias is enough to give it away.
- pgeorgi 12y ago
  media compression usually isn't perfect. For starters, data is framed with synchronization bits to recover from bad transmissions and the like. Also, audio codecs usually use prepared huffman codebooks for the final step. For most codecs they're defined as part of the format, few (such as vorbis) prepend the codebooks to the data stream so compressors can use different codebooks if they choose to do so. For vorbis there used to be an experimental tool that optimized huffman coding of an existing stream, giving another 4%-or-so compression with otherwise identical data (ie. no quality change). These 4% plus framing are probably the entropy differential they're exploiting here.
lucb1e 12y ago
Uh what? Wow, I thought for a minute it said you took a 30-second recording and I was wondering what you were going to do to get the file format right. Use a raw encoding to get close to audio levels? Then I read on, noticed something and had to read back. Indeed, you didn't record the audio, you just recorded and replayed the operating system. Say what!
- keehun 12y ago
  Yeah, it's next-level stuff, here. He said around 12 billion instructions which sounds like a lot, but with our current processors, not that much work for the CPUs.
  - jzwinck 12y ago
    More specifically, a billion instructions is one giga-instruction. A 2 GHz processor can execute roughly 2 billion instructions per second (this is a very rough estimate, thanks to superscalar, pipelining, uops, yada yada). So 12 billion (typical) instructions will take around 6 seconds to run. This sort of thing is handy when profiling: see a function taking a billion instructions? Half a second of CPU time. And this ratio hasn't changed all that much in quite a while (what has changed, of course, is how many threads can execute simultaneously).
icoder 12y ago
Would it have been possible to figure this out without the statistics by playing something very specific, like a 440Hz tone (yes, there's spotify 'music' that does just that)?

278 points by ivank 12y ago

moyix 12y ago

Post author here! I wanted to make it clear that the original technique for breaking Spotify DRM is not mine - it was developed by Wang et al. in their excellent paper, Steal This Movie: Automatically Bypassing DRM Protection in Streaming Media Services. I just thought it would be a nice showcase of PANDA's capabilities. In particular, we can avoid the crazy optimizations in that paper because we can operate on a replayed execution rather than doing it live.

rational-future 12y ago
Hello OP! Can you give a short not so technical description of this attack please?
- gcv 12y ago
  Not the OP here, but briefly, the technique uses a little statistical cleverness and the ability to examine a running program to help identify which part of the Spotify program produces the unencrypted bytes of the track it is playing. It then taps into that part of the program, and copies these bytes to a file. That file then contains playable audio. (This data must be unencrypted in order to meaningfully feed the digital-to-analog converter of the playback computer.)
- AlyssaRowan 12y ago
  - Runs it in a virtual machine/emulator (QEMU). - Records (with PANDA) every instruction run and every piece of data read or written by that virtual machine for half a minute while it's playing audio. (!) - Analyses that recording and uses some very clever statistics to identify functions that read chunks of data that looks encrypted, and write chunks of data that looks compressed (yes, you can tell the difference, compression is imperfect). - Out pops one likely candidate, which sure enough is the decrypter.
  - moyix 12y ago
    Small correction: the magic of record-replay is that you don't have to record every piece of data read/written, only the non-deterministic events (interrupts, accesses to devices, and a couple instructions like `rdtsc`). This lets the overhead during recording stay low, while still getting all the benefits of a full execution trace. This is why we're able to store the Spotify trace [1] in only 263.1 MB. [1] http://www.rrshare.org/detail/28/ http://www.rrshare.org/detail/28/
thrownaway2424 12y ago
Hello, author. I read your whole post and still didn't know what PANDA might be. Please add a description or a link.
- rememberlenny 12y ago
  His other post explains it: http://moyix.blogspot.com/2013/09/announcing-panda-platform-for.html http://moyix.blogspot.com/2013/09/announcing-panda-platform-...
  - test_account_1 12y ago
    Thanks, saved me time for searching.
msane 12y ago
On the subject of cracking streaming music DRMs, a realization I have been sitting on for a while about what people can do with it... Considering the common wisdom that: - storage will get exponentially cheaper - data transfer speeds will get higher It makes me think that eventually there will be illicit torrents of all the worlds music, plus the index, plus the metadata, and plus the interface/app for browsing it. In other words, people would not only pirate individual songs or movies, they would download their own complete copies of Spotify / Netflix. It isn't feasible now but it could be sometime in the next 5-15 years, depending bandwidth speeds. I'm not sure how many people see this coming or take it seriously but I wonder what the effect could be and what the remedy attempts would be.
- null_ptr 12y ago
  You can see the remedy attempts already * push for cloud storage over local storage * push for locked down devices over general purpose computers * push for DRM on the open web * big ISP companies fighting against net neutrality
  - msane 12y ago
    I very much agree. I hope that, capitalism, ironically enough for DRM, is the probably always the solution to overstepped DRM remedies. Regardless of the issue of piracy, which I am not advocating. For instance there will always be a market for custom computing and "PCs", and so non-locked PCs will (hopefully) always exist in a capitalist environment. That market I think ultimately circumvents any attempt at ubiquitous control of hardware. The same thing is at play with software. And hypothetical new methods of connectivity may be able to circumvent many attempts at central control of the net.
    - userbinator 12y ago
      For instance there will always be a market for custom computing and "PCs", and so non-locked PCs will (hopefully) always exist in a capitalist environment. Unfortunately, that market is slowly becoming the minority, and because those "more free" devices may have limitations that make them incompatible with a lot of proprietary content (which is the majority) and circumventing those limitations could be illegal and difficult, there will be fewer users of them. Reminds me of this https://www.gnu.org/philosophy/right-to-read.html https://www.gnu.org/philosophy/right-to-read.html and the old saying "If you outlaw freedom, only outlaws will have freedom."
AlyssaRowan 12y ago
That's a lovely piece of work. Congratulations. It's always fascinating to me how far the tools of the trade have come on since my heyday. They surpassed my own custom tools of yore a little while back, and things like this were things I could obviously never do at the time - nowhere near enough storage, for a start, but I also didn't know about the compressed/encrypted distinction (that would have saved me a lot of time!). That you can do it in such an automated fashion now, where I just paged through the disassembly and hexdumps until I saw something that leaped out at me, is stunning. I'd coded a few helper routines, and my debugger was completely stealth and I could rewind a couple thousand instructions (the secret of my success! <g>), but never anything like that back then. Perhaps I should look into this PANDA.

MichaelGG 12y ago

If you're wondering, PANDA is this:https://github.com/moyix/panda https://github.com/moyix/panda QEMU based recorder. Sounds pretty neat. Although if you wanted realtime audio decryption, I'd imagine just writing a sound driver or hooking the OS's sound functions would be far more direct.

nzmsv 12y ago
That would miss the point. The approach in the article does not hook the audio driver, but rather the decryptor. This gets you the original compressed audio file, not a stream that has been uncompressed (and would have artifacts when re-compressed).
- csense 12y ago
  Not if you recompressed with a lossless codec like FLAC.
  - KMag 12y ago
    That's true, unless the original audio is 24-bit and your audio driver is sending 16-bit audio to your sound card, or similar sorts of loss are happening post-decompression. Many devices have hardware acceleration for mp3 decoding, which also helps with battery life. I'm not sure how many devices have hardware accelerators that can be used/repurposed for Vorbis decoding in the Spotify case. I believe it's less common for hardware decoders to be useful for FLAC decoding, so there may be a battery life penalty for re-encoding mp3 or Vorbis audio as FLAC. (I also presume everyone realizes re-encoding lossily-compressed audio using a lossless codec likely pays a size penalty without any quality increase.)
- MichaelGG 12y ago
  That's fair. Although a lot of BluRay rips are re-encoded and seem quite usable. Even the YiFY stuff at 1GB/hour or so. Edit: Why can't a compressor perfectly re-compress the decompressed audio? It's obviously possible since the compressed data exists producing that specific decompressed data.

spindritf 12y ago

I'm pretty sure you used to be able to just copy .ogg files from Spotify's local cache at one point. Am I remembering right? It's a soup of 10-2000kB files now.

AlyssaRowan 12y ago
At one very early point in its history, yes, I understand that was accurate. Then they encrypted them, but the key was easy to find, and then they started doing more complex stuff. It's of idle academic interest to me. Never used Spotify, but I don't wish them harm either.

Steer 12y ago

I don't know, perhaps I'm totally wrong, but this just depresses me. I can see the point from an engineering point-of-view, this is a riddle to be solved, but why would you want to help people circumvent one of the (to me) reasonable ways of enjoying and paying for music? I know the author stopped short of giving the full solution to getting it to work, but still. Is this what we've come to? No one should get paid for anything if we can enjoy it for free regardless of the hoops we will have to jump through to not pay? Sorry if I am too dramatic. I can often see the point of pirating things, but in this case I just don't get it. Edit: I would appreciate an explanation of the downvotes.

tptacek 12y ago
I understand where you're coming from (also, piracy repulses me). However: it's worth knowing whether content protection is implemented soundly. Contrary to overwhelming popular opinion, there are content protection schemes that work. Generalist engineers mistakenly believe that content protection schemes must be unbreakable to provide value. They don't: all they have to do is cost more to break than the value of the content they protect (across all the users who might subsequently get access to it). For an example of a content protection scheme that worked extremely well, see modern satellite TV smart cards. Given that there are ways to implement content protection soundly, there's validity to research that determines whether a given content protection scheme is sound.
- Steer 12y ago
  First, I'm a bit starstruck by getting a reply from tptacek, I feel validated that I wasn't totally off base. Silly, I know. Thank you for your reply and you are of course right, I appreciate you also putting this into context (tv smart cards). As I said, I was probably overly dramatic. My point was that normally we explain piracy by "it is an easier way of enjoying ..." and I can agree with that, but few things are easier to use (and pay for) than Spotify. I can see your point about researching their content protection. Edit: Wow, learned my lesson, don't express admiration. I guess I just don't get the downvoting without leaving a comment.
- bri3d 12y ago

MichaelAza 12y ago

This is really cool. Both referenced tools (PANDA and Volatility) seem awesome. I have some experience with IDA and Olly, but learning about new tools is always nice. Any resources you guys recommend on the subject of reversing?

shmerl 12y ago

Isn't music available through DRM-free sources most of the time anyway? Just don't use Spotify if you are against DRM. It will also be a vote with your wallet against it. By using it you implicitly support DRM proliferation.

jdong 12y ago
Voting with your wallet is much less effective than publicly complaining, unless of course your wallet is worth a lot.
- shmerl 12y ago
  My comment wasn't really to the authors of this tool. Their tool is a form of the public protest and is appropriate. It was more directed to those who actually use Spotify. For them, complaining while actually using these kind of services (and helping them to spread more DRM in the process) is strange.
  - jdong 12y ago
    What if they like the product but would rather have it without the DRM?
    - shmerl 12y ago
      I'd say they should stop using it until the product drops the DRM. Otherwise their complaints don't sound sincere. Or at least not convincing. It's like as if smokers would complain that cigarets industry ruins public health.
      - MichaelGG 12y ago
        Smokers complain, and now we have e-cigarettes which may be less harmful while still allowing the enjoyment of nicotine and the fun of smoking. I'm not sure why it's insincere. I keep buying ThinkPads, but I hate the new designs with a passion. (And they hate me; they literally cause me RSI where the earlier ones didn't.) It doesn't make my arguments against the new ThinkPads any weaker.

timsally 12y ago

Shameless plug to follow... Much of the technology here was invented by Brendan and others at MIT Lincoln Laboratory, which is where I work. We have been very lucky to have Brendan join us for a few summers while he was completing his PhD at Georgia Tech and he gave a great showing at RECON. Brilliant guy. If you're interested in reverse engineering his most recent papers are essential reading: http://www.cc.gatech.edu/grads/b/brendan http://www.cc.gatech.edu/grads/b/brendan. In addition to some of the automated RE work, we've also got multi-million dollar research efforts hacking the Linux kernel and reverse engineering/analyzing embedded systems. Lot's of fun stuff. You get to work on really exciting problems and you'll have the funding and the skilled coworkers you need to execute successfully. If you find this type of stuff exciting, you should drop me a line at sally@ll.mit.edu. We're always hiring^. We've got great benefits too, like a pension, unlimited sick leave, 13 holidays, 20 vacation days, and free classes at MIT. ^One caveat is that because of how we are funded, we are only able to employ US citizens.

reitanqild 12y ago
> One caveat is that because of how we are funded, we are only able to employ US citizens. Just wanted to say thank you: just being aware of limitations like this and letting people know up front makes it a whole less annoying. Also the circumstances makes it understandable.
- test_account_1 12y ago
  I don't understand why being funded has anything to do of only hiring US citizens.
  - lsaferite 12y ago
    I think maybe you misunderstood the statement. Not funded as in "my startup just got funded." More like, "we are funded by US government grants that preclude us from hiring aliens." Just my guess.
  - KMag 12y ago
    MIT Lincoln Labs is funded by the department of defense. They do a lot of classified work, and may have large areas that are off-limits to non-US citizens. I had a Summer internship at a similar institution, where my work wasn't classified, but was sensitive enough and close enough to classified work that only US citizens were allowed past the keypad lock to my desk area.

sigzero 12y ago

I guess I could just google it but isn't breaking DRM illegal?

DanBC 12y ago
Yes, under US DMCA and EU something or other. EDIT: here's the relevant wikipedia article http://en.m.wikipedia.org/wiki/Anti-circumvention http://en.m.wikipedia.org/wiki/Anti-circumvention (sorry for mobile link)
- eli 12y ago
  "Sec. 103(f) of the DMCA (17 U.S.C. § 1201 (f)) says that if you legally obtain a program that is protected, you are allowed to reverse-engineer and circumvent the protection to achieve the ability the interoperability of computer programs (i.e., the ability to exchange and make use of information)." Though I wouldn't want to have to test that in court. IANAL.
KMag 12y ago
Here in China? Perhaps.

desarun 12y ago

I read this and thought "I am not a true developer, I am merely a journalist-style hack". Disclaimer: Contractor, native app dev.

the_cat_kittles 12y ago

this is cool, but for someone who actually wants to rip songs off spotify, why would you do this instead of use audio hijack? is there some advantage?

MichaelGG 12y ago
As mentioned in another comment, if you do this you get the original compressed file. If you capture the audio and then want to recompress it, you'll introduce artifacts. Although, it seems to me that an intelligent compressor could perfectly recompress the audio back to the original form.
- KMag 12y ago
  This presumes that downsampling in either bit depth or sample rate isn't happening somewhere (such as your audio drivers) between decompressing the audio and where you're capturing the audio.
lucb1e 12y ago
At the very least it makes a good demo for a technique that could be applied in other situations. Or even just the hack value (see the jargon file). Many things have value in some indirect way.

n0body 12y ago

A very interesting read.

ilyagr 12y ago

I am wondering if someone who understands statistics better than I do could explain conceptually how encrypted data is distinguishable from compressed data. I always assumed that Shannon's paper says that perfectly compressed data should be indistinguishable from random data (which is indistinguishable from encrypted data). Is mp3 compression not sufficiently perfect? Is my understanding wrong? Thank you! P.S. This is a really clever trick!

MichaelGG 12y ago
Pretty sure you explained it: it's not perfectly compressed data. Even a small bias is enough to give it away.
pgeorgi 12y ago
media compression usually isn't perfect. For starters, data is framed with synchronization bits to recover from bad transmissions and the like. Also, audio codecs usually use prepared huffman codebooks for the final step. For most codecs they're defined as part of the format, few (such as vorbis) prepend the codebooks to the data stream so compressors can use different codebooks if they choose to do so. For vorbis there used to be an experimental tool that optimized huffman coding of an existing stream, giving another 4%-or-so compression with otherwise identical data (ie. no quality change). These 4% plus framing are probably the entropy differential they're exploiting here.

lucb1e 12y ago

Uh what? Wow, I thought for a minute it said you took a 30-second recording and I was wondering what you were going to do to get the file format right. Use a raw encoding to get close to audio levels? Then I read on, noticed something and had to read back. Indeed, you didn't record the audio, you just recorded and replayed the operating system. Say what!

keehun 12y ago
Yeah, it's next-level stuff, here. He said around 12 billion instructions which sounds like a lot, but with our current processors, not that much work for the CPUs.
- jzwinck 12y ago
  More specifically, a billion instructions is one giga-instruction. A 2 GHz processor can execute roughly 2 billion instructions per second (this is a very rough estimate, thanks to superscalar, pipelining, uops, yada yada). So 12 billion (typical) instructions will take around 6 seconds to run. This sort of thing is handy when profiling: see a function taking a billion instructions? Half a second of CPU time. And this ratio hasn't changed all that much in quite a while (what has changed, of course, is how many threads can execute simultaneously).

icoder 12y ago

Would it have been possible to figure this out without the statistics by playing something very specific, like a 440Hz tone (yes, there's spotify 'music' that does just that)?