6 ms·

Removing Edge Magazine DRM

145 points by Metalnem 10y ago

Buge 10y ago
Could most of the initial API reverse engineering be blocked by using https with certificate pinning? I tried reverse engineering an Android app API and came up short because every MITM I tried failed, I assume due to certificate pinning. To continue I would have to statically reverse the binary, or root/jailbreak the device and nerf the pinning section of the code.
- Metalnem 10y ago
  Yes, certificate pinning would have blocked my initial steps. But there is a tool for disabling certificate pinning validation on jailbroken iOS devices, called iOS SSL Kill Switch https://github.com/iSECPartners/ios-ssl-kill-switch https://github.com/iSECPartners/ios-ssl-kill-switch.
cprecioso 10y ago
Super good article/tutorial, but I think to get the app binary, he could just have downloaded it in iTunes in his computer and unpack the .ipa file (which is basically a signed .zip file, if I remember correctly). The binary should be in it.
- robterrell 10y ago
  Yep, that still works! Just copy the .ipa from the local ~/Library/iTunes/ subdirectory and decompress it. It would have saved him all the trouble with jailbreaking, finding an old enough device, finding the correct version of iTunes to run against, etc. But I imagine a jailbroken device is its own reward.
  - 0x0 10y ago
    But those .ipa files only contain encrypted executables, I believe it's called Fairplay DRM. As far as I know there are no tools available to offline decrypt such binaries, so if you're looking to reverse engineer one, you still need a real jailbroken phone so you can piggy-back on iOS' just-in-time decryption and dump the cleartext binary.
    - Metalnem 10y ago
      You can decompress every .ipa files and see bunch of resources and configuration files in plaintext, which can falsely lead you to believe that you have everything decrypted, but the main binary file is still encrypted, and the only way to decrypt is by running software like Clutch on the jailbroken device.
  - hello_asdf 10y ago
    Wouldn't the application still be encrypted then? I thought the benefit of jailbroken in this case, was using Clutch to decrypt it.
    - robterrell 10y ago
      Sorry! Yes, to be clear, the code segments of the binary will be encrypted by FairPlay. But the other segments aren't. The "strings" command will still dump all of the strings in these binaries. You'd need to jailbreak to figure out the algorithm of how those substrings strings were concatenated, but he probably could have deduced what was going on from the "SavedPasswords" string just the same.
      - Metalnem 10y ago
        I thought about trying the "strings" command first, but I didn't expect that the password would be just laying there. Even if I found the "SavedPassword" string, I couldn't use it the way I did without jailbroken device, because configuration file that contained the saved password wasn't embedded in the application. The password was actually written to configuration file on the device first time it was used.
oxguy3 10y ago
Haha, just a few days I made a Dark Horse download script for my own purposes as well (except mine converts to CBZ right in the browser instead of requiring a Go script): https://greasyfork.org/en/scripts/26635-dark-horse-downloader https://greasyfork.org/en/scripts/26635-dark-horse-downloade...
- Metalnem 10y ago
  So there are more people who read Dark Horse comics and think like me :) Converting into CBZ directly was my initial idea, but then I discovered that there is direct download link to tar archive that contains all the images (I did that by following the network traffic of the iOS app). I didn't want to bother with half-baked JavaScript libraries to extract the tar archive and recompress it into zip, so I just quickly wrote a Go application to do that.
vaveh 10y ago
Interesting and accessible read. But, won't this stop working as soon as they change the pdf password and update the app? Although that's probably a bad idea because all the older versions of the app will stop working.
matheusmoreira 10y ago
Excellent. Wish I could have read this article before I started learning about reverse engineering. I think it's notable how he ended up with a better product: one simple DRM-free file viewable by applications and devices of his choosing. I don't really understand why the company even bothered with all these protection measures. Did they actually believe it would work?
- maksa 10y ago
  They (righfuly so) count on that that there are not many Metalnems out there.
qubex 10y ago
Almost off-topic: I bought the very first edition (detailing the as-yet-to-be-released 3DO console) in August 1992 and still have the first two years' worth in custom cases.
- Leynos 10y ago
  Likewise. I plan on hanging onto those. It was the first games magazine I took out a subscription to, and was still a subscriber up until 2010 or so, when I lost interest in games and couldn't justify the continued expense. Having come from the Sinclair Spectrum only a few years ago, by way of the Master System and Super Nintendo, the whole thing was quite eye opening, especially prior to the internet being readily available. I like to think of the years 1993-1995 being a period of irrational optimism, when the technology available just couldn't keep up with the dreams people had for it. This was kind of borne out by the slew of bizarre and not-quite capable consoles and multimedia systems released over these years. The first few years of Edge are the most interesting to read in retrospect because they tell the story of an industry not yet quite sure what it wants to be when it grows up and fighting to get there, wherever "there" might be.
  - qubex 10y ago
    Quite inexplicably I wasn't a gamer (I was 11, didn't nor ever had consoles or PC games). I was still fascinated by the technology though.
- astrod 10y ago
  Those early years up to 2002 when the budget cuts occurred were some of the best video game journalism and industry insight I had come across. It just wasnt the same after the core group of writers and editors left.
apricot 10y ago
This article reminded me of Apple II software deprotection articles published in Computist magazine (available at http://archive.org/details/computist http://archive.org/details/computist ). The way the story is told, with victories that turn out to hide yet another layer of protection, the need to delve into raw hexadecimal, and finally the discovery of a poorly-hidden key, is very much how 8-bit software cracking used to be done in the eighties. Plus ça change...
sbuttgereit 10y ago
I appreciate that this won't be a popular comment, but doesn't this sort of thing fall under the anti-circumvention provisions of the DMCA (or later)? This is not a comment about the rightness/wrongness of copyright law, but a question that seems like it's worth asking.
- traverseda 10y ago
  I'm not in the USA. Is the author?
  - matheusmoreira 10y ago
    The author is in Serbia, according to his GitHub profile.
- shmerl 10y ago
  It probably does, but it doesn't make such effort less worthy. Anti-circumvention provisions should have existed to begin with, same as DRM.
- pbhjpbhj 10y ago
  If the user were in the USA then 17 USC 1201 (which I think is at question) says this at Section C(1): >"Nothing in this section shall affect rights, remedies, limitations, or defenses to copyright infringement, including fair use, under this title." (17 USC 1201) // So presumably personal format-shifting is still a fair-use defence that supersedes considerations under this section. FWIW format-shifting was expressly removed from UK Fair Dealing after a challenge by BASCA, presumably no UK musicians rip songs to their MP3 player/phone or whatever. Wish they'd sued Apple Computers for contributory infringement, what with the massive damage iPods have done to people wanting to buy music. /sarcasm
  - sbuttgereit 10y ago
    I'm thinking more that employing such circumvention for your own purposes would be hard to argue against, but publishing it publicly may not actually be defensible under U.S. law, even with the fair use exemption. I think if I were thinking of publishing something like this I'd look at the outcome of lawsuits such as those surrounding the old DeCSS code (https://en.wikipedia.org/wiki/Universal_City_Studios,_Inc._v._Reimerdes https://en.wikipedia.org/wiki/Universal_City_Studios,_Inc._v...). I'm in the U.S. so for me something like this would be relevant. For those elsewhere there may still be issues as DMCA is the U.S. implementation of the WIPO (https://en.wikipedia.org/wiki/WIPO_Copyright_Treaty https://en.wikipedia.org/wiki/WIPO_Copyright_Treaty), so there may be issues elsewhere, too.
endgame 10y ago
This is an impressive achievement and a good read, but the shit people have to do to back up things that they bought is just sad.
smilekzs 10y ago
"secure_download_url": "http:\/\/... ugh.
Theodores 10y ago
Maybe a call through to Future Publishing could have been tried first. If you don't ask then you don't get! Of course this would have meant not collecting 'FBI grade' skills along the way, however, picking up the phone, flattering some folks at Future Publishing and getting the goods could be considered 'social engineering' and as good a skill as getting the debugger out.
shmerl 10y ago
Thanks for pointing out Dark Horse downloader.
Hondor 10y ago
Interestingly, instead of a random password, it was leet-speak for "Ford the epic honky".
jccalhoun 10y ago
Very interesting read. I have seen pdfs of Edge on piracy sites for years. I wonder if those are created through a similar method or if they are just screenshots of pages from the official app?
- Metalnem 10y ago
  That's an interesting question. There is one thing that is unique to the Edge Magazine app on iOS, though. Zip archives of the magazine do not contain just pdf files, but also bunch of high resolution game screenshots. Those screenshots can be seen in the app by clicking the gallery button in a several places. The gallery button link is in the form media://image.jpg, which probably means that the application registered custom protocol handler and handles it by displaying image on top of the magazine itself. That's the way to recognize iOS version, but I'm not sure if Google Play Newsstand has the same feature.
jpalomaki 10y ago
I was quite surprised how difficult it was to get hold of the iOS binary and how this still relied on the application being available for old iOS version.

145 points by Metalnem 10y ago

Buge 10y ago

Could most of the initial API reverse engineering be blocked by using https with certificate pinning? I tried reverse engineering an Android app API and came up short because every MITM I tried failed, I assume due to certificate pinning. To continue I would have to statically reverse the binary, or root/jailbreak the device and nerf the pinning section of the code.

Metalnem 10y ago
Yes, certificate pinning would have blocked my initial steps. But there is a tool for disabling certificate pinning validation on jailbroken iOS devices, called iOS SSL Kill Switch https://github.com/iSECPartners/ios-ssl-kill-switch https://github.com/iSECPartners/ios-ssl-kill-switch.

cprecioso 10y ago

Super good article/tutorial, but I think to get the app binary, he could just have downloaded it in iTunes in his computer and unpack the .ipa file (which is basically a signed .zip file, if I remember correctly). The binary should be in it.

robterrell 10y ago
Yep, that still works! Just copy the .ipa from the local ~/Library/iTunes/ subdirectory and decompress it. It would have saved him all the trouble with jailbreaking, finding an old enough device, finding the correct version of iTunes to run against, etc. But I imagine a jailbroken device is its own reward.
- 0x0 10y ago
  But those .ipa files only contain encrypted executables, I believe it's called Fairplay DRM. As far as I know there are no tools available to offline decrypt such binaries, so if you're looking to reverse engineer one, you still need a real jailbroken phone so you can piggy-back on iOS' just-in-time decryption and dump the cleartext binary.
  - Metalnem 10y ago
    You can decompress every .ipa files and see bunch of resources and configuration files in plaintext, which can falsely lead you to believe that you have everything decrypted, but the main binary file is still encrypted, and the only way to decrypt is by running software like Clutch on the jailbroken device.
- hello_asdf 10y ago
  Wouldn't the application still be encrypted then? I thought the benefit of jailbroken in this case, was using Clutch to decrypt it.
  - robterrell 10y ago
    Sorry! Yes, to be clear, the code segments of the binary will be encrypted by FairPlay. But the other segments aren't. The "strings" command will still dump all of the strings in these binaries. You'd need to jailbreak to figure out the algorithm of how those substrings strings were concatenated, but he probably could have deduced what was going on from the "SavedPasswords" string just the same.
    - Metalnem 10y ago
      I thought about trying the "strings" command first, but I didn't expect that the password would be just laying there. Even if I found the "SavedPassword" string, I couldn't use it the way I did without jailbroken device, because configuration file that contained the saved password wasn't embedded in the application. The password was actually written to configuration file on the device first time it was used.

oxguy3 10y ago

Haha, just a few days I made a Dark Horse download script for my own purposes as well (except mine converts to CBZ right in the browser instead of requiring a Go script): https://greasyfork.org/en/scripts/26635-dark-horse-downloader https://greasyfork.org/en/scripts/26635-dark-horse-downloade...

Metalnem 10y ago
So there are more people who read Dark Horse comics and think like me :) Converting into CBZ directly was my initial idea, but then I discovered that there is direct download link to tar archive that contains all the images (I did that by following the network traffic of the iOS app). I didn't want to bother with half-baked JavaScript libraries to extract the tar archive and recompress it into zip, so I just quickly wrote a Go application to do that.

vaveh 10y ago

Interesting and accessible read. But, won't this stop working as soon as they change the pdf password and update the app? Although that's probably a bad idea because all the older versions of the app will stop working.

matheusmoreira 10y ago

Excellent. Wish I could have read this article before I started learning about reverse engineering. I think it's notable how he ended up with a better product: one simple DRM-free file viewable by applications and devices of his choosing. I don't really understand why the company even bothered with all these protection measures. Did they actually believe it would work?

maksa 10y ago
They (righfuly so) count on that that there are not many Metalnems out there.

qubex 10y ago

Almost off-topic: I bought the very first edition (detailing the as-yet-to-be-released 3DO console) in August 1992 and still have the first two years' worth in custom cases.

Leynos 10y ago
Likewise. I plan on hanging onto those. It was the first games magazine I took out a subscription to, and was still a subscriber up until 2010 or so, when I lost interest in games and couldn't justify the continued expense. Having come from the Sinclair Spectrum only a few years ago, by way of the Master System and Super Nintendo, the whole thing was quite eye opening, especially prior to the internet being readily available. I like to think of the years 1993-1995 being a period of irrational optimism, when the technology available just couldn't keep up with the dreams people had for it. This was kind of borne out by the slew of bizarre and not-quite capable consoles and multimedia systems released over these years. The first few years of Edge are the most interesting to read in retrospect because they tell the story of an industry not yet quite sure what it wants to be when it grows up and fighting to get there, wherever "there" might be.
- qubex 10y ago
  Quite inexplicably I wasn't a gamer (I was 11, didn't nor ever had consoles or PC games). I was still fascinated by the technology though.
astrod 10y ago
Those early years up to 2002 when the budget cuts occurred were some of the best video game journalism and industry insight I had come across. It just wasnt the same after the core group of writers and editors left.

apricot 10y ago

This article reminded me of Apple II software deprotection articles published in Computist magazine (available at http://archive.org/details/computist http://archive.org/details/computist ). The way the story is told, with victories that turn out to hide yet another layer of protection, the need to delve into raw hexadecimal, and finally the discovery of a poorly-hidden key, is very much how 8-bit software cracking used to be done in the eighties. Plus ça change...

sbuttgereit 10y ago

I appreciate that this won't be a popular comment, but doesn't this sort of thing fall under the anti-circumvention provisions of the DMCA (or later)? This is not a comment about the rightness/wrongness of copyright law, but a question that seems like it's worth asking.

traverseda 10y ago
I'm not in the USA. Is the author?
- matheusmoreira 10y ago
  The author is in Serbia, according to his GitHub profile.
shmerl 10y ago
It probably does, but it doesn't make such effort less worthy. Anti-circumvention provisions should have existed to begin with, same as DRM.
pbhjpbhj 10y ago
If the user were in the USA then 17 USC 1201 (which I think is at question) says this at Section C(1): >"Nothing in this section shall affect rights, remedies, limitations, or defenses to copyright infringement, including fair use, under this title." (17 USC 1201) // So presumably personal format-shifting is still a fair-use defence that supersedes considerations under this section. FWIW format-shifting was expressly removed from UK Fair Dealing after a challenge by BASCA, presumably no UK musicians rip songs to their MP3 player/phone or whatever. Wish they'd sued Apple Computers for contributory infringement, what with the massive damage iPods have done to people wanting to buy music. /sarcasm
- sbuttgereit 10y ago
  I'm thinking more that employing such circumvention for your own purposes would be hard to argue against, but publishing it publicly may not actually be defensible under U.S. law, even with the fair use exemption. I think if I were thinking of publishing something like this I'd look at the outcome of lawsuits such as those surrounding the old DeCSS code (https://en.wikipedia.org/wiki/Universal_City_Studios,_Inc._v._Reimerdes https://en.wikipedia.org/wiki/Universal_City_Studios,_Inc._v...). I'm in the U.S. so for me something like this would be relevant. For those elsewhere there may still be issues as DMCA is the U.S. implementation of the WIPO (https://en.wikipedia.org/wiki/WIPO_Copyright_Treaty https://en.wikipedia.org/wiki/WIPO_Copyright_Treaty), so there may be issues elsewhere, too.

endgame 10y ago

This is an impressive achievement and a good read, but the shit people have to do to back up things that they bought is just sad.

smilekzs 10y ago

"secure_download_url": "http:\/\/... ugh.

Theodores 10y ago

Maybe a call through to Future Publishing could have been tried first. If you don't ask then you don't get! Of course this would have meant not collecting 'FBI grade' skills along the way, however, picking up the phone, flattering some folks at Future Publishing and getting the goods could be considered 'social engineering' and as good a skill as getting the debugger out.

shmerl 10y ago

Thanks for pointing out Dark Horse downloader.

Hondor 10y ago

Interestingly, instead of a random password, it was leet-speak for "Ford the epic honky".

jccalhoun 10y ago

Very interesting read. I have seen pdfs of Edge on piracy sites for years. I wonder if those are created through a similar method or if they are just screenshots of pages from the official app?

Metalnem 10y ago
That's an interesting question. There is one thing that is unique to the Edge Magazine app on iOS, though. Zip archives of the magazine do not contain just pdf files, but also bunch of high resolution game screenshots. Those screenshots can be seen in the app by clicking the gallery button in a several places. The gallery button link is in the form media://image.jpg, which probably means that the application registered custom protocol handler and handles it by displaying image on top of the magazine itself. That's the way to recognize iOS version, but I'm not sure if Google Play Newsstand has the same feature.

jpalomaki 10y ago

I was quite surprised how difficult it was to get hold of the iOS binary and how this still relied on the application being available for old iOS version.