19 ms·
Reverse Engineering TikTok's VM Obfuscation (Part 2)
- mrsaint 4y agoHow could Apple properly review something like this? Isn't it one of Apple's selling pitches that they'd review each app for malicious activity before it makes it to the app store?
- perttir 4y agoI used to develop Apache Cordova application that had strong obfuscation using javascript-obfuscator. Apple didn't care.
- valleyer 4y agoSo, a tricky piece here is that this appears to be behavior of the TikTok web site. Obviously Apple makes no attempt (nor claim) to review the behavior of every web site accessible in Safari from an iPhone. And other native apps can embed WebKit-based web views into their apps. The good news is that the scope of "malicious activity" is (at least in theory) much smaller when you constrain it to what web sites can do, as opposed to the scope of what can be done by executing ARM instructions and making syscalls. The bad news is that the scope of "things web sites can do" keeps growing and is fingerprintable.
- emsy 4y agoApple has previously banned Apps for their backend content if they didn't like it. It's just that TikTok is too big and Apple is full of shit.
- lazyeye 4y agoApple only cares "really deeply" about ethics where there is no financial penalty attached. China would be an obvious exception to this. https://www.theinformation.com/articles/facing-hostile-chinese-authorities-apple-ceo-signed-275-billion-deal-with-them https://www.theinformation.com/articles/facing-hostile-chine... Note: This is the same as having no ethics.
- angulardragon03 4y ago> the code that is deployed on TikTok's _website_ This isn't regarding the app at all, which is likely not as heavily obfuscated as this (mostly because you can't just "view source" on an app).
- pjmlp 4y agoThey can't and most likely would kick the app out of the store, hence why this is the Website code.
- Mindwipe 4y ago> How could Apple properly review something like this? Isn't it one of Apple's selling pitches that they'd review each app for malicious activity before it makes it to the app store? They couldn't. Apple does not perform any meaningful review of apps for malicious activity, do they do it for rent seeking.
- deleted 4y ago[deleted]
- sposeray 4y ago[dead]
- CGamesPlay 4y agoNote: this is not a continuation of the work by the same author. This is a new author who took the original and did further research. I think it's a bit disingenuous to call this by the same name, "part 2".
- vore 4y agoIf Brandon Sanderson is allowed to finish The Wheel of Time, this author is allowed to call it part 2 ;-)
- CGamesPlay 4y agoYeah, but that was 2 years after the original author died. This is a month after the original publication. I just feel it would have been more tasteful to choose a title different than the one that the original author is obviously going to use for their next blog post.
- jeroenhd 4y agoI think it's actually more truthful to admit that this is the continuation of an earlier post than it is to say that this is new work. This is just the continuation of the process the first post documented. The relation between this article and the one it's based upon is clearly indicated in the first paragraph. I don't think this is disingenuous at all.
- deleted 4y ago[deleted]
- kayson 4y agoWhat's the point of such heavy obfuscation? Are they afraid of someone cloning their frontend? At first glance it seems like a waste of performance...
- TobyTheDog123 4y agoMost likely bot prevention & data security. There's not a huge performance hit, there's a portion that runs on page load and a smaller portion that runs on each HTTP request (which isn't too often).
- jeroenhd 4y agoBots just run copies of Chrome these days, you're not protecting anyone with this obfuscation if the bots you're targeting can just run your code and automate the UI. With the obfuscated fingerprinting demonstrated by the virtualized code, I reckon they're doing something more malicious.
- charcircuit 4y agoYou answered it yourself. They are trying to fingerprint bots and to avoid people figuring out how to beat the fingerprint they obfuscate the code.
- lwansbrough 4y agoIn the first post about this, it was revealed they're exfiltrating detailed metrics covertly by piggybacking on regular requests.
- sweetjuly 4y agoYes, but that's not the only thing running under the VM--the entire user client is in there. This is not proof that they're evil, in fact it's identical to an anti-boting technique that was just on the front page yesterday [1]. [1] https://news.ycombinator.com/item?id=34285747 https://news.ycombinator.com/item?id=34285747
- jeroenhd 4y agoInteresting! I wonder, would this virtual machine be hand crafted or auto generated? When I look at obfuscated code like this, I always wonder if the code authors couldn't run their generator a second time and come up with a completely new format that existing reverse engineering efforts wouldn't work with? I don't know what they're trying to obfuscate but it must be worth hiding to allow such inefficient javascript to run on clients around the world. I can't think of any non malicious reason to develop such a system for a website about silly videos.
- TheDong 4y ago> I can't think of any non malicious reason to develop such a system for a website about silly videos. Let me give you a few examples of cases where obfuscated or inefficient code exists: 1. re-captcha is "obfuscated", and so difficult that computers aren't meant to be able to compute it. It is used to rate-limit login attempts to secure people's accounts, among many other uses. 2. Cloudflare runs some obfuscated "are you a human" javascript to fingerprint you for the purpose of DDoS protection. This one is arguably less noble, but still not obviously malicious. 3. Games obfuscate code all the time to make cheats harder to develop and maintain 4. Youtube, netflix, etc have obfuscated code (DRM code) because legally they have to make an attempt to protect copyrighted works from being downloaded etc, or else they lose access to said copyrighted works. Tiktok also allows using copyrighted music I'll point out. Depending on your viewpoint, perhaps all of those are also malicious, but I think many people view them as non-malicious, though perhaps dumb. That said, I personally wouldn't give tiktok the benefit of the doubt here. All the other large social media companies maliciously capture as much data as they can about me in order to sell it to brainwashing companies (ahem, advertisers), to the point where I think it should be illegal, so I don't really expect tiktok to be any different.
- jeroenhd 4y agoI'd consider all of these to be "dual use" technologies, in a way; I appreciate recaptcha's functionality, but dislike that the data is being collected by data sponges like Google. Cloudflare has a better track record but they also have much more control, so I'm split on their use of this tech. I accept DRM for online games where one modified client can ruin the fun for everyone else, but this isn't a multilayer game. It's a video player with comments. As for DRM, that's just plain malicious in my eyes. I need to go through the effort of pirating Netflix content I pay for because my OS isn't in the magical white list. It's not as if DRM is preventing piracy either, because pirated content in 1080p is available the day a show goes live on Netflix, sometimes even earlier because of bullshit International contracts. There's no legal requirement for their DRM either, that's all part of the contracts they sign with media companies who are just as responsible. I don't think Tiktok cares about copyright law in the slightest, as far as I can tell they just started using copyrighted music and dealt with license deals after the fact. Bytedance knows the MPAA isn't going to get the app banned, not without running the risk of American artists losing the lucrative Chinese market, so I very much doubt that they'll use this for any kind of DRM. Perhaps this is just an in house recaptcha alternative or a way to block youtube-dl, but so far I haven't seen this stop redistribution of the videos on their platform in any meaningful way. Tiktok videos get downloaded all the time and even youtube-dl has support built in these days so I don't see what it's trying to accomplish.
- TobyTheDog123 4y agoI, too, was disappointed that this was not a continuation of https://www.nullpt.rs/reverse-engineering-tiktok-vm-1 https://www.nullpt.rs/reverse-engineering-tiktok-vm-1, but as someone who could really use a way to interface with TikTok's API (for legitimate non-bot reasons that allow users to interface with TikTok differently), I'm all for more eyes on this problem.
- slimebot 4y agoYeah I'd also like to understand why they are doing this. Everyone expects these sites to scrape as much personal information as possible (China did not invent that, they are following), but beyond that any additional imagined state-ran initiative would be server side, right? What is worth hiding in the front end beyond preventing people re-using their code? (which would be overkill to use a VM for, as light obfuscation would be enough)
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- captainmuon 4y agoIt's sad how much human effort is spend in creating obfuscation schemes, and then in analyzing them. Or creating proprietary things and then reverse engineering them. Sometimes I wonder if we could just make everything open. No obfuscation, no captchas, just a neat API for everything. Of course that wouldn't work ceteris paribus, everything else unchanged, due to bad actors, spam, or just competitors who want to take your work. But if you'd change the incentives - make society non-adversarial, non-profit oriented - then all that gating and obfuscation would become unneccessary.
- PradeetPatel 4y agoSadly we do not live in a post-scarcity world, competiting for resources in an adversarial manner is the driving force behind most individuals, corporations, and on a larger scale - nation states.
- gleenn 4y agoAlso, in a slightly better light, people don't actually know what "best" is. Sure, there is a ton of ugly competition, but there is also some good forced exploration out of that to try alternatives which frequently can arrive at higher global maxima.
- PUSH_AX 4y agoThis actually branches into a fascinating discussion about privacy. Arguably the world would be a much better place if corporations, governments and people were both unable and unwilling to keep secrets. Although, again, with all things being equal, good luck finding people to "go first".
- deleted 4y ago[deleted]
- vasco 4y agoYou could say the same thing about locks, police, military. Those are way more wasteful and also just exist due to bad actors. I don't think there's a way around this, it'd be like wishing for no predators in nature so that natural evolution could just do the "good parts" and not have to optimise for survival.
- jonatron 4y agodynamic string extraction: https://gist.github.com/jonatron/f7ec44e7ffd41c4dd50d51b3451a56f5 https://gist.github.com/jonatron/f7ec44e7ffd41c4dd50d51b3451...
- sylware 4y agoAnd then we can use a small software to view tiktok videos, without the need of one of the vanguard/blackrock financed, absurdely and grotesquely massive and complex, web engines (blink/geeko/webkit).
- cactusplant7374 4y agoAt the business / management level, who decides to green light a project like this and what are their reasons? Is it regulatory obscurity or denying the competition easy understanding of the code?
- steponlego 4y agoThis is pretty cool. Imagine what Tik Tok could do from a kid's phone when their parent is working on their laptop, on the LAN.
- snazz 4y agoiOS requires user permission for each app to make local network requests (Settings -> Privacy & Security). So in theory this shouldn’t be possible, unless ByteDance came up with a way to bypass it that made it through Apple’s review process.
- steponlego 4y agoYou could log into any OS X machine by just pressing enter at the login screen enough times, for over a decade. Apple has trash security and they've stopped even saying what their security updates fix.
- brookst 4y agoThat’s not a convincing argument that this VM somehow reflects a LAN based sandbox escape attack.
- steponlego 4y agoIt's just a reflection on how Apple's security can't be trusted, regardless of the possible vector. Big companies like Apple need vulnerabilities for deniable intelligence and national security access, but anybody can find and use these to achieve a breach.
- winterqt 4y agoFWIW: 1. That wasn't a bug for "over a decade," as far as I'm aware. It was only introduced with an OS update 2. I could be wrong here, but it required you to already have created a root user with a blank password, which isn't possible unless you know how to trigger another bug that does it.
- 4y ago
- kerneloops 4y ago> Using these accessors, the VMs become able to do anything that JS can do. In fact the source language is likely to be JS itself; a JS-to-some-sort-of-vm-bytecode-to-JS compiler is made. I know that Tencent has a similar VM; an interesting aspect of that VM is that the instruction set is dependent on the code being compiled (and the opcodes are dynamically generated and shuffled when compiling), so unused instructions are not generated.
- nullpt_rs 4y agoNice work. I was going to basically cover the same topics in my second part but it looks like you beat me to it. If you'd like to collaborate with me on the next portion feel free to contact me on Discord (veritas#0001) or email (f@nullpt.rs)
- cute_boi 4y agoHmm, can you clarify how did you manage to send request in your original article? I copied as curl from network tab and tried to sent the request from curl. However, there was no response. Are they using fingerprints or cookies shenanigans?
- umasi 4y agoAs others have already touched on, it’s rather disingenuous to pretend that you are “picking up where he left off” after copying his title and labeling your article as part 2, no? Especially considering the first author made it very clear he intended to publish further work on the topic. Further, this article reads a lot like an attempt to beat him to the punch and to hijack his series. But correct me if I’m wrong?
- laptou 4y agoYou're right that copying their title exactly was probably not the right thing to do, and may cause confusion. I'm not trying to hijack anything, I think there's enough room for both of us to research this at the same time.
- umasi 4y agoThanks for your response, seems reasonable to me. That being said, this post is at best impolite, and at worse intentionally "scooping" another reverse engineer. Seems like you guys are chatting now though, so I'll leave you to figure it out.
- deleted 4y ago[deleted]
- ActionHank 4y agoInstead of writing a custom decompiler it might be a bit quicker to pose as a question to chat gpt
- yayr 4y agois there any open source reference implementation for such a vm that is dynamically generated based on JS/TS?
- ghostly_s 4y agoI skimmed the original post but still don't really grasp how these VMs are deployed. I assume they are running server-side (client-side would be a violation of App Store policies at least, right)? Is a dedicated VM spun up for each user session? Or just a sandbox in which various services run?
- kaba0 4y agowhile (hasData) { switch (code) { case INSTR1 -> …, case INSTR2 -> … … } }
- ralphc 4y agoCan someone explain why TikTok needs a virtual machine? What are they doing with it that can't be done with a normal web app?
- KiwiJohnno 4y agoThey are doing it to obfuscate the capabilities and usage of their app. Yes, this should set of big warning bells.
- The_Link 4y agoAs a thought exercise Botter: attempts to spin up phone vm for botting Okay tiktok start up TikTok: What hardware am I on? Botter: insert common phone hardware here TikTok: Okay, than this should work hyper firmware/bytecode specific virtualization commands and syscalls (segfault)
- tinus_hn 4y agoApps can’t run unsigned code on iOS, everything has to be in the bundle submitted for review. That’s why you can’t have a third party JIT compiler.