3 ms·
Anti-scraping technology is absolutely ridiculous. Modern example: TikTok. Have you seen webmsssdk.js? They simply don't want users to be able to access their
by TobyTheDog123 4y ago
Anti-scraping technology is absolutely ridiculous. Modern example: TikTok. Have you seen webmsssdk.js?
They simply don't want users to be able to access their own data and use it elsewhere in the internet that they can't see, control, and sell.
This is exaggerated by services that login to your accounts to scrape your data for you being illegal under the CFAA.
What a shame for the web.
- jbk 4y ago> Anti-scraping technology is absolutely ridiculous. Modern example: TikTok. Have you seen webmsssdk.js? What is webmssdk doing?
- EdwardDiego 4y agohttps://www.tiktok.com/acrawler/webmssdk.js https://www.tiktok.com/acrawler/webmssdk.js The source is interesting.
- bashonly 4y agofor all of the requests to their api, it constructs a very long query string that is a basic fingerprint of your browser/os. then the query needs to be signed (or else the api returns a captcha), which is done by a blob of encrypted/obfuscated-beyond-recovery JS that uses a more comprehensive browser fingerprint to validate the query string and generate a token that is appended to the query. there's another required query param which involves an xhr to phone home so that presumably the fingerprints can be checked server-side. finally all of your fingerprint data is sent to their api where it lives happily ever after and you get some 15 second videos to watch
- ssizn 4y ago>They simply don't want users to be able to access their own data and use it elsewhere in the internet that they can't see, control, and sell. That’s false, you can download a copy of your own data from settings.
- fariszr 4y agoThere is an OSS frontend for Tiktok. I added ARM support to it. https://github.com/pablouser1/ProxiTok https://github.com/pablouser1/ProxiTok