4 ms·
you get into an armsrace where twitter will obfuscate the markup to just wear out the scraper devs. which they can, because $$$
by black_puppydog 4y ago
you get into an armsrace where twitter will obfuscate the markup to just wear out the scraper devs. which they can, because $$$
- mrtksn 4y agoThen just OCR the rendered output. It's cheap and easy these days.
- mlatu 4y agoi heard it works best on iphones, best of luck
- mrtksn 4y agoApple’s vision framework is actually very good, so effective on-device extraction is really just a few lines of code on iPhone(all Apple device tbh).
- tsimionescu 4y agoGood luck OCR-ing images and videos, though. Twitter isn't just text.
- mrtksn 4y agoWhy do I need to OCR images and videos? Why not display those as-is?
- tsimionescu 4y agoWhich one do you show for which tweet, if all you have is the OCR (that is, you can't read the markup)?
- mrtksn 4y agoApple's vision framework is not limited to text, can detect shapes and other things. Twitter WebUI loads images and videos when you scroll. Which means you have a good signal what image and video is associated with what tweet.
- actualwitch 4y agoDoing this requires a fair amount of development time, especially when you can break something else with this and you previously removed all the guardrails.
- nkozyra 4y agoThey don't do this when they can just disable the app without being logged in.