4 ms·
I wonder if using something like puppeteer or playwright to actually make the server think everything is being done client-side would still raise flags.
by theturtletalks 2y ago
I wonder if using something like puppeteer or playwright to actually make the server think everything is being done client-side would still raise flags.
- pyrale 2y agoScraping a well-built API at human speed often isn't terribly useful, and once you start ramping up the scraping, it's account creation/patterns/use frequency that will set an alarm. Faking real user clients won't prevent these alarms.
- danielvaughn 2y agoThe purpose of our work wasn’t scraping - we were building a unified UI where our users could trade with any vendor of their choosing. Kinda like Plaid but specifically for retail stock trading. So the goal was to implement their trading API.
- MarcelOlsz 2y agoYou can just.. do that? I think I discovered why all my ideas suck.
- danielvaughn 2y agoYep. Though it’s really hard. To capture the US market (E*Trade, Fidelity, TD Ameritrade, Scottrade, Schwab, Interactive Brokers, Robinhood) it took me and another engineer almost 2 years. It’s non-trivial.