nikisweeting

6 ms·

91.
by nikisweeting 1y ago
Are you using a browser provider or building your own CUA scraping pipelines in-house? What's your strategy around CAPTCHAs, paywalls, rate-limits, etc.?
92.
by nikisweeting 1y ago
Yeah I agree, I changed the history section a bit: https://github.com/browser-use/browser-use/blob/2a0f4bd93a43...
93.
by nikisweeting 1y ago
Gregor was saying it works without needing playwright, and provides more detailed trace recordings than playwright does. we plan to use rr-web and maybe browsertrix for our website archival / replay system for deterministic evals.
94.
by nikisweeting 1y ago
Yes the key is we don't intercept requests and responses, that saves 60% of the headache of lifecycle management. We do exactly what you described with a 1+1 check in a loop for every target, it pops any crashed sessions from the pool,
95.
by nikisweeting 1y ago
I tweaked the article text a bit, if anyone has more history to fill in I'd love to collect browser automation lore! https://github.com/browser-use/website/blob/main/posts/playw...
96.
by nikisweeting 1y ago
Yeah I started building this in my first week at the company haha: https://github.com/browser-use/desktop Shipping a whole electron app is not a priority at the moment though, our revenue comes from cloud API users, an
97.
by nikisweeting 1y ago
iirc it's stable across sessions until the tab closes, even though their docs dont guarantee it. we cant modify the dom to add IDs because we'd get detected by block-blockers very quickly. we're gradually trying to get rid of
98.
by nikisweeting 1y ago
No we are not planning to support Firefox. We do support Brave, Edge, and ungoogled-chromium though if you have a problem with Google.
99.
by nikisweeting 1y ago
Chromium bug tracker is where issues go to die, but aside from that I've had nothing but lovely individual interactions with core chrome devs so far. The devtools frontend/protocol repo is definitely active and more approachable t
100.
by nikisweeting 1y ago
it does when we have multiple LLMs working in parallel on a single tab, which we're working towards eventually
101.
by nikisweeting 1y ago
you cannot get the list of bound onclick handlers with a snapshot
102.
by nikisweeting 1y ago
you still need separate calls to get the AXTree with full computed aria properties, and a bunch of Runtime.evaluate calls to scan all the dynamically-added event listeners.
103.
by nikisweeting 1y ago
Extensions are ok but they have limitations too, for example you cannot use extensions to automate other extensions. We need the agent to be able to drive 1password, Privacy.com, etc. to request per-task credentials, change adblock settings
104.
by nikisweeting 1y ago
I've been trying to contribute to playwright for years! All of my issues have been closed / rejected without much consideration because they're not part of the core "QA testing" use-case that playwright is built for
105.
by nikisweeting 1y ago
yeah but good luck getting rid of that with a browser extension, you're just moving the latency around / moving it to chrome.runtime message passing.
106.
by nikisweeting 1y ago
I would definitely recommend puppeteer if you can, it's maintained by the Chrome team and always does things the "approved way". The only reason we did playwright is because we're a python library and pyppeteer was aband
107.
by nikisweeting 1y ago
our use primary use-case with the AI stuff is not really scraping, we're mostly going after RPA
108.
by nikisweeting 1y ago
What do you mean? We use CDP page snapshots extensively to get full html across frames but it's not nearly enough on its own, there are lots of checks still needed for individual OOPIFs or elements.
109.
by nikisweeting 1y ago
https://github.com/rrweb-io/rrweb
110.
by nikisweeting 1y ago
yes I know, I wrote the post
111.
by nikisweeting 1y ago
whats the downside of using frameId/targetId+backendNodeId as the stable element ids?
112.
by nikisweeting 1y ago
It was, but I feel like the advent of headless browsers marked a step function explosion in browser automation. Also any earlier than 2010 is when I was like 13yo, so it's more like "the dark ages in my own memory" than &quot
113.
by nikisweeting 1y ago
Wow hi! Thanks so much for building selenium! I've used it many times in my career, and I looked at Selenium Grid for inspiration for browser devops in my last job.
114.
by nikisweeting 1y ago
CDP rountrip time on a local machine is 100µs (0.1ms), it's not slow haha
115.
by nikisweeting 1y ago
sir we are a python library, puppeteer-python was abandoned, how exactly do you propose we use puppeteer?
116.
by nikisweeting 1y ago
Here's the worst one I've seen so far, it's showing one for Quora on the Chrome settings page! https://x.com/thesquashSH/status/1788553660357915017
117.
by nikisweeting 1y ago
> the banners are never displayed in Google Chrome! this is demonstrably false, I can share dozens of screenshots of this popup appearing in Chrome despite all the options checked that are supposed to disable it.
118.
by nikisweeting 1y ago
This option never worked for me, I also tried the options under myaccount.google.com with no luck, I kept getting the popups and ended up having to switch to Brave and Safari to get rid of them.
119.
by nikisweeting 1y ago
100% the implementation is setting a terrible precedent to trust non-browser-native-looking UI for whats supposed to be a browser-level feature.
120.
by nikisweeting 1y ago
ah ok, thanks

More ›