3 ms·
Good point, we should probably integrate that. Feel free to submit a PR! BLAST can also be used to add automation to your own site/app FWIW.
by calebhwin 1y ago
Good point, we should probably integrate that. Feel free to submit a PR!
BLAST can also be used to add automation to your own site/app FWIW.
- dejobaan 1y ago> Feel free to submit a PR! I think it's cool that you're experimenting in this area, but I'm not a huge fan of this as answer to a question about responsible/respectful web crawling. This stuff seems like it should be table stakes (even if you wanted to make it optional for the end user), but "yeah probably; learn the codebase, fork it, make changes, then we'll review it" really puts the onus onto the original poster.
- lostmsu 1y agoCurious: if a user has an ad blocker, are they browsing responsibly?
- subscribed 1y agoVery. Malvertising is a thing. Adtech surveillance is a thing. Ad blocker is the least user can do.
- lostmsu 1y agoThat's not why most users use ad blockers.
- calebhwin 1y agoAh you're right, my bad. Hope I didn't sound dismissive because I think some sort of robots.txt needs to exist for AI that's scraping the web both at train or test time. I'm really not excited at all about the "scrape other people's data" use case for BLAST and if we can prevent it then awesome. I'm excited about BLAST automating science, legacy web apps, internal tools, adding AI automation to your own app, etc.
- xena 1y agoI will for my typical hourly rate plus the consulting bonus.
- BrenBarn 1y agoFeel free to take down your project until you comply with basic ethical standards.