xena

12 ms·

301.
by xena 2y ago
OMG lol I forgot that I left that in. Hilarious. I think I'm gonna keep it.
302.
by xena 2y ago
Do you respect robots.txt so administrators can block this tool?
303.
by xena 2y ago
Grep is a tool. This is a service.
304.
by xena 2y ago
How do I opt-out for my repos?
305.
by xena 2y ago
Yes, but I'm not going to implement that to avoid the implementation "leaking" and ossifying the current transitionary hack.
306.
by xena 2y ago
I'm looking at using equi-x, but failing that I may unironically do protein folding.
307.
by xena 2y ago
SHA-256 is a hack to buy time. This will be replaced with something better. It would be faster for me to replace it if I didn't have to do my dayjob: https://patreon.com/cadey
308.
by xena 2y ago
It's luck-based, I'm working on making a check that's more deterministic, but I'm also trying to figure out how to not lock out big-endian systems in the process. I may have to just give up on that though :(
309.
by xena 2y ago
This is absolutely surreal to see in action! I hope that I can manage to afford to not have to do my dayjob anymore.
310.
by xena 2y ago
I'm working on scraper bot filtering software! https://github.com/TecharoHQ/anubis . This week I'm going to publish the plans for building binary packages (both distribution-specific packages and tarballs full
311.
by xena 2y ago
TBH as someone with a lot of experience in this (I plan to write more when I get to 69 posts on the front page of hacker news, I'm only at 55 right now), about half of it is that you have to be personally jazzed about the post. If you&
312.
Impromptu Disaster Recovery(fasterthanli.me)
2 points by xena 2y ago|0 comments
313.
by xena 2y ago
But if the AI crawlers are taking the website down and money buys more server time, are you willing to do your part and use money to make sure your training data sources are solvent until you can replace them?
314.
by xena 2y ago
What does the user agent oook like for if you wanted to crawl xeiaso.net?
315.
by xena 2y ago
Thanks! The artist I'm contracting and I are in discussions on how to make the mascot better. It will be improved. And more Canadian.
316.
by xena 2y ago
So you're willing to pay my hosting bills?
317.
by xena 2y ago
I'm going to be making distro packages and binaries public. I will have to figure out white label monetization I guess.
318.
by xena 2y ago
You'd think, but no :(
319.
by xena 2y ago
What button do I click in the AWS panel for that?
320.
by xena 2y ago
You can detect the patterns in aggregate. You can't detect it easily at an individual request level.
321.
by xena 2y ago
Wow it is so surreal to see a project of mine on Ars Technica! It's such an honor!
322.
The surreal joy of having an overprovisioned homelab(xeiaso.net)
17 points by xena 2y ago|2 comments
323.
by xena 2y ago
Training attempts != training data
324.
by xena 2y ago
Do you publish an IP range?
325.
by xena 2y ago
Is there support for robots.txt so service operators can opt out of your mass scraping?
326.
by xena 2y ago
> I think Anubis is a good first start, but the fact that it uses a fixed size stamp, a small one, does not have a reputation database, and from what I can tell, does not have a robust detector of good versus bad players. These shortcomi
327.
by xena 2y ago
Probably, but if they do that, then that means that I win because I forced them to make changes to their configuration and code. It means that I would have successfully caused them to waste their time. It would be pretty fun to see that hap
328.
by xena 2y ago
It breaks websites when you do that.
329.
by xena 2y ago
Git forges are some of the worst case for this. The scrapers click on every link on every page. If you do this to a git forge, it gets very O(scary) very fast because you have to look at data that is not frequently looked at and will NOT be
330.
by xena 2y ago
Noted! I filed a bug: https://github.com/TecharoHQ/anubis/issues/38 All of this is placeholder wording, layouts, CSS, and more. It'll be fixed in time. This is teething pain that I will get through.

More ›