4 ms·
Well I'm winning than. I've written so much useless software, and projects, and starts of projects. It's all fun and joy until you come back to it 1 or 1.5 yea
by winddude 3y ago
Well I'm winning than. I've written so much useless software, and projects, and starts of projects.
It's all fun and joy until you come back to it 1 or 1.5 years later, and resources are now throttled because they're so over used because of the explosion in AI. The fun seems to go away when you come back to an old project, and things have changed.
- shnock 3y agoWhat kinds of resources?
- winddude 3y agocommoncrawl at the moment. But also used to have some stuff using the old reddit (through pushshift) and twitter apis. Plus some old classified crawlers, where sites have taken a more proactive approach.
- syndicatedjelly 3y ago> It's all fun and joy until you come back to it 1 or 1.5 years later, and resources are now throttled because they're so over used because of the explosion in AI. What does this mean?
- winddude 3y agoNo yesterday, I just tried to spin up an old project using common crawl, which is hosted on AWS open data, seems like it's gotten quite popular since I lasted worked on it, and they're experiencing throttling from over usage, maybe even ddos (or just a really bad developer). But seems to be a pretty common thing with stuff related to data and crawling. Everyone is cracking down on crawlers, adding paywalls, removing previously free APIs, so coming back to old projects in that domain seems to be impossible.
- syndicatedjelly 3y agoWhat's the lesson to be learned from that experience?
- winddude 3y agoFinish something and exploit (I mean market) the shit out of it. Or find a lot more money somewhere and build a lot more from scratch.