4 ms·
The reason I made it was I needed to re-trigger a web crawl to fetch fresh data. And I wanted to be able to do it in a way that didn't depend on cron, because I
by dosy 8y ago
The reason I made it was I needed to re-trigger a web crawl to fetch fresh data. And I wanted to be able to do it in a way that didn't depend on cron, because I needed to be able to handle thousands of intervals at all different times and durations, and the system had to keep running even if a part of it broke. So 1 server running cron jobs ( which don't go down to 1 second resolution, only 1 minute ) wasn't going to cut it. I ended up building a it on AWS SQS.