3 ms·
ArchiveBox Is Super Cool
- Tomte 3y agoIn principle, yes. In practice, I have never been able to make it work, across multiple attempts on both Linux and Windows. It archives. A few hundred URLs. At least some of the archival methods work, some fail a lot. And then it always stops. I‘ve never seen it archive a few thousand URLs, despite having it run for several days. Zero error message, it just stops and seemingly idles,
- nikisweeting 3y agoOut of curiosity, are you using it in Docker or on bare metal? Would be helpful to know the install method, some dependencies have memory leaks or child process killing issues that have caused stalling in the past. We have it running on demo.archivebox.io with tens of thousands of URLs and I haven't had to restart it in months, but different host systems have a lot of variation in stability still. In the meantime this is a stopgap measure to fix the stalling problem. archivebox schedule --every=day --update This will make it pull missing methods once per day.
- Tomte 3y agoI don‘t currently have an instance running, but thank you very much for the workaround! It was the docker composer method.
- nikisweeting 3y agoOk, docker-compose is our recommended method but I'm sorry it didn't work out great for you this time :'( If you feel inclined to try it again in a few months I guarantee we'll have changes pushed to improve stability. I'm working on it full-time these days so progress is more rapid than it's been in the past.