2 ms·

I'm still confused about the use case. Is the purpose of a cache like this to create a unified interface to accessing and storing web pages? So you can write co

I'm still confused about the use case. Is the purpose of a cache like this to create a unified interface to accessing and storing web pages? So you can write code to scrape pages based on their URLs, but then you can re-run without changing your code using local cached data?

garysieling 10y ago
It's possible that the proxy caches duplicate requests, e.g. if you're crawling articles linked on reddit.