3 ms·
Shameless appeal for code commentary review: This reminds me of a code comment I wrote about some web crawling code that is intended to maintain a continuous w
by jka 4y ago
Shameless appeal for code commentary review:
This reminds me of a code comment I wrote about some web crawling code that is intended to maintain a continuous web address "flow graph", matching any origin URL(s) (the 'water table' of a web page) down to the current location of that page on the web (the 'river delta').
This can be complicated because old URLs may be redirected (HTTP 301, 302) to different locations over time, often for SEO reasons or due to change of domain ownership.
Does the code comment linked below make sense? I'd really appreciate any feedback and improvements:
https://github.com/openculinary/backend/blob/5116c4f5d39dae1f992836c28041722b9893d7de/reciperadar/workers/recipes.py#L60-L116 https://github.com/openculinary/backend/blob/5116c4f5d39dae1...
(and yes, I realize the ASCII art here pales in comparison to some of the visually appealing and clear diagrams shared in the article. I'm doing my best :))