4 ms·

You mean if there is links to external sources? Which is the purpose of hacking actually... Then, how to tell users to include all links which are exceptions?

• rovr138 6y ago
  I don’t know the content strategy for the site, but depending on the site, most link to inner pages. Once you find individual articles, those usually can have external links. This is making a lot of assumptions on the content structure, that’s true. But it would be interesting to analyze. For how to get the links from the database, not for this, but I have used the `urlextract` Python package to find urls. Then you can use `urlparse` to the get the domain. For how to extract this from the database, you’ll have to write some sql to find the tables you want, then you can do a `select * from table`. Then it’s parsing the output, get the urls, get the domain for each. I would probably aggregate all the urls and domains, then at the end actually scan so you can deduplicate entries and not waste time scanning the same domain over and over. - urlextract - https://pypi.org/project/urlextract/ https://pypi.org/project/urlextract/ For the images portion, I’d look into imagemagick compare tool, http://www.imagemagick.org/Usage/compare/ http://www.imagemagick.org/Usage/compare/ You can take a screenshot of a page at the moment you know it’s good, then take daily screenshots. With this tool, you can grab the daily screenshot and compare it with the known good one and the changes are highlighted.
  • Akcium 6y ago
    Oh thank you so much, this is indeed cool feedback. Now I'm considering having many options for hack detecting, screenshots/crawling etc. Especially thanks for the imagemagick. When I researched for such libraries, I haven't found a lot of them, so it's treasure