The comments here focus on various AI/ML ways of classifying websites. That's surely part of a solution, but I think manual classification is still important. I

The comments here focus on various AI/ML ways of classifying websites. That's surely part of a solution, but I think manual classification is still important. If you're making a public facing website, try to crowd-source the manual classification if you can't classify pages yourself. At present, AI/ML classification probably doesn't work anywhere near as well as you all think it does, which became obvious to me when I worked as a patent examiner in the past and used some of the many AI/ML search tools. These tools are quite good at finding somewhat similar things, but will usually miss things that I'd consider very similar. And when I was a patent examiner, "somewhat similar" wasn't good enough. Many people looking for similar websites won't be satisfied with a list of somewhat similar pages if a very similar page exists in the database.