Without going too detailed into this ... Creating a domain-specific model can be done a million ways, obviously, but the nice thing about HTML is the markup gi

Without going too detailed into this ... Creating a domain-specific model can be done a million ways, obviously, but the nice thing about HTML is the markup gives hints all along the way. The tree, the class names, the tags, etc. Coupled with the content of each tag, it's absolutely possible to determine collections of items with metadata using ML. But like most ML problems, the underlying data that feeds the model is the time-consuming part. If you are working in a single or few domains, I 100% recommend this approach. If you're scraping something far more generalized, first you need to have models that you care about and then you need to create models to determine the content type of what you're scraper is looking at. 1) What kind of content do I have? 2) Does this match a known domain with a model? 3) Apply appropriate model to domain, hopefully extract correct data Another huge issue is, of course, validation, because you're going to be dealing with an inordinate amount of unknown and unpredictable data depending on what you're looking at.