found this lib in the comments, tried it and looks nice. can be good for finding the xpath automatically and save the manual traversing overhead. https://github

found this lib in the comments, tried it and looks nice. can be good for finding the xpath automatically and save the manual traversing overhead. https://github.com/alirezamika/autoscraper https://github.com/alirezamika/autoscraper