Depending on the use case you might try imaging the page, then send the image to an ML model for full text before indexing. If you need links extracted, Seleniu

Depending on the use case you might try imaging the page, then send the image to an ML model for full text before indexing. If you need links extracted, Selenium also supports parsing the assembled DOM: https://github.com/kordless/grub-2.0/tree/main/aperture https://github.com/kordless/grub-2.0/tree/main/aperture