If performance, Especially concurrency matters then we should include Go based libraries in the discussion as well. Colly[1] is a all batteries included scrapp

If performance, Especially concurrency matters then we should include Go based libraries in the discussion as well. Colly[1] is a all batteries included scrapping library, But could be bit intimidating for someone new to HTML scrapping. rod [2] is a browser automation library based on DevTools Protocol which adheres to Go'ish way of doing things and so it's very intuitive but comes with overhead of running chromium browser even if its headless. [1] https://github.com/gocolly/colly https://github.com/gocolly/colly [2] https://github.com/go-rod/rod https://github.com/go-rod/rod