Usually the approach is to use a headless browser. The headless browser instance runs purely in memory without a GUI then renders the website you're interested

Usually the approach is to use a headless browser. The headless browser instance runs purely in memory without a GUI then renders the website you're interested in. Then, it comes down to regular DOM parsing. A common library that I enjoy is Selenium with Python.