I really like the options of node + cheerio, or node + puppeteer for these kinds of jobs. Language of the browser, easy enough selection/usage, cheerio if it's

I really like the options of node + cheerio, or node + puppeteer for these kinds of jobs. Language of the browser, easy enough selection/usage, cheerio if it's plain html, puppeteer if it's dynamic/spa content. In the end, not hammering a site is key... also, on your own end, possibly hashing the main content, so you aren't creating duplicate entries on your own side are important. As to fragility, that happens... in general, you need to update to match site updates, but most sites won't be dramatically updated more than a couple times a year if they aren't in active development.