I think this one is a little cleaner. I used some of the ideas in sbr464's code. path = 'favorites?id=app4soft' while path: r = requests.get('h

I think this one is a little cleaner. I used some of the ideas in sbr464's code. path = 'favorites?id=app4soft' while path: r = requests.get('https://news.ycombinator.com/' + path) s = BeautifulSoup(r.text, 'html.parser') print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]) more = s.select_one('a.morelink') path = more['href'] if more else None