3 ms·

Thanks! One more question: what is the best way stop it when it will reaches last page? > for p in range(1, 17): Actually p=17[0] is empty (as p=16 is maximu

• gabrielsroka 7y ago
  In Python, `range(1, 17)` produces the numbers 1-16. I hard-coded it just for your favorites. A better way to solve it is to look at the `len()` of the results, and stop when it gets to 0: p = 1 while True: r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}') s = BeautifulSoup(r.text, 'html.parser') faves = [{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')] print(faves) p += 1 if len(faves) == 0: break
  • app4soft 7y ago
    Great!
    • gabrielsroka 7y ago
      I think this one is a little cleaner. I used some of the ideas in sbr464's code. path = 'favorites?id=app4soft' while path: r = requests.get('https://news.ycombinator.com/' + path) s = BeautifulSoup(r.text, 'html.parser') print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]) more = s.select_one('a.morelink') path = more['href'] if more else None
      • app4soft 7y ago
        Go on! ;)