4 ms·

Is there a certain time interval at which you scrape for data? for example - get data every x seconds. What is the 'x' here?

• visox 5y ago
  The pause is 24 hours. Once running it wont stop till there are urls with some pattern to crawl, there is also a limit to stop when enough pages didnt have an app
  • the_parzival 5y ago
    That's neat. Great project
    • visox 5y ago
      thx :) but i guess its not high quality enough.
      • the_parzival 5y ago
        Looks great to me. Why do you think it is not of high quality?