4 ms·

How long does it take to run your scraping pipeline?

• james_dev_123 2y ago
  End-to-end it takes ~10 hours. The biggest bottleneck is the actual scraping itself. The GPT part is relatively quick.