3 ms·
A crawler should be limited by the network not CPU. Outside of the language making it easier to handle multiple concurrent connections, I doubt speed would be m
by 0cf8612b2e1e 4y ago
A crawler should be limited by the network not CPU. Outside of the language making it easier to handle multiple concurrent connections, I doubt speed would be much of a consideration.
- kfrane 4y agoI'm not sure, it might be. If you're not careful, CPU can become bottleneck when you have a 10Gbit network card.