2 ms·
As I understand the point about Go complaint is to parse actual real world robots.txt. For which you don't need to behave exactly as this library does.
by zzzcpan 7y ago
As I understand the point about Go complaint is to parse actual real world robots.txt. For which you don't need to behave exactly as this library does.