3 ms·Analyzing One Million Robots.txt Files2 points by foob 9y agoPaulHoule 9y agoWhere is the insight? You have a bunch of clusters, but what do you actually do with the clusters once you have them?