4 ms·
Not all crawler have advertising giants with billions of dollars worth of resources behind them. Some are just curl or similar things. If you want to participa
by ttepasse 12y ago
Not all crawler have advertising giants with billions of dollars worth of resources behind them. Some are just curl or similar things.
If you want to participate in low-level crawling like for microformats/indieweb or similar things there is an advantage for static pages: you participate in a more democratic web not in one which just works for the big stacks.