23 ms·
you can use robots.txt to stop the ones that behave. those that don't behave, you would need to cut them off upstream. depending on how you are metered, that mi
by spooneybarger 16y ago
you can use robots.txt to stop the ones that behave.
those that don't behave, you would need to cut them off upstream. depending on how you are metered, that might mean dropping the connection when it gets to your box or it might mean having your hosting provider do it before it gets to your vps.
- yield 16y agothank you very much I use the Nginx as the web server currently. is there some best practice for this？ thank you.
- spooneybarger 16y agoyou want to drop connections either upstream from your machine or before nginx ever connects. assuming that is, you want to block certain connections.