3 ms·
You could always just have an option for it. I feel people would trust your tool more if they are able to tell it what to ignore.
by dangoldin 18y ago
You could always just have an option for it.
I feel people would trust your tool more if they are able to tell it what to ignore.
- zemaj 18y agoThat was the intention of choosing CSS selectors over robots.txt or nofollow, although a checkbox to specify following robots.txt & nofollow would be useful as you say.