What makes it unethical? Why should I be treated differently than search engine spiders? If somebody doesn’t want their site scraped then they can let people

What makes it unethical? Why should I be treated differently than search engine spiders? If somebody doesn’t want their site scraped then they can let people know with robots.txt. Get off your high horse.