robots.txt as of right now is a complete honor system, so I think it's reasonable to make a conclusion that you shouldn't rely on it protecting you because odds

robots.txt as of right now is a complete honor system, so I think it's reasonable to make a conclusion that you shouldn't rely on it protecting you because odds are overwhelming that scraping behavior will become worse in the near to mid term future