There's a big difference between A. "I'm going to scrape the web to do X, Y, Z" and B. "I'm going to scrape Twitter data to do X, Y, Z" A is far more likely

There's a big difference between A. "I'm going to scrape the web to do X, Y, Z" and B. "I'm going to scrape Twitter data to do X, Y, Z" A is far more likely to care about robots.txt because there's enough left of the web to complete their mission. B will not care if Twitter has a robots.txt file.