robots.txt actually has little to nothing to do with copyright, and more to do with "unauthorized computer access" which is protected by the CPAA. It's a guard

robots.txt actually has little to nothing to do with copyright, and more to do with "unauthorized computer access" which is protected by the CPAA. It's a guard against accidental access or publishing of sensitive data - which isn't necessarily copyrighted but has other legal protections - as well as placing an unwanted burden on the server.