I have a background in adtech and search engines. I can tell you unequivocally that any specifier to deny AI scanning is likely to be ignored (just like how rob

I have a background in adtech and search engines. I can tell you unequivocally that any specifier to deny AI scanning is likely to be ignored (just like how robots.txt is often ignored). Not only is there value to be extracted but is very difficult to prove that AI knowledge was built using a particular resource. Some people have suggested that online resources could be given a "tracking data atom" that could be used to determine if an AI did learn from a blocked site. However, I'm skeptical if such trackers couldn't be easily worked around. Tldr: robots.txt conventions are just not going to work.