A big aspect is that for everyone working on those competition systems, making an actually good game AI was not even a goal - the only purpose of these experime

A big aspect is that for everyone working on those competition systems, making an actually good game AI was not even a goal - the only purpose of these experiments is as a proof of concept (perhaps as PR) and a testbed for techniques which would then apply to other practical tasks simulating human decisions; and the research attention seems to have moved from popular strategy games to more specific simulated environments, which are better at being experimental testbeds i.e. faster/cheaper to simulate. People working on AI systems that are going to be actually used in games shipped to gamers is a separate community with (IMHO) almost no overlap.