While I am agnostic as to the advantages of RL over simpler ML, some basic structural concerns seem to apply here. The potential concern is that if RL works, th

While I am agnostic as to the advantages of RL over simpler ML, some basic structural concerns seem to apply here. The potential concern is that if RL works, the very best factors for learning are likely to be potentially dangerous. The best environment for learning is the richest -- ie. unrestricted access to be the entire Internet/ or the world via the Internet. The best reward function is quite likely to be reproduction. If the researcher allows moderate action but attempts to limit reward functions, at a certain level they may find the AI has found a better reward. If the claimed advantages to learning of richer environments & richer action capability are even somewhat true, researchers are likely to be strongly incentivized to pursue such risky structures.