> TLDR: A faulty loss function leads to faulty results. This. The use of "gaming" to describe the observed behaviour seems strange. To "game" suggests some m

> TLDR: A faulty loss function leads to faulty results. This. The use of "gaming" to describe the observed behaviour seems strange. To "game" suggests some malfeasance in the algorithm; a deliberate effort to subvert the intended goal, which in turn implies knowing what the "real" goal was - and somehow deciding to shortcut it. That isn't the case. Rather, the problem lies in appropriately specifying the desired outcome, and any relevant constraints that must be obeyed. Indeed, DeepMind has separately written about the "specification problem" elsewhere [0]. That's in no way to downplay the difficulty in specification. It's one we've wrestled with in software for decades. > Either you have the mathematical and stochastic skills to describe your problem criteria as a differentiable function, or not. Indeed. That statement could equally have been written ~30 years ago when Z, VDM and the likes were active areas of research. More recently, TLA+ has gotten some traction, but it's still very much in the margins. Perhaps ML will spur the next wave of specification formalisms. But it will face the same challenge as its pre-decessors: how to make it accessible to more than a very (very) small set of sufficiently capable few. [0] https://medium.com/@deepmindsafetyresearch/building-safe-artificial-intelligence-52f5f75058f1 https://medium.com/@deepmindsafetyresearch/building-safe-art...