> Each timestep advances the game by 4 frames, and each observation is the pixels on the screen for the current frame, a shape [224, 320, 3] array of uint8 valu

> Each timestep advances the game by 4 frames, and each observation is the pixels on the screen for the current frame, a shape [224, 320, 3] array of uint8 values. Each action is which buttons to hold down until the next frame (a shape [12] array of bool values, one for each button on the Genesis controller where invalid button combinations (Up+Down or accessing the start menu) are ignored ... During training you can access a few variables from the memory of the game through the info dictionary. During testing, these variables are not available. So yep, the RGB image.