its likely the result of explicit effort by OpenAI to counter steer the model in specific circumstances, not that the aggregate training data conveys those bias

its likely the result of explicit effort by OpenAI to counter steer the model in specific circumstances, not that the aggregate training data conveys those biases. The data is biased in the opposite direction. What it shows is that this counter-steering only works in explicit cases and whatever OpenAI is doing isn't generalizing.