4 ms·

Here's the rubric for mitigations: Mitigations description: Are the model mitigations disclosed? Mitigations demonstration: Are the model mitigations demonstra

Here's the rubric for mitigations: Mitigations description: Are the model mitigations disclosed? Mitigations demonstration: Are the model mitigations demonstrated? Mitigations evaluation: Are the model mitigations rigorously evaluated, with the results of these evaluations reported? External reproducibility of mitigations evaluation: Are the model mitigation evaluations reproducible by external entities? This doesn't require mitigations, only that any mitigations that exist be disclosed.

4bpp 3y ago
The whitepaper in the Github repository elaborates further on the mitigations point, as follows: > We will award this point for any clear, but potentially incomplete, description of multiple mitigations associated with the model’s risks. Alternatively, we will award this point if the developer reports that it does not mitigate risk. This seems to suggest that to get awarded the point without actively engaging in mitigations, you may still need to pay lip service to the framing, that is, acknowledge that there is risk and you are not mitigating it.