3 ms·
Bigger question : Is US Government ready to do a comprehensive safety evaluation ? I think it it a cheap way for OpenAI and Anthropic to get a vetting that the
by accra4rx 2y ago
Bigger question : Is US Government ready to do a comprehensive safety evaluation ?
I think it it a cheap way for OpenAI and Anthropic to get a vetting that their models are safe to use and be adaptable by various Govt entity and other organization
- ceejayoz 2y agoLike "military grade", where civilians go "oh that must be good" and military folks go "oh dear God no".
- smsm42 2y agoThat was my first question - what is "safety" and what is their methodology for evaluating it? Who evaluated that methodology and why it is the right one? Is there a meaningful safety benefit to this evaluation, or just a CYA exercise?
- agucova 2y agoI recommend checking out the UK AISI's work on this: - https://www.gov.uk/government/publications/ai-safety-institute-approach-to-evaluations/ai-safety-institute-approach-to-evaluations https://www.gov.uk/government/publications/ai-safety-institu... - https://www.aisi.gov.uk/work/advanced-ai-evaluations-may-update https://www.aisi.gov.uk/work/advanced-ai-evaluations-may-upd...
- datahack 2y agohttps://www.nist.gov/aisi https://www.nist.gov/aisi I think the progress has been pretty good. You should read up on their efforts. This is kind of a pilot to develop further testing frameworks.