3 ms·
Has the METR study been replicated?
by williamcotton 8mo ago
Has the METR study been replicated?
- jdlshore 8mo agoI haven’t heard about any similar studies, no. I’m planning to conduct one at my workplace but we’re still deciding exactly which uses of AI to test.
- lunar_mycroft 8mo agoNot a scientific study, but someone did replicate the experiment on themselves [0] and found that in their case, any effect from LLM use wasn't detectable in their sample. Notably they almost certainly had more experience with LLMs than most of the METR participants did. [0] https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-coding https://mikelovesrobots.substack.com/p/wheres-the-shovelware...