Awesome to see open and independent reproductions of Codex/Copilot being worked on! It seems like this must be in fairly early stages? The HumanEval numbers in

Awesome to see open and independent reproductions of Codex/Copilot being worked on! It seems like this must be in fairly early stages? The HumanEval numbers in the README seem to indicate the pass rate with one sample (pass@1) on the benchmark is at best 0.06%, whereas the Codex paper reports up to 28% (for the Codex-12B model). I'm actually kind of surprised by this, since even the very small Codex models (e.g., Codex-12M) manage 2% on that benchmark. I also don't see a lot of recent activity – anyone know if the project is still alive?