It's good. Rate-limited on direct chat, I had to use the arena mode. I asked the same question 10 times before I got an answer from gpt2-chatbot. My question:

It's good. Rate-limited on direct chat, I had to use the arena mode. I asked the same question 10 times before I got an answer from gpt2-chatbot. My question: "Can you write a YAML parser in Hacklang?" This is a fun question because every LLM I've seen so far (including ChatGPT 4) will come up with an answer that's about six years out of date. They'll often suggest code that flat-out doesn't work in Hacklang anymore (Hack/HHVM used to be able to execute all PHP code). This is a difficult problem because there's very little Hacklang code available in public GitHub. The vast majority of Hacklang code — many millions of LOC — is closed-source and not available for training. But there is public documentation at docs.hhvm.com The gpt2-chatbot output was still pretty bad, but at least the syntax it generated was valid in current versions of Hack, and invalid in current versions of PHP — more than can be said for any other LLM.