GPT-3 can generate well-formed programs, so yes, it does things well beyond this complexity. > After 500,000 iterations I'm still getting invalid sequences. H

GPT-3 can generate well-formed programs, so yes, it does things well beyond this complexity. > After 500,000 iterations I'm still getting invalid sequences. How frequently? If it's only the occasional issue it might be down to the temperature-based sampling that code uses, which means it will, with some small probability, return arbitrarily unlikely outputs.