> How do you expect it to produce useful programs why not ask it to produce said program, and then evaluate it on that output, rather than proxy it via asking

> How do you expect it to produce useful programs why not ask it to produce said program, and then evaluate it on that output, rather than proxy it via asking a logic puzzle? It's like doing a coding interview when hiring an employee with these brain teaser puzzles, and when they fail you disqualify them, rather than asking them to do a real task that would be something they'd encounter on the job.