I do agree with the premise that chatgpt cannot be trusted. It's very good, 80% accuracy is not trivial, but still 20% of errors is a lot. The issue with the f

I do agree with the premise that chatgpt cannot be trusted. It's very good, 80% accuracy is not trivial, but still 20% of errors is a lot. The issue with the first experiment is that it might be the free version. The author not only doesn't mention the exact revision, but there's no mention of the general model. There's a cost-saving measure that chatgpt needs to implemnet, increasing the compute and output token count is quite trivial, the fact that the table is reduced to 50 states might be just that, not necessarily a limitation on its intelligence. Finally the question on vowels is just a very specific and widely known weakness due to tokenization. Nothing new and definitely does not tell us much of how intelligent the model is. EDIT: I just noticed OP explicitly states it's the unpaid version. Complaining that a free version is not a super intelligent artificial intelligence and expecting to evaluate and make a judgment on it is just missing completely. Shell out 20$ before you want to give us your invaluable input on the state of the art. Here's the 50 states with your prompt on the paid version https://chatgpt.com/share/67a29e2d-ff88-800f-985f-a894872813ce https://chatgpt.com/share/67a29e2d-ff88-800f-985f-a894872813... If you want to use the free version, or even produce reasonable content with the paid model. Try to split your request into parts instead of asking 5 things in 1 prompt. Like you would any human, if you ask it to do 5 things in 1 day, the result will be worse than if you ask it to do 1 per day. As is this experiment design is what you get when you are actively trying to fail so you can publish your prewritten rant.