> Human brains are unpredictable. Look around you. As it was mentioned by others, we've had thousands of years to better understand how humans can fail. LLMs

> Human brains are unpredictable. Look around you. As it was mentioned by others, we've had thousands of years to better understand how humans can fail. LLMs are black boxes and it never ceases to amaze me how they can fail in such unpredictable ways. Take the following for examples. Here GPT-4o mini is asked to calculate 2+3+5 https://beta.gitsense.com/?chat=8707acda-e6d4-4f69-9c09-2cff0907b207 https://beta.gitsense.com/?chat=8707acda-e6d4-4f69-9c09-2cff... It gets the answer correct, but if you ask it to verify its own answer https://beta.gitsense.com/?chat=6d8af370-1ae6-4a36-961d-2902cc17eea0&model=GPT-4o+mini https://beta.gitsense.com/?chat=6d8af370-1ae6-4a36-961d-2902... it says the response was wrong, and contradicts itself. Now if you ask it to compare all the responses https://beta.gitsense.com/?chat=1c162c40-47ea-419d-af7a-a30aba857cca&model=GPT-4o+mini https://beta.gitsense.com/?chat=1c162c40-47ea-419d-af7a-a30a... it correctly identifies that GPT-4o mini was incorrect. It is this unpredictable nature that makes LLM insanely powerful and scary. Note: The chat on the beta site doesn't work.