3 ms·
> Every definition you say? When I was growing up, "very smart" people had a law degree or a medical degree or understood quantum physics and/or relativity (bu
by ben_w 2y ago
> Every definition you say?
When I was growing up, "very smart" people had a law degree or a medical degree or understood quantum physics and/or relativity (but all three would be silly), or spoke perhaps three or four languages (but thirty would be a joke). They might perhaps also be above average at games of skill like chess, or perhaps could improvise poetry in some specific style (but every style would be a bit of a Mary Sue).
I may not trust ChatGPT's medical diagnosis or legal analysis, but it does pass those exams; I may not expect much novel insights about unifying QM and GR, but it seems to "know" the topics well enough for people to worry it may lead to cheating in those subjects and most others besides.
And the chess score may not be the absolute best, but it's way above the average human player.
And it seems entirely adequate in quite a lot of languages (though not native), and most styles of poetry (again, not what anyone would call the Nobel committee for, but way above what most people can dream of).
> https://bsky.app/profile/edzitron.com/post/3lclo77koj22y https://bsky.app/profile/edzitron.com/post/3lclo77koj22y
We also didn't have spellcheck running all the time when I was growing up, and when we did there was quite a lot of red, but even that misses things like "I'm applying for collage" and "what lovely wether*" which were also real mistakes made by actual humans.
* spelled like that it means "castrated ram"
- disgruntledphd2 2y ago> I may not trust ChatGPT's medical diagnosis or legal analysis, but it does pass those exams; It's really, really, really important to note that this could easily just be memorisation (and indeed, the GSM Symbolic paper from Apple suggests that it most likely is). We don't have a good way of assessing the ability of models literally trained on all free text in existence, but comparing their performance based on text they've seen in training is definitely not it.
- ben_w 2y agoWhat I've always disliked about calling it "memorisation" is that we already have machines with much better memorisation than what LLMs do: the hard drives with the source documents. To the extent that it's able to respond to and with natural language, this is already an improvement over, say, grep or PageRank. To the extent that it is memorisation in the way we mean when we say a human has memorised something, it's an example of another aspect of what was meant by "intelligence" when I was growing up. It's definitely weird and alien as an intelligence, but it still ticks the boxes we used to have even a decade ago, 2014, let alone the late 80s to 1999 when I turned 16.