23 ms·

AI hype is built on flawed test scores

204 points by antondd 3y ago

GuB-42 3y ago
I don't think test scores have anything to do with the hype. Most people don't even realize test scores exist. One is just to wow factor. It will be short lived. A bit like VR, which is awesome when you first try it, but it wears out quickly. Here, you can have a bot write convincing stories and generate nice looking images, which is awesome until you notice that the story doesn't make sense and that the images has many details wrong. This is not just a score, it is something you can see and experience. And there is also the real thing. People start using GPT for real work. I have used it to document my code for instance, and it works really well, with it I can do a better job than without, and I can do it faster. Many students use it to do their homework, which may not be something you want, but it no less of a real use. Many artists are strongly protesting against generative AI, this in itself is telling, it means it is taken seriously, and at the same time, other artists are making use of it. It is even use for great effect where you don't notice. Phone cameras are a good example, by enhancing details using AI, they give you much better pictures than what the optics are capable of. Some people don't like that because the picture are "not real", but most enjoy the better perceived quality. Then, there are image classifiers, speech-to-text and OCR, fuzzy searching, content ranking algorithms we love to hate, etc... that all make use of AI. Note: here AI = machine learning with neural networks, which is what the hype is about. AI is a vague term that can mean just about anything.
- Jensson 3y ago
  > I don't think test scores have anything to do with the hype. Most people don't even realize test scores exist. They put the test scores front and center in the initial announcement with a huge image showing improvements on AP exams, it was the main thing people talked about during the announcement and the first thing anyone who read anything about gpt-4 sees. I don't think many who are hyped about these things missed that. https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
  - GuB-42 3y ago
    It is what they talk about during announcements because people like numbers. It looks more serious than "hey look, GPT-4 smart" with some example quotes that anyone knows are cherry picked. But the real hype comes from people trying for themselves. I seriously don't remember hearing these test results being mentioned in any casual conversation, and I heard a lot of casual conversations about AI. The majority of these center around personal experiences ("I asked ChatGPT this and I got that..."), homework is another common topic. When we compare systems, we won't say "this one got a 72 and the other got a 94", but more like "I asked new system to give me a specific piece of code (or cocktail recipe, or anything) and the result is much better". Again, personal experience and anecdotes before scores. Maybe people in the field hype themselves with score, but not the general public, and probably not the investors either, who will most likely look at the financial performance of the likes of OpenAI instead.
  - bensecure 3y ago
    If you followed the initial announcement, then you were presumably already hyped. The novel thing about chatgpt has been the mass amount of people who hadn't heard about generative AI in the past glomming onto the technology. Most of these people heard about it via word of mouth. They then tried it themselves and told people about it. They never even heard of tests, let alone based their perception on them.
yieldcrv 3y ago
This was 2 months ago, irrelevant in AI time
MrYellowP 3y ago
I disagree entirely. The hype is based entirely on the fact that I can talk (in text) to a machine and it responds like a human. It might sometimes make up stuff, but so do humans. I therefore don't consider that a significant downside, or problem. In the end chatgpt is still ... a baby. The hype builds around the fact that I can run a language model that fits into my graphics cards and responds at faster-than-typing speed, which is sufficient. The hype builds around the fact that it can create and govern whole text based games for me, if I just properly ask it to do so. The hype builds around the fact that I can have this everywhere with me, all day long, whenever I want. It never grows tired, it never stops answering, it never scoffs at me, it never hates me, it never tells me that I'm stupid, it never tells me that I'm not capable of doing something. It always teaches me, always offers me more to learn, it always is willingly helping me, it never intentionally tries to hide the fact that it doesn't know something and never intentionally tries to impress me just to get something from me. Can it get things wrong? Sure! Happens! Happens to everyone. Me, you, your neighbour, parents, teachers, plumbers. Not a single minute did I, or dozens of millions of others, give a single flying fuck about test scores.
nojvek 3y ago
Related paper https://arxiv.org/pdf/2309.08632.pdf https://arxiv.org/pdf/2309.08632.pdf ‘Pre-training on the Test Set Is All You Need‘ GPT-4 is really smart to dig information it has seen before, but please don’t use it for any serious reasoning. Always take the answer with a grain of salt.
Garvi 3y ago
Counterpoint: Journalism is dead and has been replaced with algorithms that supply articles on a supply and demand basis. "25% of the potential target audience dislikes AI and do not have their opinion positively represented in the media they consume. The potential is unsaturated. Maximum saturation estimated at 15 articles per week." A bit more serious: AI hasn't even scratched the surface. Once we apply LLMs to speech synth and improve the visual generators by just a tiny bit, to fix faces, we can basically tell the AI to "create the best romantic comedy ever made". "Oh, and repeat 1000 times, please".
kfk 3y ago
AI hype is really problematic in Enterprise. Big companies are now spending C executive time figuring out a company "AI strategy". This is going to be another cycle of money-wasted/biz-upset, very similar to what I have seen with Big data. The thing in Enterprise is that everyone serious about biz operations knows AI test scores and AI quality is not there, but very few are able to communicate these concerns in a constructive way, rather everyone is embracing the hype because, maybe they get a promotion? Tech, as usual, is very happy to feed the hype and never, as usual, telling businesses honestly that, at best, this is an incremental productivity improvement, nothing life changing. I think the issue is overall lack of honesty, professionalism, and accountability across the board, with tech leading this terrible way of pushing product and "adding value".
- afro88 3y ago
  In my case executives were more focussed on how it could be built into new projects, presales etc rather than internal efficiency improvements. A lot of people were amazed to see someone getting value out of it (efficiency gains) without building stuff around it. Blew my mind that this was the case.
- JaDogg 3y ago
  This is exactly correct.
- huijzer 3y ago
  > Tech, as usual, is very happy to feed the hype I agree completely with you on this. In defence of the executives however is that some businesses will be seriously affected. Call centres and plagiarism scanner have already been affected, but it’s unclear which industries will be affected too. Maybe the probability is low, but the impact could be very high. In think this reasoning is driving the executives.
  - kfk 3y ago
    Look, I am going to wait and see on this, maybe new facts will make me reconsider. In the meanwhile, github Copilot is just cost to my company, haven't seen much additional productivity. I guess my concern, given how hard is to hire developers and technologists, is replacing simpler job roles, like a customer service representative, with complicated new ones, like "MLOps Engineer".
mg 3y ago
I don't think the "hype" is built on test scores. It is built on the observation how fast AI is getting better. If the speed of improvement stays anywhere near the level it was the last two years, then over the next two decades, it will lead to massive changes in how we work and which skills are valuable. Just two years ago, I was mesmerized by GPT-3's ability to understand concepts: https://twitter.com/marekgibney/status/1403414210642649092 https://twitter.com/marekgibney/status/1403414210642649092 Nowadays, using it daily in a productive fashion feels completely normal. Yesterday, I was annoyed with how cumbersome it is to play long mp3s on my iPad. I asked GPT-4 something like "Write an html page which lets me select an mp3, play it via play/pause buttons and offers me a field to enter a time to jump to". And the result was usable out of the box and is my default mp3 player now. Two years ago it didn't even dawn on me that this would be my way of writing software in the near future. I have been coding for over 20 years. But for little tools like this, it is faster to ask ChatGPT now. It's hard to imagine where we will be in 20 years.
- kristopolous 3y ago
  Almost nothing happened in AI for about 50 years. That's the normal in the field.
- ulnarkressty 3y ago
  Most of the improvements apparently come from training larger models with more data. Which is part of the problem mentioned in the article - the probability that the model just memorizes the answers to the tests is greatly increased. AI is getting subjectively better, and we need better tests to figure out if this improvement is objectively significant or not.
  - Nevermark 3y ago
    That’s backwards. Training a model on more data improves generalization not memorization. To store more information in the same number of parameters requires the commonality between examples to be encoded. In contrast, the less data trained on, especially if repeated, lets the network learn to provide good answers for that limited set without generalizing. I.e. memorizing. —— It’s the same as with people. The more variations people see of something, the more likely they intuit the underlying pattern. The fewer examples, the more likely they just pattern match.
dmezzetti 3y ago
This video from Yann LeCun gives a great summary on where things stand. https://www.youtube.com/watch?v=pd0JmT6rYcI https://www.youtube.com/watch?v=pd0JmT6rYcI He is of the opinion the current generation transformers architecture is flawed and it will take a new generation of models to get close to the hype.
epups 3y ago
I think ironically there has been an "AI-anti-hype hype", with people like Gary Marcus trying to blow up every single possible issue into a deal breaker. Most of the claims in this article are based on tests performed only on GPT-3, and researchers often seem to make tests in a way that proves their point - see an earlier comment from me here with an example: https://news.ycombinator.com/item?id=37503944 https://news.ycombinator.com/item?id=37503944 I agree there has been many attention-grabbing headlines that are due to simple issues like contamination. However, I think AI has already proved its business value far beyond those issues, as anyone using ChatGPT with a code base not present in their dataset can attest.
- smcl 3y ago
  I think some amount of that is necessary, though no? We have people claiming that this generation of AI will replace jobs - and plenty of companies have taken the bait and tried to get started with LLM-based bots. We even had a pretty high-profile case of a Google AI engineer going public with claims that their LaMDA AI was sentient. Regardless of what you think of that individual or Google's AI efforts, this resonates with the public. Additionally a pretty common sentiment I've seen has been non-tech people suggesting AI should handle content moderation - the idea being that since they're not human and don't have "feelings" they won't have biases and won't attempt to "silence" any one political group (without realising that bias can be built in via the training data). It seems pretty important to counter that and to debunk any wild claims such as these. To provide context and to educate the world on their shortcomings.
  - epups 3y ago
    I think skepticism is always welcome and we should continue to explore what LLM's can and cannot do. However, what I'm referring to is trying to get a quick win by defeating some inferior version of GPT or trying to apply a test which you don't even expect most humans to pass. The article is actually fine and pretty balanced, but it is a bit unfortunate that 80% of their examples are not illustrative of current capabilities. At least for me, most of my optimism about the utility of LLM's comes from GPT-4 specifically.
PeterisP 3y ago
It's not built on high test scores - while academics do benchmark models on various tests, all the many people who built up the hype mostly did it based on their personal experience with a chatbot, not by running some long (and expensive) tests on those datasets. The tests are used (and, despite their flaws, useful) to compare various facets of model A to model B - however, the validation whether a model is good now comes from users, and that validation really can't be flawed much - if it's helpful (or not) to someone, then it is what it is, the proof of the pudding is in the eating.
bondarchuk 3y ago
>But there’s a problem: there is little agreement on what those results really mean. Some people are dazzled by what they see as glimmers of human-like intelligence; others aren’t convinced one bit. I find the whole hype & anti-hype dynamic so tiresome. Some are over-hyping, others are responding with over-anti-hyping. Somewhere in-between are many reasonable, moderate and caveated opinions, but neither the hypesters or anti-hypesters will listen to these (considering all of them to come from people at the opposite extreme), nor will outside commentators (somehow being unable to categorize things as anything more complicated than this binary).
- Closi 3y ago
  Depends if the hype is invalid - Let's remember that "There will be a computer in every home!" was once considered hype. There is a possible world where AI will be a truly transformative technology in ways we can't possibly understand. There is a possible world where this tech fizzles out. So one of the reasons that there is a broad 'hype' dynamic here is because the range of possibilities is broad. I sit firmly in the first camp though - I believe it's truly a transformative technology, and struggle to see the perspective of the 'anti-hype' crowd.
  - TerrifiedMouse 3y ago
    I’m in the second camp. To every hyped up tech, all I can say is “prove it”. Give me actual real world results. There are millions of hustlers out there pushing snake oil. The probability that something is the real deal and not snake oil is small. Better to assuming the glass is half empty.
    - Closi 3y ago
      There will be millions of hustlers regardless of if the technology is transformative or not. The invention of the PC market was filled with hustlers but that doesn't mean that the PC didn't match the hype. The .com boom was filled with hustlers, but that doesn't mean that the Internet wasn't transformative. Actual real world results... well the technology is already responsible for c40% of code on Github. Image recognition technologies are soaring and self driving feels within reach. Few people doubt that a real-world Jarvis will be in your home within 12 months. The turing test is smashed, and LLM's are already replacing live chat operatives. And this is just the start of the technology...
chewxy 3y ago
I note something very interesting in the AI hype, and I would like someone to help explain it. Whenever there's a news or article noting the limits of current LLM tech (especially the GPT class of models from OpenAI), there's always a comment that says something along the lines of "ah did you test it on GPT-4"? Or if it's clear that it's the limitation of GPT-4, then you have comments along the lines of "what's the prompt?", or "the prompt is poor". Usually, it's someone who hasn't in the past indicated that they understand that prompt engineering is model specific, and the papers' point is to make a more general claim as opposed to a claim on one model. Can anyone explain this? It's like the mere mention of LLMs being limited in X, Y, Z fashion offends their lifestyle/core beliefs. Or perhaps it's a weird form of astroturfing. To which, I ask, to what end?
- abm53 3y ago
  Perhaps they are trying to help people get the best out of a tool which they themselves find very useful?
- jacobr1 3y ago
  As someone who has this instinct myself, there is a line of reactionism to modern AI/ML that says, "this is just a toy, look it can't do something simple." But often the case, if _can_ do that thing with a either a more advanced model, or a more built-out system. So the instinct is to try and explain that the pessimism is wrong. That we really can push the boundary and do more, even if it isn't going to work out of the box yet. I react that way against all forms of poppy snipping.
  - Jensson 3y ago
    Hyping up tech based on what you think it will be able to do in the future is the misplaced overhyping that is the problem. The issues people say are easy to fix aren't easy to fix. Expect the model to continue to perform like it does today, and then lots of dumb integrations added to it, and you will get a very accurate prediction of how most of new tech hype turns out. Dumb integrations can't add intelligence, but it can add a lot of value, so the rational hype still sees this as a very valuable and exciting thing, but it isn't a complete revolution in its current form.
Cloudef 3y ago
AI is honestly wrong word to use. These are ML models and they are able to only do the task they have been specifically trained for (not saying the results aren't impressive!). There really isn't competition either as the only people who can train these giant models are those who have the cash.
- TeMPOraL 3y ago
  > These are ML models and they are able to only do the task they have been specifically trained for Yes, but the models we're talking about have been trained specifically on the task of "complete arbitrary textual input in a way that makes sense to humans", for arbitrary textual input, and then further tuned for "complete it as if you were a person having conversation with a human", again for arbitrary text input - and trained until they could do so convincingly. (Or, you could say that with instruct fine-tuning, they were further trained to behave as if they were an AI chatbot - the kind of AI people know from sci-fi. Fake it 'till you make it, via backpropagation.) In short, they've been trained on an open-ended, general task of communicating with humans using plain text. That's very different to typical ML models which are tasked to predict some very specific data in a specialized domain. It's like comparing a Python interpreter to Notepad - both are just regular software, but there's a meaningful difference in capabilities. As for seeing glimpses of understanding in SOTA LLMs - this makes sense under the compression argument: understanding is lossy compression of observations, and this is what the training process is trying to force to happen, squeezing more and more knowledge into a fixed set of model weights.
  - Cloudef 3y ago
    Yes, this is why I think the LLM and image generation models are still impressive. Knowing they are ML models in the end and still produce a results that surprise us, makes you wonder what we are in the end. Could we essentially simulate something similar to us given enough inputs and parameters in the network, with enough memory, computing power and a training process that would aim to simulate a human with emotions. I would imagine the training process alone would need bunch of other models to teach the final model "concepts" and from there perhaps "reasoning". Why I think AI is not the appropriate term is that if it were AI, the AI would have already figured everything out for us (or for itself). LLM can only chain text, it does not really understand the content of the text, and can't come up with new novel solutions (or if it accidentally does, it's due to hallucination), this can be easily confirmed by giving current LLMs some simple puzzles, math problems and so on.. Image models have similar issues.
refulgentis 3y ago
This is my favorite new AI argument, took me a few months to see it. Enjoyed it at first. You start with everyone knows there's AI hype from tech bros. Then you introduce a PhD or two at institutions with good names. Then they start grumbling about anthropomorphizing and who knows what AI is anyway. Somehow, if it's long enough, you forget that this kind of has nothing to do with anything. There is no argument. Just imagining other people must believe crazy things and working backwards from there to find something to critique. Took me a bit to realize it's not even an argument, just parroting "it's a stochastic parrot!" Assumes other people are dunces and genuinely believe it's a minihuman. I can't believe MIT Tech Review is going for this, the only argument here is the tests are flawed if you think they're supposed to show the AI model is literally human.
dleslie 3y ago
Two years ago I didn't use AI at all. Now I wouldn't go without it; I have Copilot integrated with Emacs, VSCode, and Rider. I consider it a ground-breaking productivity accelerator, a leap similar to when I transitioned from Turbo Pascal 2 to Visual C 6. That's why I'm hyped. If it's that good for me, and it's generalizable, then it's going to rock the world.
- airstrike 3y ago
  Coding on something without copilot these days feels like having my hands tied. I'm looking at you, XCode and Colab...
- thomasfromcdnjs 3y ago
  Life longer programmer, and same sentiments, I use it everywhere I can. I am currently transliterating a language PDF into a formatted lexicon, I wouldn't even be able to do this without co-pilot, it has turned this seemingly impossibly arduous task into a pleasurable one.
Kalanos 3y ago
Didn't it perform well on both the SAT and LSAT though?
javier_e06 3y ago
As a developer when I work with ChatGPT I can see ChatGPT eventually taking over my JIRA stories. Then ChatGPT will take over management creating product roadmaps, prioritizing and assigning tasks to itself. All dictated by customer feedback. The clock is ticking. But reasoning like a human? No.
waynenilsen 3y ago
This article is absurd. > But when a large language model scores well on such tests, it is not clear at all what has been measured. Is it evidence of actual understanding? A mindless statistical trick? Rote repetition? It is measuring how well it does _at REPLACING HUMANS_. It is hard to believe how the author clearly does not understand this. I don't care how it obtains its results. GPT-4 is like a hyperspeed entry to mid level dev that has almost no ability to contextualize. Tools built on top of 32k will allow repo ingestion. This is the worst it will ever be.
- COAGULOPATH 3y ago
  >It is measuring how well it does _at REPLACING HUMANS_ It's possible to do well on a test and have no ability to do the thing the job tests for. GPT-4 scores well on an advanced sommelier exam, but obviously cannot replace a human sommelier, because it does not have a mouth.
- iudqnolq 3y ago
  GPT passed a test on the theoretical fundamentals of selling and serving wine in fancy restaurants. In a human passing such a test provides a useful signal of job suitability because people who pass it are often also capable of the physical bits, like theatrically opening wine bottles. But obviously that doesn't work for an AI. Lots of things usefully correlate with test scores in humans but might not in an AI.
- dartos 3y ago
  Which tests test specifically for “replacing humans?” That seems like a wild metric to try and capture in a test. Also an aside: > This is the worse it will ever be. I hear this a lot and it really bothers me. Just because something is the worst it’ll ever be doesn’t mean it’ll get much better. There could always be a plateau on the horizon. It’s akin to “just have faith.” A real weird sentiment that I didn’t notice in tech before 2021.
- RandomLensman 3y ago
  It is measuring how well it does replacing humans - in those tests.
derbOac 3y ago
This was interesting to me but mostly because of a question I thought it was going to focus on, which is how should we interpret these tests when a human takes it? I wasn't sure that the phenomena they discussed was as relevant to the question of whether AI is overhyped as they made it out to be, but I did think a lot of questions about the meaning of the performances were important. What's interesting to me is you could flip this all on its head and, instead of asking "what can we infer about the machine processes these test scores are measuring?", we could ask "what does this imply about the human processes these test scores are measuring?" A lot of these test are well-validated but overinterpreted I think, and leaned on too heavily to make inferences about people. If a machine can pass a test, for instance, what does it say about the test as used in people? Should we be putting as much weight on them as we do? I'm not arguing these tests are useless or something, just that maybe we read into them too much to begin with.
robertlagrant 3y ago
> AI hype is built on high test scores No, it's built on people using DALLE and Midjourney and ChatGPT.
- yCombLinks 3y ago
  Exactly, chatpgt is double checking my homework problems and pointing out my errors, it's teaching me the material better than any of my lectures. It's writing tons of code I'm getting paid for, with way less overhead than trying to explain the problem to a junior, less mistakes and faster iteration. Test scores, ridiculous
iambateman 3y ago
This really is a good article, and is seriously researched. But the conclusion in the headline - “AI hype is built on flawed test scores” - feels like a poor summary of the article. It _is_ correct to say that an LLM is not ready to be a medical doctor, even if it can pass the test. But I think a better conclusion is that test scores don’t help us understand LLM capabilities like we think they do. Using a human test for an LLM is like measuring a car’s “muscles” and calling it horsepower. They’re just different. But the AI hype is justified, even if we struggle to measure it.
mcguire 3y ago
"When Horace He, a machine-learning engineer, tested GPT-4 on questions taken from Codeforces, a website that hosts coding competitions, he found that it scored 10/10 on coding tests posted before 2021 and 0/10 on tests posted after 2021. Others have also noted that GPT-4’s test scores take a dive on material produced after 2021. Because the model’s training data only included text collected before 2021, some say this shows that large language models display a kind of memorization rather than intelligence." I'm sure that is just a matter of prompt engineering, though.
- COAGULOPATH 3y ago
  But it got 10/10 on pre-2021 questions, with the same prompting method...
danielvaughn 3y ago
I remember watching a documentary about an old blues guitar player from the 1920's. They were trying to learn more about him and track down his whereabouts during certain periods of his life. At one point, they showed some old footage which featured a montage of daily life in a small Mississippi town. You'd see people shopping for groceries, going on walks, etc. Some would stop and wave at the camera. In the documentary, they noted that this footage exists because at the time, they'd show it on screen during intermission at movie theaters. Film was still in its infancy in that time, and was so novel that people loved seeing themselves and other people on the big screen. It was an interesting use of a new technology, and today it's easy to understand why it died out. Of course, it likely wasn't obvious at the time. I say all that because I don't think we can know at this point what AI is capable of, and how we want to use it, but we should expect to see lots of failure while we figure it out. Over the next decade there's undoubtedly going to be countless ventures similar to the "show the townspeople on the movie screen" idea, blinded by the novelty of technological change. But failed ventures have no relevance to the overall impact or worth of the technology itself.
- kenjackson 3y ago
  What died out? Film?
  - actionfromafar 3y ago
    Showing locals little movie clips of themselves in intermissions at the local theater.
  - savanaly 3y ago
    >What died out? The custom of showing film consisting of footage of the general public in movie theaters.
  - danielvaughn 3y ago
    The practice of filming a montage around your local neighborhood or town to play during intermission. Though you could say intermission as well, since that was a legacy concept that was inherited from plays and eventually died out as well.
rvz 3y ago
Most of the hype comes from the AI grifters who need to find the next sucker to dump their VC shares onto to the next greater fool to purchase their ChatGPT-wrapper snake oil project to at an overvalued asking price. The ones who have to dismantle the hype are the proper technologies such as Yann LeCun and Grady Booch who know exactly what they are talking about.
- rvz 3y ago
  *technologists
aldousd666 3y ago
Only idiots are basing their excitement about what's possible on those test scores. They're just an attempt to measure one bot against another. There is a strong possibility that they are only measuring how well the bot takes the test, and nothing at all about what the tests themselves purport to measure. I mean, those tests are probably similar to stuff that's in the training data.
- ehutch79 3y ago
  Yeah... there's a lot of idiots out there.
randcraw 3y ago
The debate over whether LLMs are "intelligent" seem a lot like the old debate among NLP experts whether English must be modeled as a context-free grammar (push down automaton) or finite-state machine (regular expression). Yes, any language can be modeled using regular expressions; you just need an insane number of FSMs (perhaps billions). And that seems to be the model that LLMs are using to model cognition today. LLMs seem to use little or no abstract reasoning (is-a) or hierarchical perception (has-a), as humans do -- both of which are grounded in semantic abstraction. Instead, LLMs can memorize a brute force explosion in finite state machines (interconnected with Word2Vec-like associations) and then traverse those machines and associations as some kind of mashup, akin to a coherent abstract concept. Then as LLMs get bigger and bigger, they just memorize more and more mashup clusters of FSMs augmented with associations. Of course, that's not how a human learns, or reasons. It seems likely that synthetic cognition of this kind will fail to enable various kinds of reasoning that humans perceive as essential and normal (like common sense based on abstraction, or physically-grounded perception, or goal-based or counterfactual reasoning, much less insight into the thought processes / perceptions of other sentient beings). Even as ever-larger LLMs "know more" by memorizing ever more FSMs, I suspect they'll continue to surprise us with persistent cognitive and perceptual deficits that would never arise in organic beings that do use abstract reasoning and physically grounded perception.
- freejazz 3y ago
  Can you recommend any books on this?
- TeMPOraL 3y ago
  > LLMs can memorize a brute force explosion in finite state machines (interconnected with Word2Vec-like associations) and then traverse those machines and associations as some kind of mashup, akin to a coherent abstract concept. That's actually the closest to a working definition of what a concept is. The discussion about language representation has little bearing on humans or intelligence, because it's not how we learn and use language. Similarly, the more people - be it armchair or diploma-carrying philosophers - try to find the essence of a meaning of some word, the more they fail, because it seems that meaning of any concept is defined entirely through associations with other concepts and some remembered experiences. Which again seems pretty similar to how LLMs encode information through associations in high-dimensional spaces.
aidenn0 3y ago
Any task that gets solved with AI retroactively becomes something that doesn't require reasoning.
- janalsncm 3y ago
  I wouldn’t say that. Chess certainly requires reasoning even if that reasoning is minimax. I suppose in the context of this article “AI” means statistical language models.
  - nomel 3y ago
    Why does chess require reasoning? Do all of the these [1] "reason"? ChatGPT-4 is supposedly rated worse than 500, in this list (1400 or so, although I think a recent update improved it a bit). [1] https://ccrl.chessdom.com/ccrl/4040/ https://ccrl.chessdom.com/ccrl/4040/
    - janalsncm 3y ago
      Within the domain of chess, searching the domain of possible future positions is synonymous with reasoning. It requires an explicit understanding of latent board representations and an explicit understanding of possible actions and the consequences of those actions. Whether ChatGPT has any of those things is questionable. From what I’ve seen, it has an unreliable latent representation, an unreliable understanding of possible moves, and a positional understanding that’s little better than a coin flip.
      - nomel 3y ago
        > Within the domain of chess, searching the domain of possible future positions is synonymous with reasoning. So, with this definition, these chess engines already exhibit (fairly substantial) reasoning? Or, are you saying it would be required, in the context of an LLM?
janalsncm 3y ago
The only test I need is the amount of time it takes me to do common tasks with and without ChatGPT. I’m aware it’s not perfect but perfect was never necessary.
rahimnathwani 3y ago
“People have been giving human intelligence tests—IQ tests and so on—to machines since the very beginning of AI,” says Melanie Mitchell, an artificial-intelligence researcher at the Santa Fe Institute in New Mexico. “The issue throughout has been what it means when you test a machine like this. It doesn’t mean the same thing that it means for a human.” The last sentence above is an important point that most people don't consider.
- api 3y ago
  It seems a bit like having a human face off in a race against a car and then concluding that cars have exceeded human physical dexterity. It's not an apples/apples comparison. The nature of the capability profile of a human vs. any known machine is radically different. Machines are intentionally designed to have extreme peaks of performance in narrow areas. Present-generation AI might be wider in its capabilities than what we've previously built, but it's still rather narrow as you quickly discover if you start trying to use it on real tasks.

204 points by antondd 3y ago

I don't think test scores have anything to do with the hype. Most people don't even realize test scores exist. One is just to wow factor. It will be short lived. A bit like VR, which is awesome when you first try it, but it wears out quickly. Here, you can have a bot write convincing stories and generate nice looking images, which is awesome until you notice that the story doesn't make sense and that the images has many details wrong. This is not just a score, it is something you can see and experience. And there is also the real thing. People start using GPT for real work. I have used it to document my code for instance, and it works really well, with it I can do a better job than without, and I can do it faster. Many students use it to do their homework, which may not be something you want, but it no less of a real use. Many artists are strongly protesting against generative AI, this in itself is telling, it means it is taken seriously, and at the same time, other artists are making use of it. It is even use for great effect where you don't notice. Phone cameras are a good example, by enhancing details using AI, they give you much better pictures than what the optics are capable of. Some people don't like that because the picture are "not real", but most enjoy the better perceived quality. Then, there are image classifiers, speech-to-text and OCR, fuzzy searching, content ranking algorithms we love to hate, etc... that all make use of AI. Note: here AI = machine learning with neural networks, which is what the hype is about. AI is a vague term that can mean just about anything.

Jensson 3y ago
> I don't think test scores have anything to do with the hype. Most people don't even realize test scores exist. They put the test scores front and center in the initial announcement with a huge image showing improvements on AP exams, it was the main thing people talked about during the announcement and the first thing anyone who read anything about gpt-4 sees. I don't think many who are hyped about these things missed that. https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
- GuB-42 3y ago
  It is what they talk about during announcements because people like numbers. It looks more serious than "hey look, GPT-4 smart" with some example quotes that anyone knows are cherry picked. But the real hype comes from people trying for themselves. I seriously don't remember hearing these test results being mentioned in any casual conversation, and I heard a lot of casual conversations about AI. The majority of these center around personal experiences ("I asked ChatGPT this and I got that..."), homework is another common topic. When we compare systems, we won't say "this one got a 72 and the other got a 94", but more like "I asked new system to give me a specific piece of code (or cocktail recipe, or anything) and the result is much better". Again, personal experience and anecdotes before scores. Maybe people in the field hype themselves with score, but not the general public, and probably not the investors either, who will most likely look at the financial performance of the likes of OpenAI instead.
- bensecure 3y ago
  If you followed the initial announcement, then you were presumably already hyped. The novel thing about chatgpt has been the mass amount of people who hadn't heard about generative AI in the past glomming onto the technology. Most of these people heard about it via word of mouth. They then tried it themselves and told people about it. They never even heard of tests, let alone based their perception on them.

yieldcrv 3y ago

This was 2 months ago, irrelevant in AI time

MrYellowP 3y ago

I disagree entirely. The hype is based entirely on the fact that I can talk (in text) to a machine and it responds like a human. It might sometimes make up stuff, but so do humans. I therefore don't consider that a significant downside, or problem. In the end chatgpt is still ... a baby. The hype builds around the fact that I can run a language model that fits into my graphics cards and responds at faster-than-typing speed, which is sufficient. The hype builds around the fact that it can create and govern whole text based games for me, if I just properly ask it to do so. The hype builds around the fact that I can have this everywhere with me, all day long, whenever I want. It never grows tired, it never stops answering, it never scoffs at me, it never hates me, it never tells me that I'm stupid, it never tells me that I'm not capable of doing something. It always teaches me, always offers me more to learn, it always is willingly helping me, it never intentionally tries to hide the fact that it doesn't know something and never intentionally tries to impress me just to get something from me. Can it get things wrong? Sure! Happens! Happens to everyone. Me, you, your neighbour, parents, teachers, plumbers. Not a single minute did I, or dozens of millions of others, give a single flying fuck about test scores.

nojvek 3y ago

Related paper https://arxiv.org/pdf/2309.08632.pdf https://arxiv.org/pdf/2309.08632.pdf ‘Pre-training on the Test Set Is All You Need‘ GPT-4 is really smart to dig information it has seen before, but please don’t use it for any serious reasoning. Always take the answer with a grain of salt.

Garvi 3y ago

Counterpoint: Journalism is dead and has been replaced with algorithms that supply articles on a supply and demand basis. "25% of the potential target audience dislikes AI and do not have their opinion positively represented in the media they consume. The potential is unsaturated. Maximum saturation estimated at 15 articles per week." A bit more serious: AI hasn't even scratched the surface. Once we apply LLMs to speech synth and improve the visual generators by just a tiny bit, to fix faces, we can basically tell the AI to "create the best romantic comedy ever made". "Oh, and repeat 1000 times, please".

kfk 3y ago

AI hype is really problematic in Enterprise. Big companies are now spending C executive time figuring out a company "AI strategy". This is going to be another cycle of money-wasted/biz-upset, very similar to what I have seen with Big data. The thing in Enterprise is that everyone serious about biz operations knows AI test scores and AI quality is not there, but very few are able to communicate these concerns in a constructive way, rather everyone is embracing the hype because, maybe they get a promotion? Tech, as usual, is very happy to feed the hype and never, as usual, telling businesses honestly that, at best, this is an incremental productivity improvement, nothing life changing. I think the issue is overall lack of honesty, professionalism, and accountability across the board, with tech leading this terrible way of pushing product and "adding value".

afro88 3y ago
In my case executives were more focussed on how it could be built into new projects, presales etc rather than internal efficiency improvements. A lot of people were amazed to see someone getting value out of it (efficiency gains) without building stuff around it. Blew my mind that this was the case.
JaDogg 3y ago
This is exactly correct.
huijzer 3y ago
> Tech, as usual, is very happy to feed the hype I agree completely with you on this. In defence of the executives however is that some businesses will be seriously affected. Call centres and plagiarism scanner have already been affected, but it’s unclear which industries will be affected too. Maybe the probability is low, but the impact could be very high. In think this reasoning is driving the executives.
- kfk 3y ago
  Look, I am going to wait and see on this, maybe new facts will make me reconsider. In the meanwhile, github Copilot is just cost to my company, haven't seen much additional productivity. I guess my concern, given how hard is to hire developers and technologists, is replacing simpler job roles, like a customer service representative, with complicated new ones, like "MLOps Engineer".

mg 3y ago

I don't think the "hype" is built on test scores. It is built on the observation how fast AI is getting better. If the speed of improvement stays anywhere near the level it was the last two years, then over the next two decades, it will lead to massive changes in how we work and which skills are valuable. Just two years ago, I was mesmerized by GPT-3's ability to understand concepts: https://twitter.com/marekgibney/status/1403414210642649092 https://twitter.com/marekgibney/status/1403414210642649092 Nowadays, using it daily in a productive fashion feels completely normal. Yesterday, I was annoyed with how cumbersome it is to play long mp3s on my iPad. I asked GPT-4 something like "Write an html page which lets me select an mp3, play it via play/pause buttons and offers me a field to enter a time to jump to". And the result was usable out of the box and is my default mp3 player now. Two years ago it didn't even dawn on me that this would be my way of writing software in the near future. I have been coding for over 20 years. But for little tools like this, it is faster to ask ChatGPT now. It's hard to imagine where we will be in 20 years.

kristopolous 3y ago
Almost nothing happened in AI for about 50 years. That's the normal in the field.
ulnarkressty 3y ago
Most of the improvements apparently come from training larger models with more data. Which is part of the problem mentioned in the article - the probability that the model just memorizes the answers to the tests is greatly increased. AI is getting subjectively better, and we need better tests to figure out if this improvement is objectively significant or not.
- Nevermark 3y ago
  That’s backwards. Training a model on more data improves generalization not memorization. To store more information in the same number of parameters requires the commonality between examples to be encoded. In contrast, the less data trained on, especially if repeated, lets the network learn to provide good answers for that limited set without generalizing. I.e. memorizing. —— It’s the same as with people. The more variations people see of something, the more likely they intuit the underlying pattern. The fewer examples, the more likely they just pattern match.

dmezzetti 3y ago

This video from Yann LeCun gives a great summary on where things stand. https://www.youtube.com/watch?v=pd0JmT6rYcI https://www.youtube.com/watch?v=pd0JmT6rYcI He is of the opinion the current generation transformers architecture is flawed and it will take a new generation of models to get close to the hype.

epups 3y ago

I think ironically there has been an "AI-anti-hype hype", with people like Gary Marcus trying to blow up every single possible issue into a deal breaker. Most of the claims in this article are based on tests performed only on GPT-3, and researchers often seem to make tests in a way that proves their point - see an earlier comment from me here with an example: https://news.ycombinator.com/item?id=37503944 https://news.ycombinator.com/item?id=37503944 I agree there has been many attention-grabbing headlines that are due to simple issues like contamination. However, I think AI has already proved its business value far beyond those issues, as anyone using ChatGPT with a code base not present in their dataset can attest.

smcl 3y ago
I think some amount of that is necessary, though no? We have people claiming that this generation of AI will replace jobs - and plenty of companies have taken the bait and tried to get started with LLM-based bots. We even had a pretty high-profile case of a Google AI engineer going public with claims that their LaMDA AI was sentient. Regardless of what you think of that individual or Google's AI efforts, this resonates with the public. Additionally a pretty common sentiment I've seen has been non-tech people suggesting AI should handle content moderation - the idea being that since they're not human and don't have "feelings" they won't have biases and won't attempt to "silence" any one political group (without realising that bias can be built in via the training data). It seems pretty important to counter that and to debunk any wild claims such as these. To provide context and to educate the world on their shortcomings.
- epups 3y ago
  I think skepticism is always welcome and we should continue to explore what LLM's can and cannot do. However, what I'm referring to is trying to get a quick win by defeating some inferior version of GPT or trying to apply a test which you don't even expect most humans to pass. The article is actually fine and pretty balanced, but it is a bit unfortunate that 80% of their examples are not illustrative of current capabilities. At least for me, most of my optimism about the utility of LLM's comes from GPT-4 specifically.

PeterisP 3y ago

It's not built on high test scores - while academics do benchmark models on various tests, all the many people who built up the hype mostly did it based on their personal experience with a chatbot, not by running some long (and expensive) tests on those datasets. The tests are used (and, despite their flaws, useful) to compare various facets of model A to model B - however, the validation whether a model is good now comes from users, and that validation really can't be flawed much - if it's helpful (or not) to someone, then it is what it is, the proof of the pudding is in the eating.

bondarchuk 3y ago

>But there’s a problem: there is little agreement on what those results really mean. Some people are dazzled by what they see as glimmers of human-like intelligence; others aren’t convinced one bit. I find the whole hype & anti-hype dynamic so tiresome. Some are over-hyping, others are responding with over-anti-hyping. Somewhere in-between are many reasonable, moderate and caveated opinions, but neither the hypesters or anti-hypesters will listen to these (considering all of them to come from people at the opposite extreme), nor will outside commentators (somehow being unable to categorize things as anything more complicated than this binary).

Closi 3y ago
Depends if the hype is invalid - Let's remember that "There will be a computer in every home!" was once considered hype. There is a possible world where AI will be a truly transformative technology in ways we can't possibly understand. There is a possible world where this tech fizzles out. So one of the reasons that there is a broad 'hype' dynamic here is because the range of possibilities is broad. I sit firmly in the first camp though - I believe it's truly a transformative technology, and struggle to see the perspective of the 'anti-hype' crowd.
- TerrifiedMouse 3y ago
  I’m in the second camp. To every hyped up tech, all I can say is “prove it”. Give me actual real world results. There are millions of hustlers out there pushing snake oil. The probability that something is the real deal and not snake oil is small. Better to assuming the glass is half empty.
  - Closi 3y ago
    There will be millions of hustlers regardless of if the technology is transformative or not. The invention of the PC market was filled with hustlers but that doesn't mean that the PC didn't match the hype. The .com boom was filled with hustlers, but that doesn't mean that the Internet wasn't transformative. Actual real world results... well the technology is already responsible for c40% of code on Github. Image recognition technologies are soaring and self driving feels within reach. Few people doubt that a real-world Jarvis will be in your home within 12 months. The turing test is smashed, and LLM's are already replacing live chat operatives. And this is just the start of the technology...

chewxy 3y ago

I note something very interesting in the AI hype, and I would like someone to help explain it. Whenever there's a news or article noting the limits of current LLM tech (especially the GPT class of models from OpenAI), there's always a comment that says something along the lines of "ah did you test it on GPT-4"? Or if it's clear that it's the limitation of GPT-4, then you have comments along the lines of "what's the prompt?", or "the prompt is poor". Usually, it's someone who hasn't in the past indicated that they understand that prompt engineering is model specific, and the papers' point is to make a more general claim as opposed to a claim on one model. Can anyone explain this? It's like the mere mention of LLMs being limited in X, Y, Z fashion offends their lifestyle/core beliefs. Or perhaps it's a weird form of astroturfing. To which, I ask, to what end?

abm53 3y ago
Perhaps they are trying to help people get the best out of a tool which they themselves find very useful?
jacobr1 3y ago
As someone who has this instinct myself, there is a line of reactionism to modern AI/ML that says, "this is just a toy, look it can't do something simple." But often the case, if _can_ do that thing with a either a more advanced model, or a more built-out system. So the instinct is to try and explain that the pessimism is wrong. That we really can push the boundary and do more, even if it isn't going to work out of the box yet. I react that way against all forms of poppy snipping.
- Jensson 3y ago
  Hyping up tech based on what you think it will be able to do in the future is the misplaced overhyping that is the problem. The issues people say are easy to fix aren't easy to fix. Expect the model to continue to perform like it does today, and then lots of dumb integrations added to it, and you will get a very accurate prediction of how most of new tech hype turns out. Dumb integrations can't add intelligence, but it can add a lot of value, so the rational hype still sees this as a very valuable and exciting thing, but it isn't a complete revolution in its current form.

Cloudef 3y ago

AI is honestly wrong word to use. These are ML models and they are able to only do the task they have been specifically trained for (not saying the results aren't impressive!). There really isn't competition either as the only people who can train these giant models are those who have the cash.

TeMPOraL 3y ago
> These are ML models and they are able to only do the task they have been specifically trained for Yes, but the models we're talking about have been trained specifically on the task of "complete arbitrary textual input in a way that makes sense to humans", for arbitrary textual input, and then further tuned for "complete it as if you were a person having conversation with a human", again for arbitrary text input - and trained until they could do so convincingly. (Or, you could say that with instruct fine-tuning, they were further trained to behave as if they were an AI chatbot - the kind of AI people know from sci-fi. Fake it 'till you make it, via backpropagation.) In short, they've been trained on an open-ended, general task of communicating with humans using plain text. That's very different to typical ML models which are tasked to predict some very specific data in a specialized domain. It's like comparing a Python interpreter to Notepad - both are just regular software, but there's a meaningful difference in capabilities. As for seeing glimpses of understanding in SOTA LLMs - this makes sense under the compression argument: understanding is lossy compression of observations, and this is what the training process is trying to force to happen, squeezing more and more knowledge into a fixed set of model weights.
- Cloudef 3y ago
  Yes, this is why I think the LLM and image generation models are still impressive. Knowing they are ML models in the end and still produce a results that surprise us, makes you wonder what we are in the end. Could we essentially simulate something similar to us given enough inputs and parameters in the network, with enough memory, computing power and a training process that would aim to simulate a human with emotions. I would imagine the training process alone would need bunch of other models to teach the final model "concepts" and from there perhaps "reasoning". Why I think AI is not the appropriate term is that if it were AI, the AI would have already figured everything out for us (or for itself). LLM can only chain text, it does not really understand the content of the text, and can't come up with new novel solutions (or if it accidentally does, it's due to hallucination), this can be easily confirmed by giving current LLMs some simple puzzles, math problems and so on.. Image models have similar issues.

refulgentis 3y ago

This is my favorite new AI argument, took me a few months to see it. Enjoyed it at first. You start with everyone knows there's AI hype from tech bros. Then you introduce a PhD or two at institutions with good names. Then they start grumbling about anthropomorphizing and who knows what AI is anyway. Somehow, if it's long enough, you forget that this kind of has nothing to do with anything. There is no argument. Just imagining other people must believe crazy things and working backwards from there to find something to critique. Took me a bit to realize it's not even an argument, just parroting "it's a stochastic parrot!" Assumes other people are dunces and genuinely believe it's a minihuman. I can't believe MIT Tech Review is going for this, the only argument here is the tests are flawed if you think they're supposed to show the AI model is literally human.

dleslie 3y ago

Two years ago I didn't use AI at all. Now I wouldn't go without it; I have Copilot integrated with Emacs, VSCode, and Rider. I consider it a ground-breaking productivity accelerator, a leap similar to when I transitioned from Turbo Pascal 2 to Visual C 6. That's why I'm hyped. If it's that good for me, and it's generalizable, then it's going to rock the world.

airstrike 3y ago
Coding on something without copilot these days feels like having my hands tied. I'm looking at you, XCode and Colab...
thomasfromcdnjs 3y ago
Life longer programmer, and same sentiments, I use it everywhere I can. I am currently transliterating a language PDF into a formatted lexicon, I wouldn't even be able to do this without co-pilot, it has turned this seemingly impossibly arduous task into a pleasurable one.

Kalanos 3y ago

Didn't it perform well on both the SAT and LSAT though?

javier_e06 3y ago

As a developer when I work with ChatGPT I can see ChatGPT eventually taking over my JIRA stories. Then ChatGPT will take over management creating product roadmaps, prioritizing and assigning tasks to itself. All dictated by customer feedback. The clock is ticking. But reasoning like a human? No.

waynenilsen 3y ago

This article is absurd. > But when a large language model scores well on such tests, it is not clear at all what has been measured. Is it evidence of actual understanding? A mindless statistical trick? Rote repetition? It is measuring how well it does _at REPLACING HUMANS_. It is hard to believe how the author clearly does not understand this. I don't care how it obtains its results. GPT-4 is like a hyperspeed entry to mid level dev that has almost no ability to contextualize. Tools built on top of 32k will allow repo ingestion. This is the worst it will ever be.

COAGULOPATH 3y ago
>It is measuring how well it does _at REPLACING HUMANS_ It's possible to do well on a test and have no ability to do the thing the job tests for. GPT-4 scores well on an advanced sommelier exam, but obviously cannot replace a human sommelier, because it does not have a mouth.
iudqnolq 3y ago
GPT passed a test on the theoretical fundamentals of selling and serving wine in fancy restaurants. In a human passing such a test provides a useful signal of job suitability because people who pass it are often also capable of the physical bits, like theatrically opening wine bottles. But obviously that doesn't work for an AI. Lots of things usefully correlate with test scores in humans but might not in an AI.
dartos 3y ago
Which tests test specifically for “replacing humans?” That seems like a wild metric to try and capture in a test. Also an aside: > This is the worse it will ever be. I hear this a lot and it really bothers me. Just because something is the worst it’ll ever be doesn’t mean it’ll get much better. There could always be a plateau on the horizon. It’s akin to “just have faith.” A real weird sentiment that I didn’t notice in tech before 2021.
RandomLensman 3y ago
It is measuring how well it does replacing humans - in those tests.

derbOac 3y ago

This was interesting to me but mostly because of a question I thought it was going to focus on, which is how should we interpret these tests when a human takes it? I wasn't sure that the phenomena they discussed was as relevant to the question of whether AI is overhyped as they made it out to be, but I did think a lot of questions about the meaning of the performances were important. What's interesting to me is you could flip this all on its head and, instead of asking "what can we infer about the machine processes these test scores are measuring?", we could ask "what does this imply about the human processes these test scores are measuring?" A lot of these test are well-validated but overinterpreted I think, and leaned on too heavily to make inferences about people. If a machine can pass a test, for instance, what does it say about the test as used in people? Should we be putting as much weight on them as we do? I'm not arguing these tests are useless or something, just that maybe we read into them too much to begin with.

robertlagrant 3y ago

> AI hype is built on high test scores No, it's built on people using DALLE and Midjourney and ChatGPT.

yCombLinks 3y ago
Exactly, chatpgt is double checking my homework problems and pointing out my errors, it's teaching me the material better than any of my lectures. It's writing tons of code I'm getting paid for, with way less overhead than trying to explain the problem to a junior, less mistakes and faster iteration. Test scores, ridiculous

iambateman 3y ago

This really is a good article, and is seriously researched. But the conclusion in the headline - “AI hype is built on flawed test scores” - feels like a poor summary of the article. It _is_ correct to say that an LLM is not ready to be a medical doctor, even if it can pass the test. But I think a better conclusion is that test scores don’t help us understand LLM capabilities like we think they do. Using a human test for an LLM is like measuring a car’s “muscles” and calling it horsepower. They’re just different. But the AI hype is justified, even if we struggle to measure it.

mcguire 3y ago

"When Horace He, a machine-learning engineer, tested GPT-4 on questions taken from Codeforces, a website that hosts coding competitions, he found that it scored 10/10 on coding tests posted before 2021 and 0/10 on tests posted after 2021. Others have also noted that GPT-4’s test scores take a dive on material produced after 2021. Because the model’s training data only included text collected before 2021, some say this shows that large language models display a kind of memorization rather than intelligence." I'm sure that is just a matter of prompt engineering, though.

COAGULOPATH 3y ago
But it got 10/10 on pre-2021 questions, with the same prompting method...

danielvaughn 3y ago

I remember watching a documentary about an old blues guitar player from the 1920's. They were trying to learn more about him and track down his whereabouts during certain periods of his life. At one point, they showed some old footage which featured a montage of daily life in a small Mississippi town. You'd see people shopping for groceries, going on walks, etc. Some would stop and wave at the camera. In the documentary, they noted that this footage exists because at the time, they'd show it on screen during intermission at movie theaters. Film was still in its infancy in that time, and was so novel that people loved seeing themselves and other people on the big screen. It was an interesting use of a new technology, and today it's easy to understand why it died out. Of course, it likely wasn't obvious at the time. I say all that because I don't think we can know at this point what AI is capable of, and how we want to use it, but we should expect to see lots of failure while we figure it out. Over the next decade there's undoubtedly going to be countless ventures similar to the "show the townspeople on the movie screen" idea, blinded by the novelty of technological change. But failed ventures have no relevance to the overall impact or worth of the technology itself.

kenjackson 3y ago
What died out? Film?
- actionfromafar 3y ago
  Showing locals little movie clips of themselves in intermissions at the local theater.
- savanaly 3y ago
  >What died out? The custom of showing film consisting of footage of the general public in movie theaters.
- danielvaughn 3y ago
  The practice of filming a montage around your local neighborhood or town to play during intermission. Though you could say intermission as well, since that was a legacy concept that was inherited from plays and eventually died out as well.

rvz 3y ago

Most of the hype comes from the AI grifters who need to find the next sucker to dump their VC shares onto to the next greater fool to purchase their ChatGPT-wrapper snake oil project to at an overvalued asking price. The ones who have to dismantle the hype are the proper technologies such as Yann LeCun and Grady Booch who know exactly what they are talking about.

rvz 3y ago
*technologists

aldousd666 3y ago

Only idiots are basing their excitement about what's possible on those test scores. They're just an attempt to measure one bot against another. There is a strong possibility that they are only measuring how well the bot takes the test, and nothing at all about what the tests themselves purport to measure. I mean, those tests are probably similar to stuff that's in the training data.

ehutch79 3y ago
Yeah... there's a lot of idiots out there.

randcraw 3y ago

The debate over whether LLMs are "intelligent" seem a lot like the old debate among NLP experts whether English must be modeled as a context-free grammar (push down automaton) or finite-state machine (regular expression). Yes, any language can be modeled using regular expressions; you just need an insane number of FSMs (perhaps billions). And that seems to be the model that LLMs are using to model cognition today. LLMs seem to use little or no abstract reasoning (is-a) or hierarchical perception (has-a), as humans do -- both of which are grounded in semantic abstraction. Instead, LLMs can memorize a brute force explosion in finite state machines (interconnected with Word2Vec-like associations) and then traverse those machines and associations as some kind of mashup, akin to a coherent abstract concept. Then as LLMs get bigger and bigger, they just memorize more and more mashup clusters of FSMs augmented with associations. Of course, that's not how a human learns, or reasons. It seems likely that synthetic cognition of this kind will fail to enable various kinds of reasoning that humans perceive as essential and normal (like common sense based on abstraction, or physically-grounded perception, or goal-based or counterfactual reasoning, much less insight into the thought processes / perceptions of other sentient beings). Even as ever-larger LLMs "know more" by memorizing ever more FSMs, I suspect they'll continue to surprise us with persistent cognitive and perceptual deficits that would never arise in organic beings that do use abstract reasoning and physically grounded perception.

freejazz 3y ago
Can you recommend any books on this?
TeMPOraL 3y ago
> LLMs can memorize a brute force explosion in finite state machines (interconnected with Word2Vec-like associations) and then traverse those machines and associations as some kind of mashup, akin to a coherent abstract concept. That's actually the closest to a working definition of what a concept is. The discussion about language representation has little bearing on humans or intelligence, because it's not how we learn and use language. Similarly, the more people - be it armchair or diploma-carrying philosophers - try to find the essence of a meaning of some word, the more they fail, because it seems that meaning of any concept is defined entirely through associations with other concepts and some remembered experiences. Which again seems pretty similar to how LLMs encode information through associations in high-dimensional spaces.

aidenn0 3y ago

Any task that gets solved with AI retroactively becomes something that doesn't require reasoning.

janalsncm 3y ago
I wouldn’t say that. Chess certainly requires reasoning even if that reasoning is minimax. I suppose in the context of this article “AI” means statistical language models.
- nomel 3y ago
  Why does chess require reasoning? Do all of the these [1] "reason"? ChatGPT-4 is supposedly rated worse than 500, in this list (1400 or so, although I think a recent update improved it a bit). [1] https://ccrl.chessdom.com/ccrl/4040/ https://ccrl.chessdom.com/ccrl/4040/
  - janalsncm 3y ago
    Within the domain of chess, searching the domain of possible future positions is synonymous with reasoning. It requires an explicit understanding of latent board representations and an explicit understanding of possible actions and the consequences of those actions. Whether ChatGPT has any of those things is questionable. From what I’ve seen, it has an unreliable latent representation, an unreliable understanding of possible moves, and a positional understanding that’s little better than a coin flip.
    - nomel 3y ago
      > Within the domain of chess, searching the domain of possible future positions is synonymous with reasoning. So, with this definition, these chess engines already exhibit (fairly substantial) reasoning? Or, are you saying it would be required, in the context of an LLM?

janalsncm 3y ago

The only test I need is the amount of time it takes me to do common tasks with and without ChatGPT. I’m aware it’s not perfect but perfect was never necessary.

rahimnathwani 3y ago

“People have been giving human intelligence tests—IQ tests and so on—to machines since the very beginning of AI,” says Melanie Mitchell, an artificial-intelligence researcher at the Santa Fe Institute in New Mexico. “The issue throughout has been what it means when you test a machine like this. It doesn’t mean the same thing that it means for a human.” The last sentence above is an important point that most people don't consider.

api 3y ago
It seems a bit like having a human face off in a race against a car and then concluding that cars have exceeded human physical dexterity. It's not an apples/apples comparison. The nature of the capability profile of a human vs. any known machine is radically different. Machines are intentionally designed to have extreme peaks of performance in narrow areas. Present-generation AI might be wider in its capabilities than what we've previously built, but it's still rather narrow as you quickly discover if you start trying to use it on real tasks.