4 ms·

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or done the reading myself. You have to be very credulous to think for even a second that anything like a human or even animal mentation is going on with these models unless your interaction with them is anything but glancing. Things I tried: 1) there are certain paradigms I find useful for game programming. I tried to use ChatGPT to implement these systems in my favorite programming language. It gave me code that generally speaking made no sense. It was very clear that it did not understand how code actually works. Eg: I asked it to use a hash table to make a certain task more efficient and it just created a temporary hash table in the inner loop which it then threw away when the loop was finished. The modification did not make the code more efficient than the previous version and missed the point of the suggestion entirely, even after repeated attempts to get it to correct the issue. 2) I'm vaguely interested in exploring SU(7) for a creative project. Asked to generate code to deal with this group resulted in clearly absurd garbage that again clearly indicated that while ChatGPT can generate vaguely plausible text about groups it doesn't actually understand anything about them. Eg: ChatGPT can say that SU(7) is made of matrices with unit norm but when asked to generate examples failed to generate any with this property. 3) A very telling experiment is to ask ChatGPT to generate logo code that draws anything beyond simple shapes. Totally unable to do so for obvious reasons. Using ChatGPT convinced me that if this technology is going to disrupt anything, its going to be _search_ rather than _people_. Its just a search engine with the benefit that it can do some simple analogizing and the downside that it has no idea how anything in the real world works and will confidently produce total garbage without telling you.

mannykannot 4y ago
LLMs are polarizing: while there is a lot of hype from some quarters, there's also a faction that seems only interested in dismissing them as the same old something-or-other. What I personally find most interesting about them is what seems to me to be their unreasonable effectiveness, despite their flaws and limitations, and what that might tell us about ourselves. The more one stresses how simple (conceptually) their method of operation is, the more surprising their capabilities seem - to the point where I wonder how much of everyday human dialogue is being produced this way. This vein of innovation may start showing diminishing returns at any time, but if it keeps going for a while, It might deliver insights into what human intelligence is.
- nathan_compton 4y ago
  Well, my take here is that LLMs are do about half of what makes human intelligence work well: which is recall tons of information using relatively broad and imprecise queries or reminders. And even this they don't really do: humans are quite capable of introspecting about what they know and this seems to be a challenge for this class of model. A challenge I don't see this architecture really easily addressing since, in the end, its just a big markov model with chunking. But anyway, the other part of the equation is missing here. When a human encounters a novel phenomenon about which they know little they can engage an entire separate system: one which _reasons_ about the system, uses principals and intuitions and iteration to produce new knowledge. That is the thing missing from LLMs: they don't ever really produce new knowledge, although they may reveal correlations between texts that people have yet to notice. I'm not fundamentally skeptical about whether artificial intelligence will ever get there. In fact, the progress of LLMs has me wondering if its not going to be sooner rather than later. But at this moment, I feel quite confident saying that LLMs are just knowledge retrieval systems with some pretty undesirable properties (and some pretty interesting ones). My _hunch_ is that the next step this is going to be like self driving cars, though: technology which appears stubbornly _just out of reach_ for an indeterminate amount of time.
aeternum 4y ago
Understanding the lifetime/scope of variables is not easy, many experienced programmers get it wrong, especially when moving between languages.
mikodin 4y ago
Edit: I also see that I am falling prey to exactly what the paper itself is talking about. "The more adept LLMs become at mimicking human language, the more vulnerable we become to anthropomorphism, to seeing the systems in which they are embedded as more human-like than they really are. This trend is amplified by the natural tendency to use philosophically loaded terms, such as "knows", "believes", and "thinks", when describing these systems." -- An ignorant statement / question I have is why are you using it write code? It's a chatbot, no? As you've mentioned, it's a really powerful search, and is like having a conversation with someone who is literally the internet. For example "What is the glycemic index of oatmeal?" "What is Eihei Dogen's opinion of the Self and how does it differ from Bassui's?" I get highly detailed and accurate output with these. The first question is simple and the second is far from it. It's breaking down two Zen masters experiences and comparing them in an amazing way. I've been thoroughly impressed with Chat GPT so far. Ask it to breakdown the high level points of a book you've read. Ask it to rewrite a song in the style of a different artist. It's so cool, I feel like I legitimately have an answer to any random question at my finger tips and have to do zero filtering for it.
- nathan_compton 4y ago
  "An ignorant statement / question I have is why are you using it write code? It's a chatbot, no? I've found it so incredibly useful to simply replace Google." Heard of Stack Exchange? I teach and I expect many students to use language models like ChatGPT to do their homework, which involves writing code. Lots of what people are doing with it is coding (there have been quite a few posts here using it that way). I've actually also used ChatGPT for literary/song writing experiments and it stinks, aesthetically. The lyrics it wrote, even with a lot of prompting, were totally asinine. And how could they not be?
alsodumb 4y ago
I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversational style search engine (with it without LLMs) if they wanted to, but a lot of websites would have complained about Google taking away traffic from them. I believe the same happened when Google started showing it’s own reviews instead of redirecting to Yelp. I wonder how openAI or any LLM powered search is going to deal with it. They don’t have to worry about it anytime soon, they still have a lot of time to get to a stage where they come anywhere close to the number of queries Google handles in a day, but it’ll be interesting to see how things go.
- nathan_compton 4y ago
  I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond that I don't really get it.
  - solidasparagus 4y ago
    The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow. LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable
- TeMPOraL 4y ago
  Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.
axg11 4y ago
I disagree that LLMs are overhyped, but it's very subjective. Are current LLMs a few steps from AGI? No. Will LLMs change the computing landscape? Yes, I believe they will. ChatGPT, without any major changes, is already the best tool out there for answering programming questions. Nothing else comes close. I can ask it to provide code for combining two APIs and it will give useful and clean output. No need to trudge through documentation, SEO-hacked articles, or 10 different Stack Overflow answers. Output quality will only improve from here. Does it sometimes make mistakes? Yes. There are also mistakes in many of the top SO answers, especially as your questions become more obscure. Aside from programming, how many other fields are there where LLMs will become an indispensable tool? I have a PhD and ChatGPT can write a more coherent paragraph on my thesis topic than most people in my field. It does this in seconds. If you give a human enough time, they will be able to do better than ChatGPT. The problem is, we're already producing more science within niche scientific fields than most scientists could ever read. As an information summary tool, I think LLMs will be revolutionary. LLMs can help individuals leverage knowledge in a way that's impossible today and has been impossible for the last 30 years since the explosion in the number of scientific publications.
- nathan_compton 4y ago
  It can reproduce a statistically plausible paragraph, certainly. But there is a great deal more to research than producing statistically plausible paragraphs. It doesn't _understand_ anything! I've actually worked on a project where there have been attempts to use GPT like models to summarize scientific results and the problem is it gets shit wrong all the time! You have to be an expert to separate the wheat from the chaff. It operates like a mendacious search engine pretending to be a person.
  - visarga 4y ago
    The problem is that we need to pair generative models with verification systems. We have the models, but no verification yet. Fortunately code and math are easier to verify. Some things require simulation. In other cases you can substitute an ensemble of solutions & picking the most frequent answer as consistency based verification. But for each domain we need to create verifiers and that will take some time. The good thing is that we'll be able to generate training data with our models by filtering the junk with the verifiers. Then we can retrain the models. It's important because we are getting to the limit of available training data. We need to generate more data, but it's worthless unless we verify it. If we succeed we can train GPT-5. Human data will be just 1%, the race is on to generate the master dataset of the future. I read in a recent paper that such a method was used to improve text captions in the LAION dataset. https://laion.ai/blog/laion-5b/ https://laion.ai/blog/laion-5b/
monkmartinez 4y ago
I disagree and think this is a very controversial opinion. Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit. I asked it to create a UI with input boxes for material, weight of material, price and loss due to wastage. The program takes all of those inputs and converts the material into grams from KG, pounds, ounces. It then calculates the price per gram and takes a loss percentage (estimate given by user). It then writes a text file and saves it to a directory. I literally pasted the code into VS code and had to change Tkinter to tkinter. Hit run and it worked flawlessly. I have NEVER used tkinter and it took about 30 minutes from start to finish. This morning, I asked my 9th grade son what he is learning in 9th grade biology. He told me he is learning cellular endocytosis. I asked chapGPT to explain endocytosis like I was a 5 year old and read it to him... he says; "Ask it to explain it like a scientist now." After that he said it was a really good and we started asking it all kinds of biology questions. I happen to agree that search will be the first thing disrupted. However, I think simply saying "search" doesn't come close to capturing how deep this will change the way we think, use and progress in terms of the way we define "search" right now.
- nathan_compton 4y ago
  I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you. I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exchange or in documentation somewhere it doesn't work that well. Like I said, its a search engine with a lot of downsides and some upsides.
  - marcinzm 4y ago
    > If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you. Fooling a 9th grader is amazing. That's a pretty well formed human being right there except with less life experience. Fundamentally no different from you in general reasoning terms except on a smaller set of information. So fooling you is merely a question of model size.
hodgesrm 4y ago
> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, parts of words, or individual characters including punctuation marks. Rather than focus on the limitations of this approach to answer general queries, which are manifest, it seems more interesting to ask a different question. Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? The answer would: 1. Illuminate where we can use LLMs safely. 2. Direct work to make them better. It's already impressive that within certain scopes ChatGPT gives very good answers, indeed better than most humans.
- dragonwriter 4y ago
  > Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? Which humans? Humans give a... fairly wide range of responses.
  - TeMPOraL 4y ago
    Average humans? Within 1 stdev from the mean?
    - dragonwriter 4y ago
      > Average humans? Within 1 stdev from the mean? This implies that performance has unqiue natural, objective, ratio-level (or at least, a unique consistent interval-level) measure. Otherwise the mean is, itself, meaningless. “How well you answer a question” doesn't seem to fit that, its maybe at best a (somewhat subjective, still) ordinal quality, so the median (or mode) is the only useful average. But I think you'll find that without restricting things more than “humans”, both the median and mode of responses to most prompts is... quite bad.
  - hodgesrm 4y ago
armoredkitten 4y ago
Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsible for the huge improvements in machine translation; my team has to deal with text in multiple languages, and these models are vital there too. We have used LLM on real-world problems that are in production now and are saving hundreds of person-hours of tedious work. ChatGPT? Yeah, it's neat. I'm sure people will find some useful niche for it. And I do think generative models will eventually have a big impact, once researchers find good ways to ground them to data and facts. This is already an active area of research -- combining generative LLMs with info retrieval methods, or targeting it to a specific context. (Meta just gave a talk last week at the NeurIPS conference about teaching a model to play Diplomacy, a game that mostly involves talking and negotiating deals with the other players. ChatGPT is too broad for that -- they just need a model that can talk about the state of the game board.) So in general, I'm optimistic about generative LLMs. But ChatGPT...is just a toy, really. It's not the solution -- it's one of the signposts along the way toward the real solution. It's a measure of progress.
- jerrygenser 4y ago
  Do you feel like for some tasks though that people might be too eager to reach for GPU and more expensive inference and training costs? In some projects I'm working on, I'm seeing close to 95F1 score using deep learning to do token classification based NER. However, using non-deep learning approaches (still training statistical models) I can get to 91F1 on my use case but have a much faster inference and not need to use a GPU. I'm similarly optimistic about LLM, but fear that a lot of other really useful workhouse algorithms and strategies are going to be pushed aside and people will forget about them. Back to my use case, some strategies I'm looking at are using cheap and fast CPU powered models/smaller models for inference and then based on certain signals decide whether a particular instance should be passed to a GPU based model for better accuracy.
  - 4y ago
macrolocal 4y ago
Points taken, but LLMs are still outpacing expert predictions, so empirically they’re under-hyped.
btbuildem 4y ago
It is very, very good with language, and very bad with facts and numbers. That's an oversimplification, but also the gist of it. You have to recognize how it works, why it works - then you can use it as basically an incredible superpower force multiplier.
tshaddox 4y ago
Maybe I’m just old, but there just isn’t much that I want to computers to tell me about that they don’t already do a decent job at. Everyone loves to complain about how bad Google search is, but I very rarely find myself desperately looking for something and unable to find it. There’s certainly no normal conversational interactions I can think of that I would love to have with a computer but have been unable to before ChatGPT and similar. That limits how impressed I can be by ChatGPT and similar beyond just being impressed by it on a purely technical level. And it’s certainly very technically impressive, but not in some transcendental way. It’s also very impressive how could recent video games with ray tracing look, or how good computers are at chess, or how many really cool databases there are these days, or how fast computers can sort data.
Jack000 4y ago
LLMs may be overhyped, but transformers in general are underhyped. LLMs make a lot of mistakes because they don't actually know what words mean. The key thing is though - it's much harder to generate coherent text when you don't know what the words mean. In a similar vein it's completely unreasonable to expect an LLM to perform visual tasks when it literally has no sense of sight. The fact that it can kind of sort of do these things at all is evidence of the super-human generalization potential of the transformer architecture. This isn't very obvious for English because we have prior knowledge of what words mean, but it's a lot more obvious when applied to languages humans don't understand, like DNA and amino acid sequences.
- fourfivefour 4y ago
  How can these things not know what words mean? Did you not see how they created a virtual machine under chatGPT? They told it to imitate bash and they typed ls, and cat jokes.txt and it outputted things completely identical to what you'd expect. Look it up. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ I don't see how you can explain this as not knowing what words mean. It KNOWS.
  - xg15 4y ago
    Yeah, that's the actual bit that baffles me about ChatGPT still. Producing coherent, fluent text is alright, but we could already sort of do that 20 years ago with markov models or even just grammars (see Chomsky). Understanding text in the depth that ChatGPT (and GPT-3) appear to understand the prompts is something entirely different and has to my knowledge never been archieved before the current architectures.
  - Jack000 4y ago
    LLMs are trained exclusively on text, which means they lack crucial context behind the meaning of sentences. The universe of information outside of pure text - vision, sound, etc is completely unknown to it. LLMs are basically the aliens in blindsight. They have a superhuman ability to memorize the context of words it has seen and generalize to new contexts, but it can never be perfect because it's working on incomplete information.
Al-Khwarizmi 4y ago
Do my core work? No, it's not going to, at the moment. But it's already saving me nontrivial amounts of time on tasks like "write a polite followup email reminding person X, who didn't reply to the email I sent last week, that the deadline for doing Y expires at date Z". I typically spend at least 3-4 minutes finding the words for such a trivial email and thinking how to write it best, e.g. trying to make the other person react without coming across as annoying, etc. (Being a non-native English speaker who communicates mostly in English at work may be a factor). ChatGPT is really good with words. Using it, it takes a few seconds and I can use the output with only trivial edits.
genidoi 4y ago
The difference in wether you think ChatGPT is game changing or another overhyped LLM seems to come down to: 1) do you acknowledge prompt engineering is a real skill set? 2) are you willing to improve your prompt engineering skill set through research and iteration? There is much to learn about prompt engineering from that “Linux VM in ChatGPT” post and other impressive examples (where the goal of is to constrain ChatGPT to only engage in a specific task)
fourfivefour 4y ago
I used chatGPT to solve a sqlite bug involving a query that was taking 4 seconds to run. I pasted the query and it identified many possible issues with the query including the offending problem (it was missing an index on a timestamp). It also passed 3/4 of our companies interview process including forging a resume that passed the recruiter filter. That being said, I COMPLETELY agree with you that chatGPT will not disrupt anything. Your example cases are completely as VALID as are my example cases. chatGPT is, however, the precursor to the thing that will disrupt everything.
Zababa 4y ago
> This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or done the reading myself. > You have to be very credulous to think for even a second that anything like a human or even animal mentation is going on with these models unless your interaction with them is anything but glancing. I've used ChatGPT, and I'd say it's right now as useful as a google search, which is already a lot. Most humans would be absolutely unable to help me (and probably you) for your projects because they aren't specialized in that area. That's not even talking about animals. I love my cats but they've never really helped me when programming.

mannykannot 4y ago

LLMs are polarizing: while there is a lot of hype from some quarters, there's also a faction that seems only interested in dismissing them as the same old something-or-other. What I personally find most interesting about them is what seems to me to be their unreasonable effectiveness, despite their flaws and limitations, and what that might tell us about ourselves. The more one stresses how simple (conceptually) their method of operation is, the more surprising their capabilities seem - to the point where I wonder how much of everyday human dialogue is being produced this way. This vein of innovation may start showing diminishing returns at any time, but if it keeps going for a while, It might deliver insights into what human intelligence is.

nathan_compton 4y ago
Well, my take here is that LLMs are do about half of what makes human intelligence work well: which is recall tons of information using relatively broad and imprecise queries or reminders. And even this they don't really do: humans are quite capable of introspecting about what they know and this seems to be a challenge for this class of model. A challenge I don't see this architecture really easily addressing since, in the end, its just a big markov model with chunking. But anyway, the other part of the equation is missing here. When a human encounters a novel phenomenon about which they know little they can engage an entire separate system: one which _reasons_ about the system, uses principals and intuitions and iteration to produce new knowledge. That is the thing missing from LLMs: they don't ever really produce new knowledge, although they may reveal correlations between texts that people have yet to notice. I'm not fundamentally skeptical about whether artificial intelligence will ever get there. In fact, the progress of LLMs has me wondering if its not going to be sooner rather than later. But at this moment, I feel quite confident saying that LLMs are just knowledge retrieval systems with some pretty undesirable properties (and some pretty interesting ones). My _hunch_ is that the next step this is going to be like self driving cars, though: technology which appears stubbornly _just out of reach_ for an indeterminate amount of time.

aeternum 4y ago

Understanding the lifetime/scope of variables is not easy, many experienced programmers get it wrong, especially when moving between languages.

mikodin 4y ago

Edit: I also see that I am falling prey to exactly what the paper itself is talking about. "The more adept LLMs become at mimicking human language, the more vulnerable we become to anthropomorphism, to seeing the systems in which they are embedded as more human-like than they really are. This trend is amplified by the natural tendency to use philosophically loaded terms, such as "knows", "believes", and "thinks", when describing these systems." -- An ignorant statement / question I have is why are you using it write code? It's a chatbot, no? As you've mentioned, it's a really powerful search, and is like having a conversation with someone who is literally the internet. For example "What is the glycemic index of oatmeal?" "What is Eihei Dogen's opinion of the Self and how does it differ from Bassui's?" I get highly detailed and accurate output with these. The first question is simple and the second is far from it. It's breaking down two Zen masters experiences and comparing them in an amazing way. I've been thoroughly impressed with Chat GPT so far. Ask it to breakdown the high level points of a book you've read. Ask it to rewrite a song in the style of a different artist. It's so cool, I feel like I legitimately have an answer to any random question at my finger tips and have to do zero filtering for it.

nathan_compton 4y ago
"An ignorant statement / question I have is why are you using it write code? It's a chatbot, no? I've found it so incredibly useful to simply replace Google." Heard of Stack Exchange? I teach and I expect many students to use language models like ChatGPT to do their homework, which involves writing code. Lots of what people are doing with it is coding (there have been quite a few posts here using it that way). I've actually also used ChatGPT for literary/song writing experiments and it stinks, aesthetically. The lyrics it wrote, even with a lot of prompting, were totally asinine. And how could they not be?

alsodumb 4y ago

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversational style search engine (with it without LLMs) if they wanted to, but a lot of websites would have complained about Google taking away traffic from them. I believe the same happened when Google started showing it’s own reviews instead of redirecting to Yelp. I wonder how openAI or any LLM powered search is going to deal with it. They don’t have to worry about it anytime soon, they still have a lot of time to get to a stage where they come anywhere close to the number of queries Google handles in a day, but it’ll be interesting to see how things go.

nathan_compton 4y ago
I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond that I don't really get it.
- solidasparagus 4y ago
  The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow. LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable
TeMPOraL 4y ago
Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.

axg11 4y ago

I disagree that LLMs are overhyped, but it's very subjective. Are current LLMs a few steps from AGI? No. Will LLMs change the computing landscape? Yes, I believe they will. ChatGPT, without any major changes, is already the best tool out there for answering programming questions. Nothing else comes close. I can ask it to provide code for combining two APIs and it will give useful and clean output. No need to trudge through documentation, SEO-hacked articles, or 10 different Stack Overflow answers. Output quality will only improve from here. Does it sometimes make mistakes? Yes. There are also mistakes in many of the top SO answers, especially as your questions become more obscure. Aside from programming, how many other fields are there where LLMs will become an indispensable tool? I have a PhD and ChatGPT can write a more coherent paragraph on my thesis topic than most people in my field. It does this in seconds. If you give a human enough time, they will be able to do better than ChatGPT. The problem is, we're already producing more science within niche scientific fields than most scientists could ever read. As an information summary tool, I think LLMs will be revolutionary. LLMs can help individuals leverage knowledge in a way that's impossible today and has been impossible for the last 30 years since the explosion in the number of scientific publications.

nathan_compton 4y ago
It can reproduce a statistically plausible paragraph, certainly. But there is a great deal more to research than producing statistically plausible paragraphs. It doesn't _understand_ anything! I've actually worked on a project where there have been attempts to use GPT like models to summarize scientific results and the problem is it gets shit wrong all the time! You have to be an expert to separate the wheat from the chaff. It operates like a mendacious search engine pretending to be a person.
- visarga 4y ago
  The problem is that we need to pair generative models with verification systems. We have the models, but no verification yet. Fortunately code and math are easier to verify. Some things require simulation. In other cases you can substitute an ensemble of solutions & picking the most frequent answer as consistency based verification. But for each domain we need to create verifiers and that will take some time. The good thing is that we'll be able to generate training data with our models by filtering the junk with the verifiers. Then we can retrain the models. It's important because we are getting to the limit of available training data. We need to generate more data, but it's worthless unless we verify it. If we succeed we can train GPT-5. Human data will be just 1%, the race is on to generate the master dataset of the future. I read in a recent paper that such a method was used to improve text captions in the LAION dataset. https://laion.ai/blog/laion-5b/ https://laion.ai/blog/laion-5b/

monkmartinez 4y ago

I disagree and think this is a very controversial opinion. Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit. I asked it to create a UI with input boxes for material, weight of material, price and loss due to wastage. The program takes all of those inputs and converts the material into grams from KG, pounds, ounces. It then calculates the price per gram and takes a loss percentage (estimate given by user). It then writes a text file and saves it to a directory. I literally pasted the code into VS code and had to change Tkinter to tkinter. Hit run and it worked flawlessly. I have NEVER used tkinter and it took about 30 minutes from start to finish. This morning, I asked my 9th grade son what he is learning in 9th grade biology. He told me he is learning cellular endocytosis. I asked chapGPT to explain endocytosis like I was a 5 year old and read it to him... he says; "Ask it to explain it like a scientist now." After that he said it was a really good and we started asking it all kinds of biology questions. I happen to agree that search will be the first thing disrupted. However, I think simply saying "search" doesn't come close to capturing how deep this will change the way we think, use and progress in terms of the way we define "search" right now.

nathan_compton 4y ago
I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you. I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exchange or in documentation somewhere it doesn't work that well. Like I said, its a search engine with a lot of downsides and some upsides.
- marcinzm 4y ago
  > If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you. Fooling a 9th grader is amazing. That's a pretty well formed human being right there except with less life experience. Fundamentally no different from you in general reasoning terms except on a smaller set of information. So fooling you is merely a question of model size.

hodgesrm 4y ago

> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, parts of words, or individual characters including punctuation marks. Rather than focus on the limitations of this approach to answer general queries, which are manifest, it seems more interesting to ask a different question. Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? The answer would: 1. Illuminate where we can use LLMs safely. 2. Direct work to make them better. It's already impressive that within certain scopes ChatGPT gives very good answers, indeed better than most humans.

dragonwriter 4y ago
> Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? Which humans? Humans give a... fairly wide range of responses.
- TeMPOraL 4y ago
  Average humans? Within 1 stdev from the mean?
  - dragonwriter 4y ago
    > Average humans? Within 1 stdev from the mean? This implies that performance has unqiue natural, objective, ratio-level (or at least, a unique consistent interval-level) measure. Otherwise the mean is, itself, meaningless. “How well you answer a question” doesn't seem to fit that, its maybe at best a (somewhat subjective, still) ordinal quality, so the median (or mode) is the only useful average. But I think you'll find that without restricting things more than “humans”, both the median and mode of responses to most prompts is... quite bad.
- hodgesrm 4y ago

armoredkitten 4y ago

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsible for the huge improvements in machine translation; my team has to deal with text in multiple languages, and these models are vital there too. We have used LLM on real-world problems that are in production now and are saving hundreds of person-hours of tedious work. ChatGPT? Yeah, it's neat. I'm sure people will find some useful niche for it. And I do think generative models will eventually have a big impact, once researchers find good ways to ground them to data and facts. This is already an active area of research -- combining generative LLMs with info retrieval methods, or targeting it to a specific context. (Meta just gave a talk last week at the NeurIPS conference about teaching a model to play Diplomacy, a game that mostly involves talking and negotiating deals with the other players. ChatGPT is too broad for that -- they just need a model that can talk about the state of the game board.) So in general, I'm optimistic about generative LLMs. But ChatGPT...is just a toy, really. It's not the solution -- it's one of the signposts along the way toward the real solution. It's a measure of progress.

jerrygenser 4y ago
Do you feel like for some tasks though that people might be too eager to reach for GPU and more expensive inference and training costs? In some projects I'm working on, I'm seeing close to 95F1 score using deep learning to do token classification based NER. However, using non-deep learning approaches (still training statistical models) I can get to 91F1 on my use case but have a much faster inference and not need to use a GPU. I'm similarly optimistic about LLM, but fear that a lot of other really useful workhouse algorithms and strategies are going to be pushed aside and people will forget about them. Back to my use case, some strategies I'm looking at are using cheap and fast CPU powered models/smaller models for inference and then based on certain signals decide whether a particular instance should be passed to a GPU based model for better accuracy.
- 4y ago

macrolocal 4y ago

Points taken, but LLMs are still outpacing expert predictions, so empirically they’re under-hyped.

btbuildem 4y ago

It is very, very good with language, and very bad with facts and numbers. That's an oversimplification, but also the gist of it. You have to recognize how it works, why it works - then you can use it as basically an incredible superpower force multiplier.

tshaddox 4y ago

Maybe I’m just old, but there just isn’t much that I want to computers to tell me about that they don’t already do a decent job at. Everyone loves to complain about how bad Google search is, but I very rarely find myself desperately looking for something and unable to find it. There’s certainly no normal conversational interactions I can think of that I would love to have with a computer but have been unable to before ChatGPT and similar. That limits how impressed I can be by ChatGPT and similar beyond just being impressed by it on a purely technical level. And it’s certainly very technically impressive, but not in some transcendental way. It’s also very impressive how could recent video games with ray tracing look, or how good computers are at chess, or how many really cool databases there are these days, or how fast computers can sort data.

Jack000 4y ago

LLMs may be overhyped, but transformers in general are underhyped. LLMs make a lot of mistakes because they don't actually know what words mean. The key thing is though - it's much harder to generate coherent text when you don't know what the words mean. In a similar vein it's completely unreasonable to expect an LLM to perform visual tasks when it literally has no sense of sight. The fact that it can kind of sort of do these things at all is evidence of the super-human generalization potential of the transformer architecture. This isn't very obvious for English because we have prior knowledge of what words mean, but it's a lot more obvious when applied to languages humans don't understand, like DNA and amino acid sequences.

fourfivefour 4y ago
How can these things not know what words mean? Did you not see how they created a virtual machine under chatGPT? They told it to imitate bash and they typed ls, and cat jokes.txt and it outputted things completely identical to what you'd expect. Look it up. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ I don't see how you can explain this as not knowing what words mean. It KNOWS.
- xg15 4y ago
  Yeah, that's the actual bit that baffles me about ChatGPT still. Producing coherent, fluent text is alright, but we could already sort of do that 20 years ago with markov models or even just grammars (see Chomsky). Understanding text in the depth that ChatGPT (and GPT-3) appear to understand the prompts is something entirely different and has to my knowledge never been archieved before the current architectures.
- Jack000 4y ago
  LLMs are trained exclusively on text, which means they lack crucial context behind the meaning of sentences. The universe of information outside of pure text - vision, sound, etc is completely unknown to it. LLMs are basically the aliens in blindsight. They have a superhuman ability to memorize the context of words it has seen and generalize to new contexts, but it can never be perfect because it's working on incomplete information.

Al-Khwarizmi 4y ago

Do my core work? No, it's not going to, at the moment. But it's already saving me nontrivial amounts of time on tasks like "write a polite followup email reminding person X, who didn't reply to the email I sent last week, that the deadline for doing Y expires at date Z". I typically spend at least 3-4 minutes finding the words for such a trivial email and thinking how to write it best, e.g. trying to make the other person react without coming across as annoying, etc. (Being a non-native English speaker who communicates mostly in English at work may be a factor). ChatGPT is really good with words. Using it, it takes a few seconds and I can use the output with only trivial edits.

genidoi 4y ago

The difference in wether you think ChatGPT is game changing or another overhyped LLM seems to come down to: 1) do you acknowledge prompt engineering is a real skill set? 2) are you willing to improve your prompt engineering skill set through research and iteration? There is much to learn about prompt engineering from that “Linux VM in ChatGPT” post and other impressive examples (where the goal of is to constrain ChatGPT to only engage in a specific task)

fourfivefour 4y ago

I used chatGPT to solve a sqlite bug involving a query that was taking 4 seconds to run. I pasted the query and it identified many possible issues with the query including the offending problem (it was missing an index on a timestamp). It also passed 3/4 of our companies interview process including forging a resume that passed the recruiter filter. That being said, I COMPLETELY agree with you that chatGPT will not disrupt anything. Your example cases are completely as VALID as are my example cases. chatGPT is, however, the precursor to the thing that will disrupt everything.

Zababa 4y ago

> This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or done the reading myself. > You have to be very credulous to think for even a second that anything like a human or even animal mentation is going on with these models unless your interaction with them is anything but glancing. I've used ChatGPT, and I'd say it's right now as useful as a google search, which is already a lot. Most humans would be absolutely unable to help me (and probably you) for your projects because they aren't specialized in that area. That's not even talking about animals. I love my cats but they've never really helped me when programming.