10 ms·

If you expect the AI to do independent work, yes, it is a dead end. These LLM AIs need to be treated and handled as what they are: idiot savants with vast and

If you expect the AI to do independent work, yes, it is a dead end. These LLM AIs need to be treated and handled as what they are: idiot savants with vast and unreliable intelligence. What does any advanced organization do when they hire a new PhD, let them loose in the company or pair them with experienced staff? When paired with experienced staff, they use the new person for their knowledge but do not let them change things on their own until much later, when confidence is established and the new staffer has been exposed to how things work "around here". The big difference with LLM AIs is they never graduate to an experienced staffer, they are always the idiot savant that is really dang smart but also clueless and needs to be observed. That means the path forward with this current state of LLM AIs is to pair them with people, personalized to their needs, and treat them as very smart idiot savants great for strategy and problem solving discussion, where the human users are driving the situation, using the LLM AIs like a smart assistant that requires validation - just like a real new hire. There is an interactive state that can be achieved with these LLM AIs, like being in a conversation with experts, where they advise, they augment and amplify individual persons. A group of individuals adept with use of such an idiot savant enhanced environment would be incredibly capable. They'd be a force unseen in human civilization before today.

Alex3917 2y ago
> The big difference with LLM AIs is they never graduate to an experienced staffer, they are always the idiot savant that is really dang smart but also clueless and needs to be observed. Basically this. They already have vastly better-than-human ability at finding syntax errors within code, which on its own is quite useful; think of how many people have probably dropped out of CS as a major after staying up all night and failing to find a missing semicolon.
- lionkor 2y ago
  I don't know of a single person who got so stuck on syntax errors that they quit
  - FroshKiller 2y ago
    Added to which we already have tools that are great at finding syntax errors. They're called compilers.
    - Philpax 2y ago
      Compilers can detect errors in the grammar, but they cannot infer what your desired intent was. Even the best compilers in the diagnostics business (rustc, etc) aren't mind-readers. A LLM isn't perfect, but it's much more capable of figuring out what you wanted to do and what went wrong than a compiler is.
      - lionkor 2y ago
        none of that is a syntax issue, though, that's semantics
  - bsenftner 2y ago
    Try being a TA to freshmen CS majors; a good 1/3 change majors because they can't handle the syntax strictness coupled with their generally untrained logical mind. They convince themselves it is "too hard" and their buddies over in the business school are having a heck of a lot of fun throwing parties...
    - cesaref 2y ago
      Sounds like CS is not for them, and they find something else to do which is more applicable to their skills and interest. This is good. I don't think you should see a high drop out rate from a course as necessarily indicating a problem.
      - Philpax 2y ago
        Losing potentially good talent because they don't know how or where to look for mistakes yet is foolhardy. I'm happy for them to throw in the towel if the field is truly not for them, but I would wager that a not-insignificant portion of that crowd would be able to meaningfully progress once they get past the immediate hurdles in front of them.
- deleted 2y ago
  [deleted]
- fire_lake 2y ago
  Syntax checking is not an “AI” problem - use any compiler or linter.
- rsynnott 2y ago
  … One odd thing I’ve noticed about the people who are very enthusiastic about the use of LLMs in programming is that they appear to be unaware of any _other_ programming tools. Like, this is a solved problem, more or less; code-aware editors have been a thing since the 90s (maybe before?)
  - torginus 2y ago
    true.. in the past few days I used my time off to work on my hobby video game - writing the game logic required me to consider problems that, are quite self-contained and domain specific, and probably globally unique (if not particularly complex). I started out in Cursor, but I quickly realized Claude's erudite knowledge of AWS would not help me here, but what I needed was to refactor the code quickly and often, so that I'd finally find the perfect structure. For that, IDE tools were much more appropriate than AI wizardry.
  - Alex3917 2y ago
    > code-aware editors have been a thing since the 90s These will do things like highlight places where you're trying to call a method that isn't defined on the object, but they don't understand the intent of what you're trying to do. The latter is actually important in terms of being able to point you toward the correct solution.
- dgfitz 2y ago
  I know a lot of people who dropped out of CS in college. Not a single one dropped out because of a semicolon syntax issue.
  - CalRobert 2y ago
    I spent 8 hours trying to fix a bug once because notepad used smart quotation marks (really showing my age here - and now I'm pretty annoyed that the instructor was telling us to use notepad, but it was 2001 and I didn't know any better).
    - dgfitz 2y ago
      I did something like that once too, a long time ago. And because of that I see syntax errors of such I’ll within seconds now, having learned once the hard way.
      - CalRobert 2y ago
        I also know how important the right tools are. I should’ve been using vi.
        dgfitz 2y ago
        It amuses me to no end: in college I was a total emacs person, and now I’m a total vi person. There might be dozens of us! I think both editors have their flaws, I just know the native vi keybindings better at the moment.
        CalRobert 2y ago
        Either of them would've made the problem obvious! This was about a million years ago. I had just installed a pirated copy of Windows XP (FCKGW-RHQQ2...) and was in the first quarter of my physics degree, taking a class in C. Different times....
- raincole 2y ago
  > think of how many people have probably dropped out of CS as a major after staying up all night and failing to find a missing semicolon. ... like a dozen? And in 100% cases it's their teacher's fault.
- layer8 2y ago
  They are still worse at finding syntax errors than the actual parser. And at best they could be equally good. So what’s the point?
lobsterthief 2y ago
I agree with all of what you said except this: > idiot savants with vast and unreliable intelligence. Remember, intelligence !== knowledge. These LLMs indeed have vast and unreliable knowledge banks.
- bsenftner 2y ago
  Yes, you are correct. They provide knowledge and the human is the operator of the intelligence portion.
  - uxhacker 2y ago
    It goes back to the old wisdom DIKW pyramid. EDITED My ASCI art pyramid did not work. So imagine a pyramid with DATA at the bottom, INFORMATION on top of the data, and KNOWLEDGE sitting on top of the INFORMATION, with WISDOM at the top. And then trying top guess where AI is? Some people say that Information is the knowing, what, knowledge the how, and Wisdom the why.
- wanderingstan 2y ago
  In general conversation, “intelligence”, “knowledge”, “smartness”, “expertise”, etc are used mostly interchangeably. If we want to get pedantic, I would point out that “knowledge” is formally defined as “justified true belief”, and I doubt we want to get into the quagmire of whether LLM’s actually have beliefs. I took OP’s point in the casual meaning, i.e. that LLMs are like what I would call an “intelligent coworker”, or how one might call a Jeopardy game show contestant as intelligent.
skydhash 2y ago
One of the core tenet of technology is that it makes the job less consuming of a person resources (time, strength,…). While I’ve read a lot of claims, I’ve yet to see someone make a proper argument on how LLMs can be such a tool. > A group of individuals adept with use of such an idiot savant enhanced environment would be incredibly capable. They'd be a force unseen in human civilization before today More than the people who landed someone on the moon?
- bsenftner 2y ago
  They would be capable of landing someone on the moon, if they chose to pursue that goal, and had the finances to do so. And they'd do so with fewer people too.
  - wizzwizz4 2y ago
    I have witnessed no evidence that would support this claim. The only contribution of LLMs to mathematics is in being useful to Terry Tao: they're not capable of solving novel orbital mechanics problems (except through brute-force search, constrained sufficiently that you could chuck a uniform distribution in and get similar outputs). That's before you get into any of the engineering problems.
    - bsenftner 2y ago
      You do not have them solving such problems, but you do have them in the conversation as the human experts knowledgeable in that area work to solve the problem. This is not the LLM AIs doing independent work, this is them interactively working with the human person that is capable of solving that problem, it is their career, and the AI just makes them better at it, but not by doing their work, but by advising them as they work.
      - wizzwizz4 2y ago
        But they aren't useful for that. Terry Tao uses them to improve his ability to use poorly-documented boilerplatey things like Lean and matplotlib, but receiving advice from them‽ Frankly, if a chatbot is giving you much better advice than a rubber duck, you're either a Jack-of-all-Trades (in which case, I'd recommend better tools) or a https://ploum.net/2024-12-23-julius-en.html https://ploum.net/2024-12-23-julius-en.html Julius (in which case, I'd recommend staying away from anything important). I recommend reading his interview with Matteo Wong, where he proposes the opposite: https://www.theatlantic.com/technology/archive/2024/10/terence-tao-ai-interview/680153/ https://www.theatlantic.com/technology/archive/2024/10/teren... > With o1, you can kind of do this. I gave it a problem I knew how to solve, and I tried to guide the model. First I gave it a hint, and it ignored the hint and did something else, which didn’t work. When I explained this, it apologized and said, “Okay, I’ll do it your way.” And then it carried out my instructions reasonably well, and then it got stuck again, and I had to correct it again. The model never figured out the most clever steps. It could do all the routine things, but it was very unimaginative. I agree with his overall vision, but transformer-based chatbots will not be the AI algorithm that supports it. Highly-automated proof assistants like Isabelle's Sledgehammer are closer (and even those are really, really crude, compared to what we could have).
coliveira 2y ago
It's even worse. AI is a really smart but inexperienced person who also lies frequently. Because AI is not accountable to anything, it'll always come up with a reasonable answer to any question, if it is correct or not.
- belZaah 2y ago
  To put it in other words: it is not clear when and how they hallucinate. With a person, their competence could be understood and also their limits. But a llm can happily give different answers based on trivial changes in the question with no warning.
  - zozbot234 2y ago
    LLM's are non-deterministic: they'll happily give different answers to the same prompt based on nothing at all. This is actually great if you want to use them for "creative" content generation tasks, which is IMHO what they're best at. (Along with processing of natural language input.) Expecting them to do non-trivial amounts of technical or mathematical reasoning, or even something as simple as code generation (other than "translate these complex natural-language requirements into a first sketch of viable computer code") is a total dead end; these will always be language systems first and foremost.
    - mapt 2y ago
      This confuses me. You have your model, you have your tokens. If the tokens are bit-for-bit-identical, where does the non-determinism come in? If the tokens are only roughly-the-same-thing-to-a-human, sure I guess, but convergence on roughly the same output for roughly the same input should be inherently a goal of LLM development.
      - zozbot234 2y ago
        The model outputs probabilities, which you have to sample randomly. Choosing the "highest" probability every time leads to poor results in practice, such as the model tending to repeat itself. It's a sort of Monte-Carlo approach.
api 2y ago
After using them for a long time I am convinced they have no true intelligence beyond what is latent in training data. In other words I think we are kind of fooling ourselves. That being said they are very useful. I mostly use them as a far superior alternative to web search and as a kind of junior research assistant. Anything they find must be checked of course. I think we have invented the sci-fi trope of the AI librarian of the galactic archive. It can’t solve problems but it can rifle through the totality of human knowledge and rapidly find things. It’s a search engine.
- weakfish 2y ago
  I mean, it’s known that there’s no intelligence if you simply look at how it works on a technical level - it’s a prediction of the next token. That wasn’t really ever in question as to whether they have “intelligence”
  - api 2y ago
    To people who really understand them and are grounded, I think you're right. There has been a lot of hype among people who don't understand them as much, a lot of hype among the public, and a lot of schlock about "superintelligence" and "hard takeoff" etc. among smart but un-grounded people. The latter kind of fear mongering hype has been exploited by companies like ClosedAI in a bid for regulatory capture.
    - danielbln 2y ago
      A little humility would do us good regardless, because we don't know what intelligence is and what consciousness is, we can't truly define it nor do we understand what makes humans conscious and sentient/sapient. Categorically ruling out intelligence because "it's just a token predictor" puts us at the opposite of the spectrum, and that's not necessarily a better place to be.
  - jghn 2y ago
    > it’s known that there’s no intelligence To you & I that's true. But especially for the masses that's not true. It seems like at least once to day that I either talk to someone or hear someone via tv/radio/etc who does not understand this. An example that amused me recently was a radio talk show host who had a long segment describing how he & a colleague had a long argument with ChatGPT to correct a factual inaccuracy about their radio show. And that they finally convinced ChatGPT that they were correct due to their careful use of evidence & reasoning. And the part they were most happy about was how it had now learned, and going forward ChatGPT would not spread these inaccuracies. That anecdote is how the public at large sees these tools.
coffeefirst 2y ago
The problem is it's still a computer. And that's okay. I can ask the computer "hey I know this thing exists in your training data, tell me what it is and cite your sources." This is awesome. Seriously. But what that means is you can ask it for sample code, or to answer a legal question, but fundamentally you're getting a search engine reading something back to you. It is not a programmer and it is not a lawyer. The hype train really wants to exaggerate this to "we're going to steal all the jobs" because that makes the stock price go up. They would be far less excited about that if they read a little history.
- insane_dreamer 2y ago
  > "we're going to steal all the jobs" It won't steal them all, but it will have a major impact by stealing the lower level jobs which are more routine in nature -- but the problem is that those lower level jobs are necessary to gain the experience needed to get to the higher level jobs. It also won't eliminate jobs completely, but it will greatly reduce the number of people needed for a particular job. So the impact that it will have on certain trades -- translators, paralegals, journalists, etc. -- is significant.
  - deleted 2y ago
    [deleted]
- ethbr1 2y ago
  The thing that makes the smarter search use case interesting is how LLMs are doing their search result calculations: dynamically and at metadata scales previously impossible. LLM-as-search is essentially the hand-tuned expert systems AI vs deep learning AI battle all over again. Between natural language understanding and multiple correlations, it's going to scale a lot further than previous search approaches.
- cshores 2y ago
  I find it fascinating that I can achieve about 85-90% of what I need for simple coding projects in my homelab using AI. These projects often involve tasks like scraping data from the web and automating form submissions. My workflow typically starts with asking ChatGPT to analyze a webpage where I need to authenticate. I guide it to identify the username and password fields, and it accurately detects the credential inputs. I then inform it about the presence of a session cookie that maintains login persistence. Next, I show it an example page with links—often paginated with numbered navigation at the bottom—and ask it to recognize the pattern for traversing pages. It does so effectively. I further highlight the layout pattern of the content, such as magnet links or other relevant data presented by the CMS. From there, I instruct it to generate a Python script that spiders through each page sequentially, navigates to every item on those pages, and pushes magnet links directly into Transmission. I can also specify filters, such as only targeting items with specific media content, by providing a sample page for the AI to analyze before generating the script. This process demonstrates how effortlessly AI enables coding without requiring prior knowledge of libraries like beautifulsoup4 or transmission_rpc. It not only builds the algorithm but also allows for rapid iteration. Through this exercise, I assume the role of a manager, focusing solely on explaining my requirements to the AI and conducting a code review.
insane_dreamer 2y ago
> with vast and unreliable intelligence I would say "knowledge" rather than "intelligence" The key feature of LLMs is the vast amounts of information and data they have access to, and their ability to quickly process and summarize, using well-written prose, that information based on pattern matching.
- ethbr1 2y ago
  This is what LLM (and AI in general) naysayers are missing. LLMs will likely never get us to 100% solutions on a large class of problems. But! A lot of problems can be converted into versions with a subcomponent that LLMs can solve 100% of. And the fusion of LLMs doing 100% of that subportion + humans doing the remainder = increased productivity. Re-engineering problems to be LLM-tolerant, then using LLMs to automate that portion of the problem, is the winning approach.
- cess11 2y ago
  So you think machines running statistical inference have awareness. That's quite the embarassment if you actually mean it.
  - insane_dreamer 2y ago
    I said nothing of the sort
    - cess11 2y ago
      OK, so you have your own definition of knowledge. Please share it.
cactacea 2y ago
> A group of individuals adept with use of such an idiot savant enhanced environment would be incredibly capable. They'd be a force unseen in human civilization before today. I'm sorry but your comment is a good example of the logical shell game many people play with AI when applying it to general problem solving. Your LLM AI is both an idiot and an expert somehow? Where is this expertise derived from and why should you trust it? If LLMs were truly as revolutionary as all the grifters would have you believe then why do we not see "forces unseed in human civilization before today" by humans that employ armies of interns? That these supposed ubermensch do not presently exist is firm evidence in support of current AI being a dead end in my opinion. Humans are infinitely more capable than current AI, the limiting factor is time and money. Not capability!
- dylan604 2y ago
  > Your LLM AI is both an idiot and an expert somehow? Maybe you are unfamiliar with the term idiot savant?
  - cactacea 2y ago
    I am indeed familiar with the term. Savant and expert are not perfect synonyms. That is beside my point anyway.
monkeynotes 2y ago
I was so stupid when GPT3 came out. I knew so little about token prediction, I argued with folks on here that it was capable of so many things that I now understand just aren't compatible with the tech. Over the past couple of years of educating myself a bit, whilst I am no expert I have been anticipating a dead end. You can throw as much training at these things as you like, but all you'll get is more of the same with diminishing returns. Indeed in some research the quality of responses gets worse as you train it with more data. I am yet to see anything transformative out of LLMs other than demos which have prompt engineers working night and day to do something impressive with. Those Sora videos took forever to put together, and cost huge amounts of compute. No one is going to make a whole production quality movie with an LLM and disrupt Hollywood. I agree, an LLM is like an idiot savant, and whilst it's fantastic for everyone to have access to a savant, it doesn't change the world like the internet, or internal combustion engine did. OpenAI is heading toward some difficult decisions, they either admit their consumer business model is dead and go into competing with Amazon for API business (good luck), become a research lab (give up on being a billion dollar company), or get acquired and move on.
pnut 2y ago
Criticisms like this are levied against an excessively narrow (obsolete?) characterisation of what is happening in the AI space currently. After reading about o3's performance on ARC-AGI, I strongly suspect people will not be so flippantly dismissive of the inherent limits of these technologies by this time next year. I'm genuinely surprised at how myopic HN commentary is on this topic in general. Maybe because the implications are almost unthinkably profound. Anyway, OpenAI, Anthropic, Meta, and everyone else are well aware of these types of criticisms, and are making significant, measurable progress towards architecturally solving the deficiencies. https://arcprize.org/blog/oai-o3-pub-breakthrough https://arcprize.org/blog/oai-o3-pub-breakthrough
- jokethrowaway 2y ago
  Nah, the trick with o3 solving IQ tests seems to be that they bruteforce solutions and then pick the best option. That's why calls that are trivial for humans end up costing a lot. It still can't think and it won't think. LANGUAGE models (keyword: language) is a language model, it should be paired with a reasoning engine to translate the inner thought of the machine into human language. It should not be the source of decisions because it sucks at doing so, even though the network can exhibit some intelligence. We will never have AGI with just a language model. That said, most jobs people do are still at risk, even with chatgpt-3.5 (especially outside of knowledge work, where difficult decisions need to be taken). So we'll see the problems with AGI and the job market way earlier than AGI, as soon as we apply robotics and vision models + chatgpt 3.5 level intelligence. Goodbye baristas, goodbye people working in factories. Let's start working on a reasoning engine so we can replace those pesky knowledge workers too.
  - esafak 2y ago
    The important thing is that you can use inference-time computation to improve results. Now the race is on to optimize that.
  - ithkuil 2y ago
    How many attempts can you have when running an evaluation run of an ARC competition?
FuriouslyAdrift 2y ago
LLMs are fuzzy compression with a really good natural language parser...
- danielbln 2y ago
  And strong in-context learning, the true killer feature.
  - ithkuil 2y ago
    In order to understand natural language well you need quite a lot of general knowledge just to understand what the sentence actually means
ALittleLight 2y ago
You really shouldn't say LLMs "never graduate" to experienced staff - rather that they haven't yet. But there are recent and continuing improvements in the ability of the LLMs, and in time, perhaps a small amount of time, this situation may flip.
- bsenftner 2y ago
  I'm talking about the current SOTA. In the future, all bets are off. For today, they are very capable when paired with a capable person, and that is how one uses them successfully today. Tomorrow will be different, of course.
brookst 2y ago
I think you’ve exactly captured the two disparate views we see on HN: 1. LLMs have little value, are totally unreliable, will never amount to much because they don’t learn and grow and mature like people do., so they cannot replace a person like me who is well advanced in a career. 2. LLMs are incredible useful and will change the world because they excel at entry level work and can replace swaths of relatively undifferentiated information workers. LLM flaws are not that different from those workers’ flaws. I’m in camp 2, but I appreciate and agree with the articulation of why they will not replace every information worker.
layer8 2y ago
This all sounds plausible, but personally I find being paired to a new idiot-savant hire who never learns anything from the interaction incredibly exhausting. It can augment and amplify one’s own capabilities, but it’s also continuously frustrating and cumbersome.
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]

Alex3917 2y ago

> The big difference with LLM AIs is they never graduate to an experienced staffer, they are always the idiot savant that is really dang smart but also clueless and needs to be observed. Basically this. They already have vastly better-than-human ability at finding syntax errors within code, which on its own is quite useful; think of how many people have probably dropped out of CS as a major after staying up all night and failing to find a missing semicolon.

lionkor 2y ago
I don't know of a single person who got so stuck on syntax errors that they quit
- FroshKiller 2y ago
  Added to which we already have tools that are great at finding syntax errors. They're called compilers.
  - Philpax 2y ago
    Compilers can detect errors in the grammar, but they cannot infer what your desired intent was. Even the best compilers in the diagnostics business (rustc, etc) aren't mind-readers. A LLM isn't perfect, but it's much more capable of figuring out what you wanted to do and what went wrong than a compiler is.
    - lionkor 2y ago
      none of that is a syntax issue, though, that's semantics
- bsenftner 2y ago
  Try being a TA to freshmen CS majors; a good 1/3 change majors because they can't handle the syntax strictness coupled with their generally untrained logical mind. They convince themselves it is "too hard" and their buddies over in the business school are having a heck of a lot of fun throwing parties...
  - cesaref 2y ago
    Sounds like CS is not for them, and they find something else to do which is more applicable to their skills and interest. This is good. I don't think you should see a high drop out rate from a course as necessarily indicating a problem.
    - Philpax 2y ago
      Losing potentially good talent because they don't know how or where to look for mistakes yet is foolhardy. I'm happy for them to throw in the towel if the field is truly not for them, but I would wager that a not-insignificant portion of that crowd would be able to meaningfully progress once they get past the immediate hurdles in front of them.
deleted 2y ago
[deleted]
fire_lake 2y ago
Syntax checking is not an “AI” problem - use any compiler or linter.
rsynnott 2y ago
… One odd thing I’ve noticed about the people who are very enthusiastic about the use of LLMs in programming is that they appear to be unaware of any _other_ programming tools. Like, this is a solved problem, more or less; code-aware editors have been a thing since the 90s (maybe before?)
- torginus 2y ago
  true.. in the past few days I used my time off to work on my hobby video game - writing the game logic required me to consider problems that, are quite self-contained and domain specific, and probably globally unique (if not particularly complex). I started out in Cursor, but I quickly realized Claude's erudite knowledge of AWS would not help me here, but what I needed was to refactor the code quickly and often, so that I'd finally find the perfect structure. For that, IDE tools were much more appropriate than AI wizardry.
- Alex3917 2y ago
  > code-aware editors have been a thing since the 90s These will do things like highlight places where you're trying to call a method that isn't defined on the object, but they don't understand the intent of what you're trying to do. The latter is actually important in terms of being able to point you toward the correct solution.
dgfitz 2y ago
I know a lot of people who dropped out of CS in college. Not a single one dropped out because of a semicolon syntax issue.
- CalRobert 2y ago
  I spent 8 hours trying to fix a bug once because notepad used smart quotation marks (really showing my age here - and now I'm pretty annoyed that the instructor was telling us to use notepad, but it was 2001 and I didn't know any better).
  - dgfitz 2y ago
    I did something like that once too, a long time ago. And because of that I see syntax errors of such I’ll within seconds now, having learned once the hard way.
    - CalRobert 2y ago
      I also know how important the right tools are. I should’ve been using vi.
      - dgfitz 2y ago
        It amuses me to no end: in college I was a total emacs person, and now I’m a total vi person. There might be dozens of us! I think both editors have their flaws, I just know the native vi keybindings better at the moment.
        CalRobert 2y ago
        Either of them would've made the problem obvious! This was about a million years ago. I had just installed a pirated copy of Windows XP (FCKGW-RHQQ2...) and was in the first quarter of my physics degree, taking a class in C. Different times....
raincole 2y ago
> think of how many people have probably dropped out of CS as a major after staying up all night and failing to find a missing semicolon. ... like a dozen? And in 100% cases it's their teacher's fault.
layer8 2y ago
They are still worse at finding syntax errors than the actual parser. And at best they could be equally good. So what’s the point?

lobsterthief 2y ago

I agree with all of what you said except this: > idiot savants with vast and unreliable intelligence. Remember, intelligence !== knowledge. These LLMs indeed have vast and unreliable knowledge banks.

bsenftner 2y ago
Yes, you are correct. They provide knowledge and the human is the operator of the intelligence portion.
- uxhacker 2y ago
  It goes back to the old wisdom DIKW pyramid. EDITED My ASCI art pyramid did not work. So imagine a pyramid with DATA at the bottom, INFORMATION on top of the data, and KNOWLEDGE sitting on top of the INFORMATION, with WISDOM at the top. And then trying top guess where AI is? Some people say that Information is the knowing, what, knowledge the how, and Wisdom the why.
wanderingstan 2y ago
In general conversation, “intelligence”, “knowledge”, “smartness”, “expertise”, etc are used mostly interchangeably. If we want to get pedantic, I would point out that “knowledge” is formally defined as “justified true belief”, and I doubt we want to get into the quagmire of whether LLM’s actually have beliefs. I took OP’s point in the casual meaning, i.e. that LLMs are like what I would call an “intelligent coworker”, or how one might call a Jeopardy game show contestant as intelligent.

skydhash 2y ago

One of the core tenet of technology is that it makes the job less consuming of a person resources (time, strength,…). While I’ve read a lot of claims, I’ve yet to see someone make a proper argument on how LLMs can be such a tool. > A group of individuals adept with use of such an idiot savant enhanced environment would be incredibly capable. They'd be a force unseen in human civilization before today More than the people who landed someone on the moon?

bsenftner 2y ago
They would be capable of landing someone on the moon, if they chose to pursue that goal, and had the finances to do so. And they'd do so with fewer people too.
- wizzwizz4 2y ago
  I have witnessed no evidence that would support this claim. The only contribution of LLMs to mathematics is in being useful to Terry Tao: they're not capable of solving novel orbital mechanics problems (except through brute-force search, constrained sufficiently that you could chuck a uniform distribution in and get similar outputs). That's before you get into any of the engineering problems.
  - bsenftner 2y ago
    You do not have them solving such problems, but you do have them in the conversation as the human experts knowledgeable in that area work to solve the problem. This is not the LLM AIs doing independent work, this is them interactively working with the human person that is capable of solving that problem, it is their career, and the AI just makes them better at it, but not by doing their work, but by advising them as they work.
    - wizzwizz4 2y ago
      But they aren't useful for that. Terry Tao uses them to improve his ability to use poorly-documented boilerplatey things like Lean and matplotlib, but receiving advice from them‽ Frankly, if a chatbot is giving you much better advice than a rubber duck, you're either a Jack-of-all-Trades (in which case, I'd recommend better tools) or a https://ploum.net/2024-12-23-julius-en.html https://ploum.net/2024-12-23-julius-en.html Julius (in which case, I'd recommend staying away from anything important). I recommend reading his interview with Matteo Wong, where he proposes the opposite: https://www.theatlantic.com/technology/archive/2024/10/terence-tao-ai-interview/680153/ https://www.theatlantic.com/technology/archive/2024/10/teren... > With o1, you can kind of do this. I gave it a problem I knew how to solve, and I tried to guide the model. First I gave it a hint, and it ignored the hint and did something else, which didn’t work. When I explained this, it apologized and said, “Okay, I’ll do it your way.” And then it carried out my instructions reasonably well, and then it got stuck again, and I had to correct it again. The model never figured out the most clever steps. It could do all the routine things, but it was very unimaginative. I agree with his overall vision, but transformer-based chatbots will not be the AI algorithm that supports it. Highly-automated proof assistants like Isabelle's Sledgehammer are closer (and even those are really, really crude, compared to what we could have).

coliveira 2y ago

It's even worse. AI is a really smart but inexperienced person who also lies frequently. Because AI is not accountable to anything, it'll always come up with a reasonable answer to any question, if it is correct or not.

belZaah 2y ago
To put it in other words: it is not clear when and how they hallucinate. With a person, their competence could be understood and also their limits. But a llm can happily give different answers based on trivial changes in the question with no warning.
- zozbot234 2y ago
  LLM's are non-deterministic: they'll happily give different answers to the same prompt based on nothing at all. This is actually great if you want to use them for "creative" content generation tasks, which is IMHO what they're best at. (Along with processing of natural language input.) Expecting them to do non-trivial amounts of technical or mathematical reasoning, or even something as simple as code generation (other than "translate these complex natural-language requirements into a first sketch of viable computer code") is a total dead end; these will always be language systems first and foremost.
  - mapt 2y ago
    This confuses me. You have your model, you have your tokens. If the tokens are bit-for-bit-identical, where does the non-determinism come in? If the tokens are only roughly-the-same-thing-to-a-human, sure I guess, but convergence on roughly the same output for roughly the same input should be inherently a goal of LLM development.
    - zozbot234 2y ago
      The model outputs probabilities, which you have to sample randomly. Choosing the "highest" probability every time leads to poor results in practice, such as the model tending to repeat itself. It's a sort of Monte-Carlo approach.

api 2y ago

After using them for a long time I am convinced they have no true intelligence beyond what is latent in training data. In other words I think we are kind of fooling ourselves. That being said they are very useful. I mostly use them as a far superior alternative to web search and as a kind of junior research assistant. Anything they find must be checked of course. I think we have invented the sci-fi trope of the AI librarian of the galactic archive. It can’t solve problems but it can rifle through the totality of human knowledge and rapidly find things. It’s a search engine.

weakfish 2y ago
I mean, it’s known that there’s no intelligence if you simply look at how it works on a technical level - it’s a prediction of the next token. That wasn’t really ever in question as to whether they have “intelligence”
- api 2y ago
  To people who really understand them and are grounded, I think you're right. There has been a lot of hype among people who don't understand them as much, a lot of hype among the public, and a lot of schlock about "superintelligence" and "hard takeoff" etc. among smart but un-grounded people. The latter kind of fear mongering hype has been exploited by companies like ClosedAI in a bid for regulatory capture.
  - danielbln 2y ago
    A little humility would do us good regardless, because we don't know what intelligence is and what consciousness is, we can't truly define it nor do we understand what makes humans conscious and sentient/sapient. Categorically ruling out intelligence because "it's just a token predictor" puts us at the opposite of the spectrum, and that's not necessarily a better place to be.
- jghn 2y ago
  > it’s known that there’s no intelligence To you & I that's true. But especially for the masses that's not true. It seems like at least once to day that I either talk to someone or hear someone via tv/radio/etc who does not understand this. An example that amused me recently was a radio talk show host who had a long segment describing how he & a colleague had a long argument with ChatGPT to correct a factual inaccuracy about their radio show. And that they finally convinced ChatGPT that they were correct due to their careful use of evidence & reasoning. And the part they were most happy about was how it had now learned, and going forward ChatGPT would not spread these inaccuracies. That anecdote is how the public at large sees these tools.

coffeefirst 2y ago

The problem is it's still a computer. And that's okay. I can ask the computer "hey I know this thing exists in your training data, tell me what it is and cite your sources." This is awesome. Seriously. But what that means is you can ask it for sample code, or to answer a legal question, but fundamentally you're getting a search engine reading something back to you. It is not a programmer and it is not a lawyer. The hype train really wants to exaggerate this to "we're going to steal all the jobs" because that makes the stock price go up. They would be far less excited about that if they read a little history.

insane_dreamer 2y ago
> "we're going to steal all the jobs" It won't steal them all, but it will have a major impact by stealing the lower level jobs which are more routine in nature -- but the problem is that those lower level jobs are necessary to gain the experience needed to get to the higher level jobs. It also won't eliminate jobs completely, but it will greatly reduce the number of people needed for a particular job. So the impact that it will have on certain trades -- translators, paralegals, journalists, etc. -- is significant.
- deleted 2y ago
  [deleted]
ethbr1 2y ago
The thing that makes the smarter search use case interesting is how LLMs are doing their search result calculations: dynamically and at metadata scales previously impossible. LLM-as-search is essentially the hand-tuned expert systems AI vs deep learning AI battle all over again. Between natural language understanding and multiple correlations, it's going to scale a lot further than previous search approaches.
cshores 2y ago
I find it fascinating that I can achieve about 85-90% of what I need for simple coding projects in my homelab using AI. These projects often involve tasks like scraping data from the web and automating form submissions. My workflow typically starts with asking ChatGPT to analyze a webpage where I need to authenticate. I guide it to identify the username and password fields, and it accurately detects the credential inputs. I then inform it about the presence of a session cookie that maintains login persistence. Next, I show it an example page with links—often paginated with numbered navigation at the bottom—and ask it to recognize the pattern for traversing pages. It does so effectively. I further highlight the layout pattern of the content, such as magnet links or other relevant data presented by the CMS. From there, I instruct it to generate a Python script that spiders through each page sequentially, navigates to every item on those pages, and pushes magnet links directly into Transmission. I can also specify filters, such as only targeting items with specific media content, by providing a sample page for the AI to analyze before generating the script. This process demonstrates how effortlessly AI enables coding without requiring prior knowledge of libraries like beautifulsoup4 or transmission_rpc. It not only builds the algorithm but also allows for rapid iteration. Through this exercise, I assume the role of a manager, focusing solely on explaining my requirements to the AI and conducting a code review.

insane_dreamer 2y ago

> with vast and unreliable intelligence I would say "knowledge" rather than "intelligence" The key feature of LLMs is the vast amounts of information and data they have access to, and their ability to quickly process and summarize, using well-written prose, that information based on pattern matching.

ethbr1 2y ago
This is what LLM (and AI in general) naysayers are missing. LLMs will likely never get us to 100% solutions on a large class of problems. But! A lot of problems can be converted into versions with a subcomponent that LLMs can solve 100% of. And the fusion of LLMs doing 100% of that subportion + humans doing the remainder = increased productivity. Re-engineering problems to be LLM-tolerant, then using LLMs to automate that portion of the problem, is the winning approach.
cess11 2y ago
So you think machines running statistical inference have awareness. That's quite the embarassment if you actually mean it.
- insane_dreamer 2y ago
  I said nothing of the sort
  - cess11 2y ago
    OK, so you have your own definition of knowledge. Please share it.

cactacea 2y ago

> A group of individuals adept with use of such an idiot savant enhanced environment would be incredibly capable. They'd be a force unseen in human civilization before today. I'm sorry but your comment is a good example of the logical shell game many people play with AI when applying it to general problem solving. Your LLM AI is both an idiot and an expert somehow? Where is this expertise derived from and why should you trust it? If LLMs were truly as revolutionary as all the grifters would have you believe then why do we not see "forces unseed in human civilization before today" by humans that employ armies of interns? That these supposed ubermensch do not presently exist is firm evidence in support of current AI being a dead end in my opinion. Humans are infinitely more capable than current AI, the limiting factor is time and money. Not capability!

dylan604 2y ago
> Your LLM AI is both an idiot and an expert somehow? Maybe you are unfamiliar with the term idiot savant?
- cactacea 2y ago
  I am indeed familiar with the term. Savant and expert are not perfect synonyms. That is beside my point anyway.

monkeynotes 2y ago

I was so stupid when GPT3 came out. I knew so little about token prediction, I argued with folks on here that it was capable of so many things that I now understand just aren't compatible with the tech. Over the past couple of years of educating myself a bit, whilst I am no expert I have been anticipating a dead end. You can throw as much training at these things as you like, but all you'll get is more of the same with diminishing returns. Indeed in some research the quality of responses gets worse as you train it with more data. I am yet to see anything transformative out of LLMs other than demos which have prompt engineers working night and day to do something impressive with. Those Sora videos took forever to put together, and cost huge amounts of compute. No one is going to make a whole production quality movie with an LLM and disrupt Hollywood. I agree, an LLM is like an idiot savant, and whilst it's fantastic for everyone to have access to a savant, it doesn't change the world like the internet, or internal combustion engine did. OpenAI is heading toward some difficult decisions, they either admit their consumer business model is dead and go into competing with Amazon for API business (good luck), become a research lab (give up on being a billion dollar company), or get acquired and move on.

pnut 2y ago

Criticisms like this are levied against an excessively narrow (obsolete?) characterisation of what is happening in the AI space currently. After reading about o3's performance on ARC-AGI, I strongly suspect people will not be so flippantly dismissive of the inherent limits of these technologies by this time next year. I'm genuinely surprised at how myopic HN commentary is on this topic in general. Maybe because the implications are almost unthinkably profound. Anyway, OpenAI, Anthropic, Meta, and everyone else are well aware of these types of criticisms, and are making significant, measurable progress towards architecturally solving the deficiencies. https://arcprize.org/blog/oai-o3-pub-breakthrough https://arcprize.org/blog/oai-o3-pub-breakthrough

jokethrowaway 2y ago
Nah, the trick with o3 solving IQ tests seems to be that they bruteforce solutions and then pick the best option. That's why calls that are trivial for humans end up costing a lot. It still can't think and it won't think. LANGUAGE models (keyword: language) is a language model, it should be paired with a reasoning engine to translate the inner thought of the machine into human language. It should not be the source of decisions because it sucks at doing so, even though the network can exhibit some intelligence. We will never have AGI with just a language model. That said, most jobs people do are still at risk, even with chatgpt-3.5 (especially outside of knowledge work, where difficult decisions need to be taken). So we'll see the problems with AGI and the job market way earlier than AGI, as soon as we apply robotics and vision models + chatgpt 3.5 level intelligence. Goodbye baristas, goodbye people working in factories. Let's start working on a reasoning engine so we can replace those pesky knowledge workers too.
- esafak 2y ago
  The important thing is that you can use inference-time computation to improve results. Now the race is on to optimize that.
- ithkuil 2y ago
  How many attempts can you have when running an evaluation run of an ARC competition?

FuriouslyAdrift 2y ago

LLMs are fuzzy compression with a really good natural language parser...

danielbln 2y ago
And strong in-context learning, the true killer feature.
- ithkuil 2y ago
  In order to understand natural language well you need quite a lot of general knowledge just to understand what the sentence actually means

ALittleLight 2y ago

You really shouldn't say LLMs "never graduate" to experienced staff - rather that they haven't yet. But there are recent and continuing improvements in the ability of the LLMs, and in time, perhaps a small amount of time, this situation may flip.

bsenftner 2y ago
I'm talking about the current SOTA. In the future, all bets are off. For today, they are very capable when paired with a capable person, and that is how one uses them successfully today. Tomorrow will be different, of course.

brookst 2y ago

I think you’ve exactly captured the two disparate views we see on HN: 1. LLMs have little value, are totally unreliable, will never amount to much because they don’t learn and grow and mature like people do., so they cannot replace a person like me who is well advanced in a career. 2. LLMs are incredible useful and will change the world because they excel at entry level work and can replace swaths of relatively undifferentiated information workers. LLM flaws are not that different from those workers’ flaws. I’m in camp 2, but I appreciate and agree with the articulation of why they will not replace every information worker.

layer8 2y ago

This all sounds plausible, but personally I find being paired to a new idiot-savant hire who never learns anything from the interaction incredibly exhausting. It can augment and amplify one’s own capabilities, but it’s also continuously frustrating and cumbersome.

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]