4 ms·

Not sure I agree with this. The math guy at anthropic's prompts are essentially: "suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, ever

• zmj 2mo ago
  It's not contradictory to say that expertise is a multiplier, and that models are systematically underconfident in themselves.
  • titzer 2mo ago
    It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
• atleastoptimal 2mo ago
  This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence. Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction. However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
  • colechristensen 2mo ago
    Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about". This includes things like "before you start fixing this bug, write two tests that fail proving it exists". Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
• natsucks 2mo ago
  And what about problems that cannot be one-shotted but helped along?
  • fragmede 2mo ago
    There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it. Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
    • colechristensen 2mo ago
      Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes. The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
      • bonoboTP 2mo ago
        > Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
        • colechristensen 2mo ago
          > why do you think this would be the case? Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them. Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
• randysalami 2mo ago
  LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression. Finally, we train our LLMs on who we are. Another reinforcement of biases.
  • nullsanity 2mo ago
    [dead]
• deleted 2mo ago
  [deleted]
• budsniffer952 2mo ago
  [flagged]
  • _dwt 2mo ago
    He's on to us, boys! Call in a code 43. (Sorry, I'm in a crappy mood, but what on Earth are we supposed to take away from this? Everyone who disagrees with you is secretly an idiot, or worse, they're smart enough to know they're idiots but too proud to admit it?) On a more helpful note, I think your "confusion" if honest can probably be resolved by realizing that "skeptics" are not a monolith.
  • ofjcihen 2mo ago
    This is becoming such a common statement on here it’s almost copy pasta. Maybe the answer is more along the lines of “people are using them for different things and getting different results”? Why does it have to be snark and “these people must be stupid”
    • Terr_ 2mo ago
      Or even that people are using them for the same things, but there is a chaotic element to how it works, and people are getting different outcomes due to small differences that we humans don't think ought to matter.
    • bonoboTP 2mo ago
      It's because in many cases it exactly sounds like what happened. They asked something one shot, then seeing that the AI didn't quite solve the same task that the prompter had in mind, he just closes it and decides that AI is stupid and can't even do the task and it's all hype. I say, "what you just told me right now, did you even try typing that into the chat with the AI?", and they just stare at me. You have to iterate! You have to provide context, you have to provide access to the tools you also use to do your job. The other day someone complained here on HN that AI failed to optimize his code speed. Turns out he just pasted in the code, didn't use an agentic harness with end-to-end benchmarking ability for the model to ground its changes in and to hill-climb on. But even as a human you need to test your hypotheses and measure things, and sometimes something you thought would help actually makes it slower. It happens over and over, but it's no skin off my nose. If they don't want to learn to use it, it's on them.
• bonoboTP 2mo ago
  It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car. But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference. Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
  • foolswisdom 2mo ago
    So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
  • kamaal 2mo ago
    In short being a high agency person is now more important than ever.
• davidw 2mo ago
  > who these models reward/empower The easy, straightforward answer is "the people who own the models". Who else benefits feels like a more complex question and we'll have to see...
• antonvs 2mo ago
  > Like all of life, maybe the "winner" is the person who just does stuff. Someone who just does stuff still has to be able to deal with errors and failures. That’s where an expert or a generalist may have an advantage.
• jkhdigital 2mo ago
  But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
  • gr_norm 2mo ago
    Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding. I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
    • a2ff6eeb0 2mo ago
      > But at that point, what is the point of you versus going to the LLM myself? reply You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?" And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
      • deleted 2mo ago
        [deleted]
• its-summertime 2mo ago
  Who's end state took / is going to take more tokens / money, however? "LLMs reward expertise" is the title, not that "LLMs only make things possible for those with expertise"
• matherial 2mo ago
  So how many conjectures have you proved in your spare time?... As the old joke goes, a mechanic charges you $5 for hitting it with a wrench and $495 for knowing what and where to hit.
• defgeneric 2mo ago
  Exactly. The post read to me as another variation of the denial that people with expertise are reaching for right now. My sense is that we as programmers went through it over a year ago already (perhaps not all of us, but at least anyone paying attention), and so it's easy to overlook that it's still new to people who do other forms of "knowledge work," i.e. people whose identity is bound up with their expertise. My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor. One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
  • znnajdla 2mo ago
    What sort of denial are you reading into the author's post? I don't read it as denial. He is fully acknowledging the usefulness of AI.
    • defgeneric 2mo ago
      He's saying: "Expertise improves AI-assisted work, experts can steer better and get more out of models." The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process." That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place. I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
• adoltech 2mo ago
  [dead]
• uncivilized 2mo ago
  Hacker News is the biggest collection of idiots who think they are geniuses, so they will upvote anything that makes them feel smarter.