8 ms·

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would prob

• donatj 1y ago
  My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?
  • mock-possum 1y ago
    Same place the previous generation came from - college courses, DIY hackers, and learn-on-the-job training and apprenticeships. Why would that change?
  • maegul 1y ago
    This, for me, has been the question since the beginning. I’m yet to see anyone talk/think about the issue head on too. And whenever I’ve asked someone about it, they’ve not had any substantial thoughts.
    • PUSH_AX 1y ago
      Engineers will still exist and people will vibe code all kinds of things into existence. Some will break in spectacular ways, some of those projects will die, some will hire a real engineer to fix things. I cannot see us living in a world of ignorance where there are literally zero engineers and no one on the planet understands what's been generated. Weirdly we could end up in a place where engineering skills are niche and extremely lucrative.
      • deleted 1y ago
        [deleted]
  • shswkna 1y ago
    Most important question on this entire topic. Fast forward 30 years and modern civilisation is entirely dependent on our AI’s. Will deep insight and innovation from a human perspective perhaps come to a stop?
    • qzw 1y ago
      No, but it'll become a hobby or artistic pursuit, just like running, playing chess, or blacksmithing. But I personally think it's going to take longer than 30 years.
    • Earw0rm 1y ago
      No. Even with power tools, construction and joinery are physical work and require strength and skill. What is new is that you'll need the wisdom to figure out when the tool can do the whole job, and where you need to intervene and supervise it closely. So humans won't be doing any less thinking, rather they'll be putting their thinking to work in better ways.
      • skeeter2020 1y ago
        to use your own example though, many of these core skills are declining, mechanized or viewed through a historical lens vs. application. I don't know if this is net good or bad, but it is very different. Maybe humans will think as you say, but it feels like there will be significantly less diverse areas of thought. If you look at the front page of HN as a snapshot of "where's tech these days" it is very homgenous compared to the past. Same goes for the general internet and the AI-content continues to grow. IMO published works are a precursor to future human discovery, forming the basis of knowledge, community and growth.
    • brookst 1y ago
      Did musical creativity end with synths and sequencers? Tools will only amplify human skills. Sure, not everyone will choose to use tools for anything meaningful, but those people are driving human insight and innovation today anyway.
  • svara 1y ago
    LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don't do that. But if you're willing to learn, this is rocket fuel.
    • junon 1y ago
      On the flip side, I wanted to see what common 8 layer PCB stackups were yesterday. ChatGPT wasn't giving me an answer that really made sense. After googling a bit, I realized almost all of the top results were AI generated, and also had very little in the way of real experience or advice. It was extremely frustrating.
      • roxolotl 1y ago
        This is my big fear. We’re going to end up in a world where information that isn’t common is significantly more difficult to find than it is today.
        • andersa 1y ago
          It's going to be like the pre-internet dark ages, but worse. Back then you only didn't find the information. Now, you find unlimited information, but it is all wrong.
          • svara 1y ago
            I don't know, this sounds a lot like in the late 90s when we heard a lot about how anyone could put information on the internet and that you shouldn't trust what you read online. Well it turns out you can manage just fine. You shouldn't blindly trust anything. Not what you read, not what people say. Using LLMs effectively is a skill too, and that does involve deciding when and how to verify information.
  • kypro 1y ago
    In a few years hopefully the AI reviewers will be far more reliable than even the best human experts. This is generally how competency progresses in AI... For example, at one point a human + computer would have been the strongest combo in chess, now you'd be insane to allow a human to critic a chess bot because they're so unlikely to add value, and statistically a human in the loop would be far more likely to introduce error. Similar things can be said in fields like machine vision, etc. Software is about to become much higher quality and be written at much, much lower cost.
    • sarchertech 1y ago
      My prediction is that for that to happen we’ll need to figure out a way to measure software quality in the way we can measure a chess game, so that we can use synthetic data to continue improving the models. I don’t think we are anywhere close to doing that.
      • kypro 1y ago
        Not really... If you're an average company you're not concerned about producing perfect software, but optimising for some balance between cost and quality. At some point companies via capitalist forces will naturally realise that it's more productive to not have humans in the loop. A good analogy might be how machines gradually replaced textile workers in the 19th century. Were the machines better? Or was there a was to quantitatively measure the quality of their output? No. But at the end of the day companies which embraced the technology were more productive than those who didn't, and the quality didn't decrease enough (if it did at all) that customers would no longer do business with them – so these companies won out. The same will naturally happen in software over the next few years. You'd be an moron to hire a human expert for $200,000 to critic a cybersecurity optimised model which costs maybe a 100th of the cost of employing a human... And this would likely be true even if we assume the human will catch the odd thing the model wouldn't because there's no such thing as perfect security – it's always a trade off between cost and acceptable risk. Bookmark this and come back in a few years. I made similar predictions when ChatGPT first came out that within a few years agents would be picking up tickets and raising PRs. Everyone said LLMs were just stochastic parrots and this would not happen, well now it has and increasingly companies are writing more and more code with AI. At my company it's a little over 50% at the mo, but this is increasing every month.
  • risyachka 1y ago
    Use it or lose it. Experts will become those who use llm to learn and not to write code for them or solve tasks for them so they can build that skill.
  • paradox242 1y ago
    The implication is that they are hoping to bridge the gap between current AI capabilities and something more like AGI in the time it takes the senior engineers to leave the industry. At least, that's the best I can come up with, because they are kicking out all of the bottom rings of the ladder here in what otherwise seems like a very shortsighted move.
• marcusb 1y ago
  I’m puzzled when I hear people say ‘oh, I only use LLMs for things I don’t understand well. If I’m an expert, I’d rather do it myself.’ In addition to the ability to review output effectively, I find the more closely I’m able to describe what I want in the way another expert in that domain would, the better the LLM output. Which isn’t really that surprising for a statistical text generation engine.
  • _heimdall 1y ago
    That's why outsource most other things in our life though, why would it be different with LLMs? People don't learn how a car works before buying one, they just take it to a mechanic when it breaks. Most people don't know how to build a house, they have someone else build it and assume it was done well. I fully expect people to similarly have LLMs do what the person doesn't know how and assume the machine knew what to do.
    • marcusb 1y ago
      > why would it be different with LLMs? Because LLMs are not competent professionals to whom you might outsource tasks in your life. LLMs are statistical engines that make up answers all the time, even when the LLM “knows” the correct answer (i.e., has the correct answer hidden away in its weights.) I don’t know about you, but I’m able to validate something is true much more quickly and efficiently if it is a subject I know well.
      • _heimdall 1y ago
        > competent professionals That requires a lot of clarity and definition if you want to claim that LLMs aren't competent professionals. I assume we'd ultimately agree that LLMs aren't, but I'd add that many humans paid for a task aren't competent professionals either and, more importantly, that I can't distinguish the competent professionals from others without myself being competent enough in the topic. My point was that people have a long history of outsourcing to someone else, often to someone they have never met and never will. We do it for things that we have no real idea about and trust that the person doing it must have known what they were doing. I fully expect people to end up taking the same view of LLMs.
• ajmurmann 1y ago
  I've been using an llm to do much of a k8s deployment for me. It's quick to get something working but I've had to constantly remind it to use secrets instead of committing credentials in clear text. A dangerous way to fail. I wonder if in my case this is caused by the training data having lots of examples from online tutorials that omit security concerns to focus on the basics.
  • ants_everywhere 1y ago
    > It's quick to get something working but I've had to constantly remind it to use secrets instead of committing credentials in clear text. This is going to be a powerful feedback loop which you might call regression to the intellectual mean. On any task, most training data is going to represent the middle (or beginning) of knowledge about a topic. Most k8s examples will skip best practices, most react apps will be from people just learning react, etc. If you want the LLM to do best practices in every knowledge domain (assuming best practices can be consistently well defined), then you have to push it away from the mean of every knowledge domain simultaneously (or else work with specialized fine tuned models). As you continue to add training data it will tend to regress toward the middle because that's where most people are on most topics.
  • diggan 1y ago
    > my case this is caused by the training data having I think it's caused by you not having a strong enough system prompt. Once you've built up a slightly reusable system prompt for coding or for infra work, where you bit by bit build it up while using a specific model (since different models respond differently to prompts), you end up getting better and better responses. So if you notice it putting plaintext credentials in the code, add to the system prompt to not do that. With LLMs you really get what you ask for, and if you miss to specify anything, the LLM will do whatever the probabilities tells it to, but you can steer this by being more specific. Imagine you're talking to a very literal and pedantic engineer who argues a lot on HN and having to be very precise with your words, and you're like 80% of the way there :)
    • ajmurmann 1y ago
• bradly 1y ago
  I've found llms are very quick to add defaults, fallbacks, rescues–which all makes it very easy for code to look like it is working when it is not or will not. I call this out three different places in my CLAUDE.md trying to adjust for this, and still occasionally get.
  • pennomi 1y ago
    Agreed, I’ve found that LLMs favor wrapping entire functions in broad try:catch structures, making it nearly impossible to identify and debug problems. Dangerous if you don’t correct it.
• jstummbillig 1y ago
  You will always trust domain experts at some junction; you can't build a company otherwise. The question is: Can LLMs provide that domain expertise? I would argue, yes, clearly, given the development of the past 2 years, but obviously not on a straight line.
• ghuntley 1y ago
  See also: LLMs are mirrors of operator skill - https://ghuntley.com/mirrors https://ghuntley.com/mirrors
• loandbehold 1y ago
  Over time AI coding tools will be able to research domain knowledge. Current "AI Research" tools are already very good at it but they are not integrated with coding tools yet. The research could look at both public Internet as well as company documents that contain internal domain knowledge. Some of the domain knowledge is only in people's heads. That would need to be provided by the user.
  • wslh 1y ago
    I'd like to add a practical observation, even assuming much more capable AI in the future: not all failures are due to model limitations, sometimes it's about external [world] changes. For instance, I used Next.js to build a simple login page with Google auth. It worked great, even though I only had basic knowledge of Node.js and a bit of React. Then I tried adding a database layer using Prisma to persist users. That's where things broke. The integration didn't work, seemingly due to recent versions in Prisma or subtle breaking updates. I found similar issues discussed on GitHub and Reddit, but solving them required shifting into full manual debugging mode. My takeaway: even with improved models, fast-moving frameworks and toolchains can break workflows in ways that LLMs/ML (at least today) can't reason through or fix reliably. It's not always about missing domain knowledge, it's that the moving parts aren't in sync with the model yet.
    • SparkyMcUnicorn 1y ago
      Just close the loop and give it direct access to your console logs in chrome and node, then it can do the "full manual debugging" on its own. It's not perfect, and it's not exactly cheap, but it works.