5 ms·

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior enginee

• ChrisLTD 5mo ago
  > the idea that Amazon would allow human bottlenecks to appear across projects and underlying infrastructure is ridiculous. Why?
  • SoftTalker 5mo ago
    Amazon is fairly well known to ruthlessly optimize every process. So if they're having humans proofread what the AI produces, they must have found that to be necessary.
    • ChrisLTD 5mo ago
      Yeah, that's what I think too. They aren't going to care about optimizing a process that leads to poor results.
    • bluefirebrand 5mo ago
      Amazon is not immune to making mistakes
• _puk 5mo ago
  Part of the problem: you get given a complete document to review after it's been fully baked. I'm pushing the need for basic engineering principles across whole organisations. You wouldn't give an engineer 1000 lines of code to review without the original spec of what you're trying to achieve for context (at a minimum, ideally the reviewer was in the room when the work was introduced, and has full context). So, these docs, they're given as an all or nothing. Do you push back on the 39th metric that is defined to the utmost detail? Or just resign yourself to the fact that it is what it is? A one (6 is the goto if we're talking Amazon?!) pager.. "this is what I am proposing" at least gives the skeleton of the idea to push back at the general shape of the idea, refine it, before all the emotional investment of your precious report being complete. Y'know.. the traditional product running through the spec in a SCRUM* environment.. the engineers doing proper code reviews.. * Yes SCRUM is dead, but that's another thing.
  • JoshTriplett 5mo ago
    > Part of the problem: you get given a complete document to review after it's been fully baked. Not fully baked, worse: made to sound confidently correct, orthogonal to its actual correctness.
    • bradleyankrom 5mo ago
      Like the fake food they make for commercials. Looks great on TV.
  • s0rce 5mo ago
    I've had this situation and basically just had to throw out stuff that was written because its completely terrible/wrong. Either start again or just give up.
  • torben-friis 5mo ago
    So many people get buried in tech debt due to constantly choosing "let's be productive now and iterate later". Then later never comes. This was already proof that, when a company is given the chance of getting a reward now and putting in the effort later, they will take the reward and postpone the effort indefinitely. AI now offers "generate now, review later". Fill in the pattern. We're cooked and it's not due to AI, it's due to the fundamentals of an economy and society that only sees the short term.
• xienze 5mo ago
  > In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day. I think a lot of the time it's just pure laziness. AI gives people a magical "do all the work for me" button and it can bring out the worst in them.
  • canyp 5mo ago
    I constantly battle this dichotomy where I care about the work I do but I also cannot possibly care about the corporate model, given 0 ownership of flawed processes across the org and the looming layoff that'll happen any day now. Some people are given the button and really do not care.
• ChrisMarshallNY 5mo ago
  > AI output is not getting vetted by knowledgeable people You mean the people they fired and demoralized? One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts." Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore.
  • ilamont 5mo ago
    > Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore. Serious orgs are going to have to figure out the human layer. It will be needed, no matter how 'hallucination-free' the AI tooling gets. AI will still have some spectacularly bad fuck ups or even worse time bombs that get embedded in a system and don't become apparent until months or years later. A lot of this will be dumped on existing staff with predictable results as they don't have the bandwidth to do it right. I can envision "output compliance" or "AI QA" becoming dedicated positions at many orgs. It's clearly needed.
    • anal_reactor 5mo ago
      > It's clearly needed. Once the hallucination rate drops below error rate of human workers, it won't be needed anymore.
      • cassianoleal 5mo ago
        Once the hallucination rate drops, the remaining LLM failures will become increasingly harder to spot.
    • hansmayer 5mo ago
      [dead]
    • asdff 5mo ago
      Let's be honest, how many orgs are really serious? Playing the game of the day for shareholder appeasement is taken far more seriously than whatever the domain experts might think.
• fabian2k 5mo ago
  If the main job is putting out a report, starting with AI is wrong in any case. What's the value of an AI-generated report, even if experts fix the biggest issues with it? Maybe this kind of report didn't have all that much value before, I don't know. But starting with AI just makes sure it's generic drivel.
• deleted 5mo ago
  [deleted]
• mminer237 5mo ago
  As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form. With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.
  • Izikiel43 5mo ago
    How do you use it, as in, hey, write a doc about this, or do you iterate more like a conversation? I do the second approach for coding with smallish steps and the output is fine
    • mminer237 5mo ago
      To be honest, I don't use it at all professionally except I guess transcription or search engine summaries to point me to actual documents. I mostly come across it when a client makes a contract or legal memo on his own and sends it to me to review. If I'm drafting a lease or a trust or whatever, I already have forms that cover everything they need, have been used thousands of times, and can be easily tweaked as needed cuz I half have them memorized at this point. I can't imagine using them for a court filing. Those are either so short that they're trivial or they require painstaking research, precision, and often citations for every single sentence.
  • SV_BubbleTime 5mo ago
    I’m against “vibe” anything important, but the fundamental flaw with this reasoning is that unknown unknowns exist. I can’t cite “from scratch” for something outside of my knowledge but I side LLM training or assisted search.
  • csomar 5mo ago
    It's not really any different in programming. Like if you have a well structured code and want to do a clear refactoring across it and you know what to expect, it can speed things up. But if it's generating any significant (and relatively complex) new code, you have to go through the whole thing manually again and then you find out you have to fix way to many things and get bogged down in different paths the AI didn't do correctly. Of course, it's pretty much impossible to hear a dissenting point of view today and everyone is going crazy on these drugs. I might be hilariously wrong but I think this is the best time to start a software company.
• Ekaros 5mo ago
  Also wondering on this whole review process with someone who wrote it with AI. Even if you comment and noted all issues. Do they have skills or willingness to correctly correct it all? And how many times would you need to keep the loop going for error free outcome? Is there even enough calendar time for that?
• kloop 5mo ago
  > The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people The problem is that output sometimes take longer to verify than to create in the first place. That turns AI into a deeply negative ROI system for many applications.
• DrewADesign 5mo ago
  > The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. Yeah probably not for the same reason I left VFX rather than have a lifetime of completely disregarding my own generative creativity and cleaning up LLM-generated bullshit. Fuck that. Double-fuck creating ‘content’ to train the models. In code, LLMs automate away a lot of the drudgery. I wasn’t sad to avoid spending a couple hours looking up the usage patterns and idioms for some ported library, or do some rote task that didn’t make the project significantly better. In most other jobs, they automate away the only fun part and leave humans with all of the drudgery. The tech industry has always been arrogant to some extent, but assuming the world of talented professional knowledge workers and creatives would be content to professionally proofread, apply lipstick to pigs, and polish turds is a whole new level of out-of-touch. I’d rather live out of my car and dig through the garbage for bottles with deposits.
  • bluefirebrand 5mo ago
    > In most other jobs, they automate away the only fun part and leave humans with all of the drudgery. I don't know about "most other jobs", I feel like this applies strongly to software too. AI is being used to automate the fun stuff You may view writing code as drudgery, I like doing it
    • DrewADesign 5mo ago
      Some people enjoy bookkeeping, too. I loved solving problems with code but I don’t think it’s controversial to say most people enjoy the novel problems much much more than pulling apart an obtusely designed, non-idiomatic API. The AI is better at the details than the novel problem, so whoever is driving still needs to sort that out. In art, writing, and so-on, it only does the interesting part, but it does it poorly. So for professional use cases, it involves a lot of tedious cleanup of other people’s art that a machine (very economically) amalgamated into a pile of shit, and you’ve no opportunity to do anything interesting. At all.
• wrs 5mo ago
  But wait, if knowledgeable people have to vet the output, the process will not be 10X faster and you will not be able to fire the knowledgeable people. Therefore, your objection makes no sense. QED.
• fzeindl 5mo ago
  > In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day. This is an interesting topic. We treat vetting output the same as doing the work ourselves, but that is not the case. Doing the work is not the same as reviewing work done by others. I have heard reports of software engineering companies that have gone full agentic. Their seniors only review stuff written by LLMs and it burns them out, because they have to switch context constantly. I find this interesting because part of being a senior developer is that you are experienced enough that you won‘t make grave mistakes anymore. This is the case in many professions: you are relied upon to not make grave mistakes. But those same people are now swamped with stuff that they are not able to review, so they will let a grave mistake slip through at some point. So they really can‘t trust themselves anymore?
• watwut 5mo ago
  It is harder to check everything then to create a thing without lying in the first place.
• root-parent 5mo ago
  >>The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people I am particularly interested in Education and Human Knowledge Management. I have seen the rate of IT training going to zero. Think about specialized training, where if you make a mistake, the consequence of your errors, are talked about on the tv news of the evening. The whole idea everybody is just planning to save their butt, using these strings coming out of these numeric matrices, while suspending judgement, just shudders me in horror. A bit like those South Asia Airline companies, that were forbidding their pilots from landing airplanes with manual piloting, leading to an increase loss of skills causing some well known disasters... If well paid consultants cant even bother to check their links...
• roncesvalles 5mo ago
  The problem is that a lot of employees across the corporate world are engaged in malicious compliance with regard to AI usage.
• sandworm101 5mo ago
  >> Amazon now having senior engineers vet generative AI output Software and news are apples and oranges. The software engineer looks at AI code to spots errors and make/suggest corrections. But the software still exists. A doctor vetting a news article ("Owning a Boat Causes Cancer") isn't there to tweak language. The doctor's job is to stop the entire article. It would be like the amazon engineer deleting all the source code and telling the dev team to abandon the project. That will never be a popular task.
• gloryjulio 5mo ago
  It's similar to tokenmaxxing in various companies. Who cares about reviewing the design and the code, just code generation all the way. If it runs it's good
• erentz 5mo ago
  It looks like when you sum up: the cost to generate information using an LLM + the cost to actually verify the information, the result on average is the same as not using the LLM. That does not mean some times it isn’t faster and cheaper. It just means other times it’s slower and more costly. This along with different people’s tolerance for accuracy explains why we see such diverging experiences with it. So in order to make it pan out the forces at play are trying to make everyone believe we now have to accept wrong, even dangerous results.
• sidewndr46 5mo ago
  If they were going to have employees do the in house vetting, they could just have them write the report.