4 ms·

Not for RAG and Automation applications But for Fixing Hallucinated code that written by Non Software Engineers.

• joesimultaneous 2y ago
  [flagged]
• drewcoo 2y ago
  > But for Fixing Hallucinated code that written by Non Software Engineers. And for testing to find those inconsistencies.
  • rpigab 2y ago
    And for reviewing failed tests that were hallucinated by LLM testing software. And for fixing the consequences of LMM testing giving false negatives.
  • candiddevmike 2y ago
    This sounds like hell, tbh.
    • naasking 2y ago
      Sounds like basically the same hell as reviewing code of junior SWEs.
• kcplate 2y ago
  That is completely contingent on if present the day SWEs stop working on trying to eliminate the hallucinations.
  • lexicality 2y ago
    I'm curious as to what engineering solutions you think are available to solve this problem? All the solutions I've heard up until now is "we just need more data" which isn't particularly sustainable or something a SWE would be in charge of doing
    • kcplate 2y ago
      It’s not my expertise. However, if it’s a barrier to adoption it’s a problem to solve and will be. AI is exponentially improving and there is no reason to believe that this won’t continue.
      • lelanthran 2y ago
        > AI is exponentially improving Not for the last two years at least. Over the last year it's been obvious that there are diminishing returns already... We increase the computational power by a factor of ten, and see a factor of 1.5 improvement.
      • randomdata 2y ago
        > AI is exponentially improving The theoretical research may have improved exponentially. Implementations available for use seem to be on the decline – at best, stagnant. I find less and less utility as time goes on. Things LLMs shined at a couple of years ago now produce garbage. And for creative work, the output is much too formulaic, which was all well and good initially while still novel, but one has to keep pushing new boundaries and the current crop of tools really seems to struggle with that.
      • PleasureBot 2y ago
        That's not how technology typically advances. Technology advances in waves. There's an initial breakthrough that leads to rapid progress in certain areas. But after some time there are diminishing returns as the the applications of that breakthrough are fully explored, and progress slows or stops until the next breakthrough. Predicting that the trajectory of progress for the next 20 years will match the progress that has been made in the first 2 years after the LLM breakthrough seems naive at best. I would also disagree with the idea that LLM tech is "exponentially improving" now. There was the initial release of ChatGPT which was an enormous step forward, but since then its been small iterations on the same fundamental technology. We are already seeing significantly diminishing returns in our ability to improve these models. Most available training data has already has already been fed in, 10xing the compute results in only marginal improvements in performance, etc.
  • roenxi 2y ago
    Or just getting it below the base rate of human software programmers. Humans hallucinate all the time; we've just built the systems around ourselves to tolerate a bit of silliness.
  • neuronic 2y ago
    The fundamental technology of LLMs *is* statistical hallucination. What's the probability of a sequence of tokens given a context? That's what LLMs answer. I am not one to talk in absolutes but the basic circumstances of current LLMs have hallucination at their core and can likely not be guaranteed fact based. It's just an expression of likelihood where you rely on your training data to contain truths. This is why OpenAI gives more weight to Wikipedia than Reddit comments - the model contains no reasoning mechanism. It ought to be treated more as a fallible human rather than a superintelligent lookup resource based on fact checked data.
  • catlifeonmars 2y ago
    What is the definition of a hallucination in the context of LLM?
• nprateem 2y ago
  It's frustrating just how shit AI is at coding. The new gpt model seems even worse, splurging out a load of guff that makes it even more difficult to see the bullshit it's produced. AI a threat to humanity, my ass.
  • williamcotton 2y ago
    Weird, I find that I get very good results with GPT-4 and GPT-4o. It helps to be in a position where you can communicate very explicit instructions on what and how to build. Use it like you’re planning what to code each step but just letting this mechanism do the typing. General writing and technical communication skills are a must so I can understand why some developers struggle with LLMs.
    • randomdata 2y ago
      > General writing and technical communication skills are a must Perhaps that is what he is trying to get at? I am sure my communication skills are poor, but I found utility in using these tools a year or so ago. Whatever gibberish I was able to give it often produced a good result. These days I can't seem to get anything usable out of them. It does seem, like the parent suggests, that they have gotten worse. It may very well be that said tools are no worse, if even better, where communication ability is stronger. However, if we have chosen to optimize these systems for those who are great at communicating, at the cost of those who are not, that doesn't help with the topic at hand. The non-software engineers – implying Average Joe – are almost certainly not going to be great technical writers.
• empath75 2y ago
  Human engineers of course never make mistakes, which is why we don't have PRs or unit or integration tests and canary deploys or anything that would catch mistakes before going to production because that would be a waste of time.
• jayemar 2y ago
  Does RAG mean Retrieval-Augmented Generation? I've never heard of this term.