4 ms·

Not holding my breath for that hallucinated cure for cancer then.

ben_w 2y ago
LLMs aren't the only kind of AI, just one of the two current shiny kinds. If a "cure for cancer" (cancer is not just one disease so, unfortunately, that's not even as coherent a request as we'd all like it to be) is what you're hoping for, look instead at the stuff like AlphaFold etc.: https://en.wikipedia.org/wiki/AlphaFold https://en.wikipedia.org/wiki/AlphaFold I don't know how to tell where real science ends and PR bluster begins in such models, though I can say that the closest I've heard to a word against it is "sure, but we've got other things besides protein folding to solve", which is a good sign. (I assume AlphaFold is also a mysterious black box, and that tools such as the one under discussion may help us demystify it too).
- surfingdino 2y ago
  [flagged]
  - mvkel 2y ago
    > guys selling AI Who?
  - wg0 2y ago
    [flagged]
  - bigyikes 2y ago
    Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your take. Do you not recognize that there is utility to current models? What makes it all bullshit?
    - vitus 2y ago
      > 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to identify these failures in scenarios where I'm not a domain expert. But maybe some subset of these are solved problems, or at least problems that are actively being worked on for the next generation of models. Yes, there are many other kinds of AI. Stockfish is better at chess than any human. But when you start talking about emergent behavior from machine learning, the failure modes are much harder to reason about.
      - qingcharles 2y ago
        It's made me vastly more productive. It's a force multiplier in the areas I work, coding and content creation.
      - __MatrixMan__ 2y ago
        > I'm curious what those things are I find them pretty good at reasoning about tree structures that have a depth which I myself find difficult to navigate. For instance, I've been working with libcst (a syntax tree library) and I can say: 1. observe this refactor rule which I like 2. here's the starting code 3. here's the desired code 4. write me a refactor rule, in the style of 1, which transforms 2 into 3 It sometimes takes a few iterations where I show it a diff which highlights how its rule fails to construct 3 given 2, but it usually gets me the transformation I need much faster than I'd have done so by hand. And once I'm done I have a rule which I can apply without the LLM in the loop and which is much more robust than something like a patch file (which often fail to apply for irrelevant reasons like whitespace or comments that have changed since I wrote the rule). The key is to find cases, like this one, where you can sort of encircle the problem with context from multiple sides, one of which works as a pass/fail indicator. Hallucinations happen, but you use that indicator to ensure that they get retried with the failure text as correcting context. It helps to design your code so that those context pieces stay small and reasonably self-describing without taking a foray deep into the dependencies, but then that's just a good idea anyway.

LLMs aren't the only kind of AI, just one of the two current shiny kinds. If a "cure for cancer" (cancer is not just one disease so, unfortunately, that's not even as coherent a request as we'd all like it to be) is what you're hoping for, look instead at the stuff like AlphaFold etc.: https://en.wikipedia.org/wiki/AlphaFold https://en.wikipedia.org/wiki/AlphaFold I don't know how to tell where real science ends and PR bluster begins in such models, though I can say that the closest I've heard to a word against it is "sure, but we've got other things besides protein folding to solve", which is a good sign. (I assume AlphaFold is also a mysterious black box, and that tools such as the one under discussion may help us demystify it too).

surfingdino 2y ago
[flagged]
- mvkel 2y ago
  > guys selling AI Who?
- wg0 2y ago
  [flagged]
- bigyikes 2y ago
  Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your take. Do you not recognize that there is utility to current models? What makes it all bullshit?
  - vitus 2y ago
    > 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to identify these failures in scenarios where I'm not a domain expert. But maybe some subset of these are solved problems, or at least problems that are actively being worked on for the next generation of models. Yes, there are many other kinds of AI. Stockfish is better at chess than any human. But when you start talking about emergent behavior from machine learning, the failure modes are much harder to reason about.
    - qingcharles 2y ago
      It's made me vastly more productive. It's a force multiplier in the areas I work, coding and content creation.
    - __MatrixMan__ 2y ago
      > I'm curious what those things are I find them pretty good at reasoning about tree structures that have a depth which I myself find difficult to navigate. For instance, I've been working with libcst (a syntax tree library) and I can say: 1. observe this refactor rule which I like 2. here's the starting code 3. here's the desired code 4. write me a refactor rule, in the style of 1, which transforms 2 into 3 It sometimes takes a few iterations where I show it a diff which highlights how its rule fails to construct 3 given 2, but it usually gets me the transformation I need much faster than I'd have done so by hand. And once I'm done I have a rule which I can apply without the LLM in the loop and which is much more robust than something like a patch file (which often fail to apply for irrelevant reasons like whitespace or comments that have changed since I wrote the rule). The key is to find cases, like this one, where you can sort of encircle the problem with context from multiple sides, one of which works as a pass/fail indicator. Hallucinations happen, but you use that indicator to ensure that they get retried with the failure text as correcting context. It helps to design your code so that those context pieces stay small and reasonably self-describing without taking a foray deep into the dependencies, but then that's just a good idea anyway.