3 ms·

I'm slightly confused as to why you'd use a LLM to sort structured data in the first place?

by codyb 1y ago

jngiam1 1y ago
The goal is to do more complex data processing, like build dashboards, agentically figure out which tickets are stalled, do a quarterly review of things done, etc. Sorting is a tiny task in the bigger ones, but hopefully more easily exemplifies the problem.
- kikimora 1y ago
  I don’t understand how this can work. Given probabilistic nature of LLMs the more steps you have more chances something goes off. What is good in the dashboard if you cannot be sure it was not partially hallucinated?
  - staunton 1y ago
    > What is good in the dashboard if you cannot be sure it was not partially hallucinated? A lot of the time the dashboard contents doesn't actually matter anyway, just needs to look pretty... On a serious note, the systems being built now will eventually be "correct enough most of the time" and that will be good enough (read: cheaper than doing it any other way).
    - codyb 1y ago
      We're all just going to be glorified debuggers trawling through reams of generated code we've never seen before to root out the hallucinations.
    - kikimora 1y ago
      >On a serious note, the systems being built now will eventually be "correct enough most of the time" I don’t believe this would work. File a “good enough” tax return one year and enjoy hefty fine 5 years later. Or constantly deal with customers not understanding why one amount is in the dashboard and another is in their warehouse. Probability of error increase rapidly when you start layer one probabilistic component onto another. Four 99% reliable components sequenced one after another have error rate of 4%.
  - orbital-decay 1y ago
    Probabilistic nature means nothing on its own. LLM that can solve your deterministic task will easily assign 100% to the correct answer (or 99%, the noise floor can be truncated with a sampler). If it doesn't do that and your reply is unstable, it cannot solve it confidently. Which happens to all LLMs on a sufficiently complex task, but it's not related to their probabilistic nature. Of course that still doesn't mean that you should do that. If you want to maximize model's performance, offload as much distracting stuff as possible to the code.
- risyachka 1y ago
  Everything you described is already solved by Metabase and few other tools. It takes a few hours to make daily reports there and the dashboard of your dreams. And its not like it changes every day. KPis etc stay the same for months. And then you can easily update it in a hour. So what exactly does llm solve here?

I'm slightly confused as to why you'd use a LLM to sort structured data in the first place?

by codyb 1y ago

I'm slightly confused as to why you'd use a LLM to sort structured data in the first place?

The goal is to do more complex data processing, like build dashboards, agentically figure out which tickets are stalled, do a quarterly review of things done, etc. Sorting is a tiny task in the bigger ones, but hopefully more easily exemplifies the problem.

kikimora 1y ago
I don’t understand how this can work. Given probabilistic nature of LLMs the more steps you have more chances something goes off. What is good in the dashboard if you cannot be sure it was not partially hallucinated?
- staunton 1y ago
  > What is good in the dashboard if you cannot be sure it was not partially hallucinated? A lot of the time the dashboard contents doesn't actually matter anyway, just needs to look pretty... On a serious note, the systems being built now will eventually be "correct enough most of the time" and that will be good enough (read: cheaper than doing it any other way).
  - codyb 1y ago
    We're all just going to be glorified debuggers trawling through reams of generated code we've never seen before to root out the hallucinations.
  - kikimora 1y ago
    >On a serious note, the systems being built now will eventually be "correct enough most of the time" I don’t believe this would work. File a “good enough” tax return one year and enjoy hefty fine 5 years later. Or constantly deal with customers not understanding why one amount is in the dashboard and another is in their warehouse. Probability of error increase rapidly when you start layer one probabilistic component onto another. Four 99% reliable components sequenced one after another have error rate of 4%.
- orbital-decay 1y ago
  Probabilistic nature means nothing on its own. LLM that can solve your deterministic task will easily assign 100% to the correct answer (or 99%, the noise floor can be truncated with a sampler). If it doesn't do that and your reply is unstable, it cannot solve it confidently. Which happens to all LLMs on a sufficiently complex task, but it's not related to their probabilistic nature. Of course that still doesn't mean that you should do that. If you want to maximize model's performance, offload as much distracting stuff as possible to the code.
risyachka 1y ago
Everything you described is already solved by Metabase and few other tools. It takes a few hours to make daily reports there and the dashboard of your dreams. And its not like it changes every day. KPis etc stay the same for months. And then you can easily update it in a hour. So what exactly does llm solve here?