3 ms·

As someone who just spent 2 hours in my company's Confluence site, trying to track down the answer to a single question that could have been resolved in seconds

As someone who just spent 2 hours in my company's Confluence site, trying to track down the answer to a single question that could have been resolved in seconds by an LLM trained on an internal corporate corpus -- LLMs are very much not a fad.

EGreg 3y ago
How do you know the answer is right? Because it linked you to the source? Like a vector database would? Google offered to index sites since 1996.
- dlivingston 3y ago
  We have internal search. Finding things isn't the problem. It's contextualizing massive amounts of text and making it queryable with natural language. The question I was trying to solve was -- "what is feature XYZ? How does it work in hardware & software? How is it exposed in our ABC software, and where do the hooks exist to interface with XYZ?" The answers exist across maybe 30 different Confluence pages, plus source code, plus source code documentation, plus some PDFs. If all of that was indexed by an LLM, it would have been trivial to get the answer I spent hours manually assembling.
  - EGreg 3y ago
    So back to the question - how do you know the LLM didnt hallucinate an answer? What do you think “indexed by an LLM” is? Perhaps Anthropic with its 100K window can actually do it. But most LLM have such a small comtext window that it’s just Pinecone vector database indexing something and stuffing it in the prompt at prompt time. Come on.
  - artfulmink 3y ago
    The question to which you are replying still stands. How can you guarantee that the responses generated by the LLM are factually accurate? What if it refers to interfaces that don’t exist? One could argue that in your particular use case some inaccuracies can be tolerated, but in many use cases factual inaccuracies cannot be tolerated.
JohnFen 3y ago
LLMs are useful for particular types of things. LLMs as the solution for every, or most, problems is a fad.

EGreg 3y ago

How do you know the answer is right? Because it linked you to the source? Like a vector database would? Google offered to index sites since 1996.

dlivingston 3y ago
We have internal search. Finding things isn't the problem. It's contextualizing massive amounts of text and making it queryable with natural language. The question I was trying to solve was -- "what is feature XYZ? How does it work in hardware & software? How is it exposed in our ABC software, and where do the hooks exist to interface with XYZ?" The answers exist across maybe 30 different Confluence pages, plus source code, plus source code documentation, plus some PDFs. If all of that was indexed by an LLM, it would have been trivial to get the answer I spent hours manually assembling.
- EGreg 3y ago
  So back to the question - how do you know the LLM didnt hallucinate an answer? What do you think “indexed by an LLM” is? Perhaps Anthropic with its 100K window can actually do it. But most LLM have such a small comtext window that it’s just Pinecone vector database indexing something and stuffing it in the prompt at prompt time. Come on.
- artfulmink 3y ago
  The question to which you are replying still stands. How can you guarantee that the responses generated by the LLM are factually accurate? What if it refers to interfaces that don’t exist? One could argue that in your particular use case some inaccuracies can be tolerated, but in many use cases factual inaccuracies cannot be tolerated.

JohnFen 3y ago

LLMs are useful for particular types of things. LLMs as the solution for every, or most, problems is a fad.