3 ms·

> You can just ask Claude why it did something But will Claude give you an authentic rationale and a traceable, verifiable "line of reasoning" for those things

> You can just ask Claude why it did something But will Claude give you an authentic rationale and a traceable, verifiable "line of reasoning" for those things? Or will it just construct the next plausible Markov chain built on whatever Reddit thread it ingested at random? You can ask Claude or any LLM for citations, and it will RAG them out ex post facto. Those actually aren't citations, they're just web searches for related articles, and they don't necessarily support the assertions that you're asking to cite. I am sure that Claude and the others can produce intermediate logs of their inference and "reasoning" process while they are processing stuff, but can they really go back within the context window and construct an authentic apologia for a specific thing when you ask for it?

fluidcruft 1mo ago
You certainly can ask it to produce a verifiable report that tells you how to verify its claims. Yes, you do need to verify the results and confirm you agree with it. I do this sort of thing a lot to discover misunderstandings Claude has about things and iterating on them. There's a difference between reading everything yourself vs having a discussion where one of you (Claude) will happily dig through everything and create detailed memos for review. If you want line numbers as evidence, just ask then verify, etc.
- ButlerianJihad 1mo ago
  Really? Verifiable, really? If you ask it 3 times to generate 3 verifiable reports that confirm its claims, will it answer with the same process and same answers each time? See, when you ask a human to justify a result or a decision, they can often do this very meticulously. If a judge writes a decision from the bench, or a firefighter describes how his battalion knocked down an apartment fire, or a systems admin describes how he configured a NAS, they will all be relying on their training, and precedent, and specifications, and things like that, and they can give you reproducible results and solid justifications for the way they did things. When mistakes are made, and money or life is lost, they can be accountable and you can modify that process to set a precedent for the future. But Claude? How in the world will it produce the same results twice? It is non-deterministic. That is the fundamental issue of LLMs and genAI today. They are all non-deterministic, and SWE treat them as if they are somehow reliable, or produce reproducible results, or that they can follow a procedure or a specification, outlined in their prompts and context, and produce results. No, they only produce results by accident and happenstance, and they only justify them ex post facto by making things up. There is no humanity or deterministic activity in an LLM. You'll never verify "why" they chose that string of tokens, because they could've easily chosen a very different stream of tokens. In fact, now with watermarking, the most deterministic thing will be hitting that watermark standard at all costs!
  - fluidcruft 1mo ago
    Are you under the impression that if you ask two programmers to solve the same problem, you will receive the same code or design from both?
moring 1mo ago
Now replace "Claude" by "human" and see how your comment goes. Specifically, humans are known to decide subconsciously, then invent some "reasoning" out of thin air to justify it. This matches my experience with decision-making in software projects.
- fortzi 1mo ago
  People, sometimes weirdly called by their animal name, humans, do have memories of their thought process. We can reach back to those memories to explain why we made a decision. Sometimes we don’t remember and speculate instead. Most of the time we can tell the difference

fluidcruft 1mo ago

You certainly can ask it to produce a verifiable report that tells you how to verify its claims. Yes, you do need to verify the results and confirm you agree with it. I do this sort of thing a lot to discover misunderstandings Claude has about things and iterating on them. There's a difference between reading everything yourself vs having a discussion where one of you (Claude) will happily dig through everything and create detailed memos for review. If you want line numbers as evidence, just ask then verify, etc.

ButlerianJihad 1mo ago
Really? Verifiable, really? If you ask it 3 times to generate 3 verifiable reports that confirm its claims, will it answer with the same process and same answers each time? See, when you ask a human to justify a result or a decision, they can often do this very meticulously. If a judge writes a decision from the bench, or a firefighter describes how his battalion knocked down an apartment fire, or a systems admin describes how he configured a NAS, they will all be relying on their training, and precedent, and specifications, and things like that, and they can give you reproducible results and solid justifications for the way they did things. When mistakes are made, and money or life is lost, they can be accountable and you can modify that process to set a precedent for the future. But Claude? How in the world will it produce the same results twice? It is non-deterministic. That is the fundamental issue of LLMs and genAI today. They are all non-deterministic, and SWE treat them as if they are somehow reliable, or produce reproducible results, or that they can follow a procedure or a specification, outlined in their prompts and context, and produce results. No, they only produce results by accident and happenstance, and they only justify them ex post facto by making things up. There is no humanity or deterministic activity in an LLM. You'll never verify "why" they chose that string of tokens, because they could've easily chosen a very different stream of tokens. In fact, now with watermarking, the most deterministic thing will be hitting that watermark standard at all costs!
- fluidcruft 1mo ago
  Are you under the impression that if you ask two programmers to solve the same problem, you will receive the same code or design from both?

moring 1mo ago

Now replace "Claude" by "human" and see how your comment goes. Specifically, humans are known to decide subconsciously, then invent some "reasoning" out of thin air to justify it. This matches my experience with decision-making in software projects.

fortzi 1mo ago
People, sometimes weirdly called by their animal name, humans, do have memories of their thought process. We can reach back to those memories to explain why we made a decision. Sometimes we don’t remember and speculate instead. Most of the time we can tell the difference