2 ms·

For now, the hallucination problem is objectively only getting worse with reasoning models, and there is no solution in sight. Technical skills will matter for

For now, the hallucination problem is objectively only getting worse with reasoning models, and there is no solution in sight. Technical skills will matter for the simple reason that someone will always need to supervise the output of the AI. A non-technical person can't hope to do so too well. Without this supervision, the AI's work product will have skeletons that will cause unexpected issues due to outliers. Reality is full of outliers - it's what pays half the salary. Imagine asking an AI to design an airplane. The design passes all test flights and also software tests. Would you want to fly in it without an expert human having reviewed the ins and outs of the design? How about a CT scan machine? Would you want to risk the 10x radiation due to a hypothetical implementation error that strikes in 1 out of 10K cases?

ninetyninenine 2y ago
> For now, the hallucination problem is objectively only getting worse with reasoning models, and there is no solution in sight. You pulled this statement out of your ass. Objectively? We have baseline quantitative tests that say the opposite? LLMS are doing better on tests and it’s been improving. Where did your “objective” statement come from? Anecdotes? Quotations? > Technical skills will matter for the simple reason that someone will always need to supervise the output of the AI. Humans will never be so stupid as to lose all technical skill. In the beginning there needs to be mild technical skill at most so the human can somewhat understand what the AI is doing. But as trust grows the human will understand less and less of it. The human is like a desperate micro manager attempting to cling to understanding but that inevitably eventually erodes completely.
- OutOfHere 2y ago
  For commentary on the hallucination problem, please see https://news.ycombinator.com/item?id=43942800 https://news.ycombinator.com/item?id=43942800 and the associated news link. I have observed it myself in practice with o3 compared to o1.

ninetyninenine 2y ago

> For now, the hallucination problem is objectively only getting worse with reasoning models, and there is no solution in sight. You pulled this statement out of your ass. Objectively? We have baseline quantitative tests that say the opposite? LLMS are doing better on tests and it’s been improving. Where did your “objective” statement come from? Anecdotes? Quotations? > Technical skills will matter for the simple reason that someone will always need to supervise the output of the AI. Humans will never be so stupid as to lose all technical skill. In the beginning there needs to be mild technical skill at most so the human can somewhat understand what the AI is doing. But as trust grows the human will understand less and less of it. The human is like a desperate micro manager attempting to cling to understanding but that inevitably eventually erodes completely.

OutOfHere 2y ago
For commentary on the hallucination problem, please see https://news.ycombinator.com/item?id=43942800 https://news.ycombinator.com/item?id=43942800 and the associated news link. I have observed it myself in practice with o3 compared to o1.