3 ms·
> The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and f
by bigstrat2003 4mo ago
> The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work.
That's wrong, and that's exactly why the loss of knowledge is such a problem. LLMs do not, and cannot, actually know a single thing. They are a statistical model, not knowledge. When they give out wrong information (and they always will, by their very nature), you need someone with actual knowledge to be able to recognize the BS and correct it. But we are losing the knowledge, and unless things change we will be no better off than the people in dystopian sci-fi stories who pray to the machine god because nobody knows how it actually works.
- arm32 4mo agoI'm sure this comment will get buried, but I wholeheartedly agree with your take.
- Ekaros 4mo agoI can already see future where there is group of people goading AI to right direction by repeatedly changing what they ask slightly. And then memorise the times when you got right incantation for seemingly right outcome. Without actually understanding or being able to reason about process in the middle. Possibly a seemingly inconsequential misspoken word or typo can lead to better or worse outcome. Or maybe just not saying "Please" will sometimes produce wrong output. and other times you must not use it... Sounds like absolutely horrifying dystopia.
- thewebguyd 4mo agoImagine if we had LLMs back then to write xorg.conf for you and it hallucinated the modeline and you blew up your monitor.
- smsm42 4mo agoYes. And since LLMs can not improve knowledge - I mean, they can generate new arrangements of information, but they have no idea whether any of it real or making sense, unless humans - explicitly or through training - tell them, the more we rely on LLM knowledge the less the quality of it would be. Right now the LLMs are mainly in auxiliary role, so most of the knowledge erosion they generate is laughed at and relatively quickly corrected. But would this hold once the role of generative AIs increases? We already essentially entered the chaos period with news content - there's so much noise that it's basically impossible to know if any news message you read is true or manipulated somehow. This is going to start happening to more fundamental knowledge too, either on purpose or just by the force of the probabilistic nature of generative AI.
- Ekaros 4mo agoI was just thinking. What is the future when LLM is used for both code running itself and then also designing the hardware it runs on? Will it be sustainable? Or will there be some sort of error cascade that destroys the whole thing? Maybe AGI is impossible with current model as it simply can not reliably improve itself... Enough errors in any part of loop will stop the progression.
- simianwords 4mo ago> Yes. And since LLMs can not improve knowledge 1. GPT proved Erdős Unit Distance Conjecture entirely on its own 2. GPT-5.4 Pro Solves Erdős Problem #1196 (April 2026) In fact there's a whole benchmark that's measuring this: https://epoch.ai/frontiermath https://epoch.ai/frontiermath
- baal80spam 4mo agoI think what you describe is called the Model Collapse.
- simianwords 4mo agoThis comment might have gone hard in 2023. Now it seems out of place. LLM's do hold knowledge empirically. They sometimes give out wrong information, like humans. And you need someone with better knowledge to correct it. > But we are losing the knowledge No we aren't and this is spreading FUD. Things have always been like this. Its called specialisation and this is how society progresses. I don't know how the supply chain worked to get the food to my table. That's why its so cheap!
- Beigale 4mo ago[flagged]