3 ms·
I'm sure future LLMs will be able to comprehend more. So the debt, similarly to real world debt, is fine, as long as the line goes up.
by scotty79 1y ago
I'm sure future LLMs will be able to comprehend more. So the debt, similarly to real world debt, is fine, as long as the line goes up.
- actionfromafar 1y agoBut won't that future LLM be able to spew out even more? I mean, I regularly produce stuff I can't comprehend at a later date, why won't the same happen to an LLM?
- scotty79 1y agoBecause you comprehension grows in time only slightly or not at all and at some point will start to decline. That won't be true for LLMs for some time hopefully. And even then at some point they learn to make stuff in a more "divide and conquer" style so they don't need to understand whole big ball of spaghetti all at once.
- actionfromafar 1y agoI mean, even if we don't understand in our brains, what's stopping people from buliding ever more complex systems with LLMs, until the LLMs themselves can't keep up?
- tgv 1y agoIdk. The company I work for had a new website designed, and it was built with an (unknown) LLM, through repeated prompting, I believe (so basically a large document that starts the initial description and then adds fixes). It's deployed on an unknown stack, at a random server somewhere. Us techies simply got a few remarks from the LLM about changing the DNS (which were not up to any standard for such requests). The moment some marketeer wants some change on that website, the whole thing may come undone. It's like outsourcing to the lowest bidder. But the CEO is happy, because AI. I'm also not sure about your basic premise that understanding will improve. That depends on the size of network's internal representation(s), which will start overfitting at some point.