6 ms·

The "core reasoning" part of AI may be increasingly important to improve, and its "database of factual knowledge" aspects may be less and less important, maybe

The "core reasoning" part of AI may be increasingly important to improve, and its "database of factual knowledge" aspects may be less and less important, maybe increasingly a hindrance. So more focused and specialized training may take over toward increasing reasoning precision, and not this never-ending stream of new data. So maybe we'll get better reasoning and therefore better generated data/content in the wild, without this negative feedback loop everyone is worried about.

gerdesj 1y ago
You seem to be arguing that bollocks couched in flowery language is a stretch goal of AI. Are you sure?
- _heimdall 1y ago
  That really depends on expectations. If AI is meant to sound nearly identical to a human, you don't need more training data. If its meant to act as a natural language encyclopedia, we'll never get there with LLMs which amount to natural language processing on top of a massively compressed dataset.
  - gerdesj 1y ago
    I've read the Google page "Getting AI to write good SQL: Text-to-SQL techniques explained" and it looks like all the usual bollocks. ... and sigh ... it look like I'm twiddling with a noddy!
- mondrian 1y ago
  Trying to make AIs more factually-accurate with more training is probably hopeless. Current events and encyclopedic knowledge will be provided by tools. The LLM's core job is to choose the right tools for the job and synthesize their outputs.
meroes 1y ago
What core reasoning?
- SpicyLemonZest 1y ago
  For example, I just asked o3 "how many Is are in antidisestablishmentarianism", and this was its train-of-thought output: ------ The user wants to know how many “I”s are in the word “antidisestablishmentarianism.” They probably mean the letter "I" in lowercase, though the question was a little broader with "Is." I’ll treat it case-insensitively and count the total occurrences of the letter "i" (both uppercase and lowercase). I can count them manually now, or use a tool for faster analysis. Analyzed word = "antidisestablishmentarianism" word_lower = word.lower() word_lower.count('i') 5 The user asked how many "I"s are in "antidisestablishmentarianism." I’m taking it to mean whether it’s counting "i" or "I", and I’ll consider them as the same for simplicity. So, I’ll count the 5 occurrences of "i" in the word. I manually broke it down and verified there are indeed five "i"s, so the final count should be 5. Pretty straightforward, now that it's all clear! Done There are 5 “i”s in “antidisestablishmentarianism.” ------ You can come up with complicated definitions of "core reasoning" where this doesn't count, and I understand there's great demand for such definitions, but this clearly satisfies common-sense understanding of the term.
  - selfhoster 1y ago
    Then I guess Ubuntu has had reasoning for several decades: sudp Command 'sudp' not found, did you mean: command 'sudo' from deb sudo (1.9.9-1ubuntu2.4) command 'sudo' from deb sudo-ldap (1.9.9-1ubuntu2.4) command 'sup' from deb sup (20100519-3) command 'sfdp' from deb graphviz (2.42.2-6) Try: sudo apt install <deb name>
  - meroes 1y ago
    I might just be on the opposite side of the aisle, but to me chain-of-thought is better understood as simply more context. Of course there is ambiguity though, more context would be hard to distinguish from core-reasoning and vice versa. I think LLMs/AI mean we can substitute reasoning with vast accumulations and relations between contexts. Remember, RLHF gives the models some, and perhaps most of these chains-of-thought, when there isn’t sufficient text to scrape for each family of problems. When I see that chain-of-thought, the first thing I think of is of my peers who had write, rewrite, nudge, and correct these chains of thought, and not about core reasoning. The CoT has that same overexplained step-by-step so many RLHF’ers will be accustomed to, and much of it was authored/originated by them. And due to the infinite holes it feels like plugging, I dont call that RL reasoning.

gerdesj 1y ago

You seem to be arguing that bollocks couched in flowery language is a stretch goal of AI. Are you sure?

_heimdall 1y ago
That really depends on expectations. If AI is meant to sound nearly identical to a human, you don't need more training data. If its meant to act as a natural language encyclopedia, we'll never get there with LLMs which amount to natural language processing on top of a massively compressed dataset.
- gerdesj 1y ago
  I've read the Google page "Getting AI to write good SQL: Text-to-SQL techniques explained" and it looks like all the usual bollocks. ... and sigh ... it look like I'm twiddling with a noddy!
mondrian 1y ago
Trying to make AIs more factually-accurate with more training is probably hopeless. Current events and encyclopedic knowledge will be provided by tools. The LLM's core job is to choose the right tools for the job and synthesize their outputs.

meroes 1y ago

What core reasoning?

SpicyLemonZest 1y ago
For example, I just asked o3 "how many Is are in antidisestablishmentarianism", and this was its train-of-thought output: ------ The user wants to know how many “I”s are in the word “antidisestablishmentarianism.” They probably mean the letter "I" in lowercase, though the question was a little broader with "Is." I’ll treat it case-insensitively and count the total occurrences of the letter "i" (both uppercase and lowercase). I can count them manually now, or use a tool for faster analysis. Analyzed word = "antidisestablishmentarianism" word_lower = word.lower() word_lower.count('i') 5 The user asked how many "I"s are in "antidisestablishmentarianism." I’m taking it to mean whether it’s counting "i" or "I", and I’ll consider them as the same for simplicity. So, I’ll count the 5 occurrences of "i" in the word. I manually broke it down and verified there are indeed five "i"s, so the final count should be 5. Pretty straightforward, now that it's all clear! Done There are 5 “i”s in “antidisestablishmentarianism.” ------ You can come up with complicated definitions of "core reasoning" where this doesn't count, and I understand there's great demand for such definitions, but this clearly satisfies common-sense understanding of the term.
- selfhoster 1y ago
  Then I guess Ubuntu has had reasoning for several decades: sudp Command 'sudp' not found, did you mean: command 'sudo' from deb sudo (1.9.9-1ubuntu2.4) command 'sudo' from deb sudo-ldap (1.9.9-1ubuntu2.4) command 'sup' from deb sup (20100519-3) command 'sfdp' from deb graphviz (2.42.2-6) Try: sudo apt install <deb name>
- meroes 1y ago
  I might just be on the opposite side of the aisle, but to me chain-of-thought is better understood as simply more context. Of course there is ambiguity though, more context would be hard to distinguish from core-reasoning and vice versa. I think LLMs/AI mean we can substitute reasoning with vast accumulations and relations between contexts. Remember, RLHF gives the models some, and perhaps most of these chains-of-thought, when there isn’t sufficient text to scrape for each family of problems. When I see that chain-of-thought, the first thing I think of is of my peers who had write, rewrite, nudge, and correct these chains of thought, and not about core reasoning. The CoT has that same overexplained step-by-step so many RLHF’ers will be accustomed to, and much of it was authored/originated by them. And due to the infinite holes it feels like plugging, I dont call that RL reasoning.