4 ms·
This is a recurring sentiment but flawed, I think. First of all, neural nets do nit return averages per se. They construct space between the points and extrapo
by kolinko 3mo ago
This is a recurring sentiment but flawed, I think.
First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside
of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it.
Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepted no prob. A thing that doesn’t may still be accepted but with
challenges.
The same is true though for humans, including scientists. There is a saying that science moves one grave at a time - because often prev gen of scientists needs to die off for a new idea to take root.
Or in yet other words - even if llms produced averages, an average
of a discontinuous set can lie outside of that set. And the set of all human ideas is very much discontinuous.
- krona 3mo ago> They construct space between the points and extrapolate outside of the points. They don't. They interpolate between the points on a manifold.
- memoriyato3 3mo agoso the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM? is the proof to the Riehmann hypothesis also somewhere on the manifold and we just need to prod the LLM with the right prompt so it locates the point?
- krona 3mo ago> so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM? By design this must be the case, even when you account for stochastic sampling i.e. 'temperature'. All it's outputs are a highly-dimensional combinatorial interpolation (I'm talking about GPTs here) That's probably why Claude is very good at producing plausible nonsense rather than the often correct response of 'I don't know'.
- kolinko 3mo agoWhat you’re saying here makes no sense.
- kolinko 3mo agoManifold vs space - yes, I meant manifold, english is not my first language and I missed this word. The argument stands.
- krona 3mo agoInterpolation vs extrapolation? It's very simple distinction and fundamental. Look at ARC-AGI-3 for empirical evidence, if that's important to you. My point however is purely an a priori one.