I'm highly skeptical. The current ML paradigm is highly reliant on aggregating data, but the issue we're discussing is about distinguishing subtle details over

I'm highly skeptical. The current ML paradigm is highly reliant on aggregating data, but the issue we're discussing is about distinguishing subtle details over an extremely large search space. Sure, you can probably scale your way there but even accounting for superposition we're talking about an extremely large number of parameters because you aren't performing search, you're performing compression. You need to also remember the curse of dimensionality. The problem is that as the dimension increases the ability to distinguish the nearest neighbor from the further neighbor decreases. Effectively the notion of distance becomes undefined. (The dimensionality increases as parameters increase). So now you have to perform search over your compression. This is why ML is so fucking cool but it's also why they are really bad at details. Why you have to really wrestle with them to handle nuance. Easiest to see in image generators but they're much smaller. Do remember that these things are specific trained so that their outputs are preferential to humans. The result is that errors are in the direction of being difficult to be detected by human evaluators. Deciding if that's a feature or bug requires careful consideration. This is not to say that LLMs and ML is useless or trash. They are impressive and powerful machines but neither are they magic and the answer to everything. We got to understand the limitations if we're to move forward. I mean that's the job of us here as researchers, engineers, and developers. Using a keen eye to find limits and then solve them (easier said than done lol)