4 ms·
I'm not convinced at all. The only thing they are doing is perturbing some of the model weights in intermediate layers, and seeing if the output of the final la
by semi-extrinsic 2y ago
I'm not convinced at all. The only thing they are doing is perturbing some of the model weights in intermediate layers, and seeing if the output of the final layer is consistent with the perturbations. It would be a shitty model if that was not the case.
The fancy part in the paper is figuring out how to perturb the intermediate layers in the way you want. But the findings are not impressive.
Note also that the "probe geometry" stuff is so speculative they left it out of the academic paper completely.
In the same way, it has been known since the 90s that if you take the matrices from Finite Element Models and visualize them as graphs, structures appear that kind of resemble the physical appearance of the object being modelled. Here for instance is for a helicopter:
http://yifanhu.net/GALLERY/GRAPHS/GIF_SMALL/Pothen@commanche_dual.html http://yifanhu.net/GALLERY/GRAPHS/GIF_SMALL/Pothen@commanche...
Yet nobody thinks Finite Element Models have an internal mental representation of the world.
- FrustratedMonky 2y ago"Yet nobody thinks Finite Element Models have an internal mental representation of the world." At this point, I'm not sure some wouldn't argue that. The difference is, put the AI on a loop, with constant feedback, learning. Instead of just a 'pre-trained' model. Make the actual model, live, always learning, so the context window is infinite. This of course would not be for everyone, because it would take all the resources of the training infrastructure to be focused on one person/view. But that gets closer to the human mind, and at that point, we probably couldn't say for sure that the 'perturbations' aren't experiencing something subjective. Where is the proof that humans have an internal mental representation of the world.