3 ms·
Is it just that someone hasn't built a debugger for an LLM, or is there something fundamental that prevents them from being debugged?
by booleandilemma 4y ago
Is it just that someone hasn't built a debugger for an LLM, or is there something fundamental that prevents them from being debugged?
- sanxiyn 4y agoWe are building debuggers (really more like disassemblers) and we are reverse engineering LLMs. There is no fundamental difficulty, but it takes time. For example, one layer transformer eventually learns to do modular addition completely correctly. Looking at weights, it does so by doing rotation on unit circle. For a + b = c mod p, probability of c is proportional to cos((a+b-c) 2π/p), which works because cosine is maximized when the argument is multiple of 2π. As you can see, there is no fundamental difficulty, but it is also not trivial. https://arxiv.org/abs/2301.05217 https://arxiv.org/abs/2301.05217