3 ms·
Based on my skimming the paper, am I correct in understanding that they came up with an elaborate collection of prompts that embed the text generated by GPT-2 a
by davesque 3y ago
Based on my skimming the paper, am I correct in understanding that they came up with an elaborate collection of prompts that embed the text generated by GPT-2 as well as a representation of GPT-2's internal state? Then, in effect, they simply asked GPT-4, "What do you think about all this?"
If so, they're acting on a gigantic assumption that GPT-4 actually correctly encodes a reasonable model of the body of knowledge that went into the development of LLMs.
Help me out. Am I missing something here?
- MacsHeadroom 3y ago>If so, they're acting on a gigantic assumption Yes the initial hypothesis that GPT-4 would know was a gigantic assumption. But a falsifiable one which we can easily generate reproducible tests for. The idea that simulated neurons could learn anything useful at all was once a gigantic assumption too.
- jxmorris12 3y agoNeural networks aren’t simulated neurons…
- sebzim4500 3y agoAfter GPT-4 generates the hypothesis for a neuron they test it by comparing GPT-4's expectation for where the neuron should fire against where it actually fires. If you squint it's train/test separation.
- lennxa 3y agoWhy does it have to understand how the LLMs are built? They have used gpt-4 to just build a classifier for each neuron's activition, and given the nlp abilities of gpt-4, the hope is that it can describe the nature of activation of the neurons.