3 ms·
If you ask at the end of the prompt then it may have already deliberately tossed the information it deemed irrelevant prior to the question. These aren't transf
by hackerlight 3y ago
If you ask at the end of the prompt then it may have already deliberately tossed the information it deemed irrelevant prior to the question. These aren't transformers. In general the recall for arbitrary information will be worse.
- kgeist 3y agoSo the questions should come before the content and it might work? I think that's how also RWKV works.
- hackerlight 3y agoIt's known to help although I wouldn't expect it to be perfect recall unless the network is big enough. The network will read the data token by token. So if you put the question at the beginning it will know what information it needs to pay attention to inside the rest of your context. Of course, if the network is too small, it still won't be perfect recall for a sufficiently complicated/large question/context.