3 ms·
Yes it looks like it can use this method. This method is a preprocessor and post-processor that can be used on an existing GPT model to augment it to handle unl
by Kranar 3y ago
Yes it looks like it can use this method. This method is a preprocessor and post-processor that can be used on an existing GPT model to augment it to handle unlimited tokens.
- vintermann 3y agoAnd that makes it pretty notable compared to all the linear attention/retrieval schemes that didn't pan out. Not saying this will pan out, but we'll know more without waiting six months for the model to train.