3 ms·
Does anyone know if the attention mechanism truly does consider all of the context or does the model use some of the recent tricks for reducing the context size
by doctoboggan 3y ago
Does anyone know if the attention mechanism truly does consider all of the context or does the model use some of the recent tricks for reducing the context size before running the attention process?