2 ms·
flash-attention is a additional and later technique to accelerate computation of attention, That's why it's not the default option for llama.cpp
by informal007 2y ago
flash-attention is a additional and later technique to accelerate computation of attention, That's why it's not the default option for llama.cpp