I have been working on inference software for my own use, with both CPU and GPU versions: https://github.com/ryao/llama3.c https://github.com/ryao/llama3.c Th

I have been working on inference software for my own use, with both CPU and GPU versions: https://github.com/ryao/llama3.c https://github.com/ryao/llama3.c The only thing you wrote that makes any sense to me is “Flash attention lets you fuse the kernel”. Everything else you wrote makes no sense to me. For what it is worth, flash attention does not apply to llama 3 inference as far as I can tell.