4 ms·

Super helpful! Do you have some more info on these issues and where this is discussed? Besides following y'all at explosion, any tipps of whom to follow so I d

• microtonal 3y ago
  I found them out myself when making our own implementation of the model. We test our outputs against upstream models. In decoding without history, our tests passed, but in decoding with history there was a mismatch between our implementation and the upstream implementation. Naturally, I assumed that our implementation was wrong (being the newer implementation, not sharing code with theirs), but while debugging this I found that our implementation is actually correct. Then I was planning to report these issues. Someone else found the causal mask issue a week earlier, so there was no need to report it: https://github.com/pytorch/pytorch/issues/103082 https://github.com/pytorch/pytorch/issues/103082 I reported the issue with rotary embeddings in a discussion of problems that people were running into trying to use KV caching: https://huggingface.co/tiiuae/falcon-40b/discussions/48#648c828eb8f4a3542b7abd4a https://huggingface.co/tiiuae/falcon-40b/discussions/48#648c... More in general, I am not sure what the best place is to track these issues. Maybe a model's discussion forums?