A remarkable comment in that it is clear, confident and wrong. Fine-tunes lead to catastrophic forgetting. RAG is only irrelevant if you’re completely disinte

A remarkable comment in that it is clear, confident and wrong. Fine-tunes lead to catastrophic forgetting. RAG is only irrelevant if you’re completely disinterested in cost and latency. We also don’t have enough data to gauge performance of models >200k context window size when reasoning over inputs of that size, much of which will be irrelevant to any particular user. Multiple random needles in haystack tests work flawlessly, but rarely applies to real world activity.