> Or is it that the sauce isn’t that special? The sauce is special, but the recipe is already known. Most of the stuff things like LLMs are based on comes from

> Or is it that the sauce isn’t that special? The sauce is special, but the recipe is already known. Most of the stuff things like LLMs are based on comes from published research, so in principle coming up with the architecture that can do something very close, is doable to everyone with the skills to understand the research material. The problems start with a) taking the architecture to a finished and fine tuned model and b) running that model. Because now we are talking about non-trivial amounts of compute, storage and bandwidth, so quite simple resources suddenly become a very real problem.