3 ms·

I mentioned in another post here that I believe that one major reason LLMs aren't ubiquitous right now is inference cost. I'm going to guess that one major rea

I mentioned in another post here that I believe that one major reason LLMs aren't ubiquitous right now is inference cost. I'm going to guess that one major reason Copilot fails at complex code is because it can't read enough of your codebase to suggest complex code. The technical reason is because inference cost is too high right now for very large context queries. Therefore, the major bottleneck is inference cost. That's why I'm investing in TSMC because we will need a lot more silicon to reduce inference cost. I imagine a world in which future GPT versions are at least 2x better than GPT4, context size is 100x higher, and inference cost is 100x lower.

fakedang 3y ago
> That's why I'm investing in TSMC because we will need a lot more silicon to reduce inference cost. Or what we really need is another approach to either the hardware problem (more efficient chip designs that may be entirely different from existing ones), or the software problem (more efficient ML core models). The latter is obviously more likely to happen, rather than the former.
- aurareturn 3y ago
  It's going to have to be both. But I don't know who to bet on for solving the software problem. The hardware problem will depend on chip fabs, TSMC, Intel, Samsung. Since I can't buy Samsung stocks easily, I bet mostly on TSMC and a bit of Intel for geopolitics reasons. There are also chip designers such as Nvidia, AMD, Apple, ARM, etc that will need to solve the inference problem. I'm already invested in Nvidia and Apple. I don't know who else to invest in right now. Many of the AI chip design companies are private and startups.