3 ms·
I've seen the appearance of PCs, and the impact of 4 decades of Moore's law on price/performance. With O1 and now O3, the main limits of what you can do with an
by mikewarot 2y ago
I've seen the appearance of PCs, and the impact of 4 decades of Moore's law on price/performance. With O1 and now O3, the main limits of what you can do with an LLM seem to center on just how much compute you can give it.
I hope to offer a few more decades of performance boost to the show, with Isolinear Memory[1], to help dissolve those limitations. ;-)
[1] https://bitgrid.blogspot.com/2024/09/bitgrid-and-isolinear-memory.html https://bitgrid.blogspot.com/2024/09/bitgrid-and-isolinear-m...
- ynniv 2y agoI think o1 and o3 are OpenAI stretching the usefulness of 4o. Anecdotally the latest Claude model is naturally better than 4o, and having it use inference tokens to think through or verify work results in similar o1/o3 gains. Are we sure we can't do even better than Claude 3.5 Sonnet/new without inference tricks? We're only a few generations into LLMs... I don't see why not.