2 ms·
IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep wi
by infecto 3y ago
IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep with implementation on some of these models.
If you could both speed up inference time and reduce cost of GPT to 3.5 levels, there are incredible amount of possibilities that open up that could actually help solve a lot of problems have had with trying to interface software to the real world (robotics).
4-Turbo is actually pretty amazing but its still a tad pricey for some tasks. Altman made a comment on Lex's podcast about compute and I think its true, essentially the world does not understand how much compute will be desired as the price of these things go down.
- lee-rhapsody 3y agoAgreed, the chat interface is quite limited compared to what's possible with the API. I recently wrote a script for my job at a publishing company that automatically writes social media promos for every new article published on our site by crawling the sitemap on a cron job. Still putting the finishing touches on it, but hoping to implement it soon. But with only the chat interface, our social media manager would have to find every new article in every discrete vertical on our site, copy all the URLs to a spreadsheet, then prompt ChatGPT for every article, wait for it to respond, then copy that response into the spreadsheet...
- somewhereoutth 3y ago> If you could both speed up inference time and reduce cost of GPT to 3.5 levels Presumably this would be a major part of the big material improvement we'd need to see.