5 ms·

Given that GPT-4 was released a year ago now, and given that GPT-4 is in many ways 'not there yet', GPT-5 will need to show a big material improvement. Anything

Given that GPT-4 was released a year ago now, and given that GPT-4 is in many ways 'not there yet', GPT-5 will need to show a big material improvement. Anything less will signal the end of the road for LLMs.

leroman 3y ago
GPT-4 is already hugely useful. If they are able to lower the cost of GPT-4 further, say 5x or 10x that in itself would be useful and huge.
- consumer451 3y ago
  Tangentially, I lowered my personal cost by more than 3x, and was able to share GPT-4 with my friends and family. I installed LibreChat on a Linode droplet, put $10 on my OpenAI account for API usage, and cancelled my ChatGPT subscription. Since neither I, nor my friends and family use ChatGPT a ton, I let them sign up on my server and the API costs are under $1 so far.
  - infecto 3y ago
    I actually think this is the way. ChatGPT honestly is not that great. I would be happy to pay per token for ChatGPT and be able to have a little more control over the system messages.
    - CuriouslyC 3y ago
      If you regularly use GPT4 for coding this isn't so viable, it gets pricey with the amount of context you need and the default GPT4 chatbot is decent with code. The GPT API is best hit with questions that don't have a lot of setup, can leverage its excellent general knowledge, and can be answered succinctly.
      - infecto 3y ago
        I find it pretty viable for myself. I use this kind of setup within the IDE using API keys. I will flip between 3.5 and 4. The cost of 4 is minimal compared to my time.
  - haswell 3y ago
    This sounds like a great idea and relevant to my current goals. I haven’t used LibreChat; how are you handling authentication?
    - consumer451 3y ago
      I just let them sign up with an email and password, then I turned off the ability to sign up once everyone had done that. LibreChat does support other options [0]. I configured the yaml file to use my own OpenAI API key for all users. [0] https://docs.librechat.ai/install/configuration/user_auth_system.html https://docs.librechat.ai/install/configuration/user_auth_sy... edit: oh, the only other configuration change that I made was to set the default OpenAI model to gpt-4-0125-preview.
      - haswell 3y ago
        Got it, thanks! I’ve been wanting to do some API work anyway, and this seems like a great way to consolidate everything.
- infecto 3y ago
  IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep with implementation on some of these models. If you could both speed up inference time and reduce cost of GPT to 3.5 levels, there are incredible amount of possibilities that open up that could actually help solve a lot of problems have had with trying to interface software to the real world (robotics). 4-Turbo is actually pretty amazing but its still a tad pricey for some tasks. Altman made a comment on Lex's podcast about compute and I think its true, essentially the world does not understand how much compute will be desired as the price of these things go down.
  - lee-rhapsody 3y ago
    Agreed, the chat interface is quite limited compared to what's possible with the API. I recently wrote a script for my job at a publishing company that automatically writes social media promos for every new article published on our site by crawling the sitemap on a cron job. Still putting the finishing touches on it, but hoping to implement it soon. But with only the chat interface, our social media manager would have to find every new article in every discrete vertical on our site, copy all the URLs to a spreadsheet, then prompt ChatGPT for every article, wait for it to respond, then copy that response into the spreadsheet...
  - somewhereoutth 3y ago
    > If you could both speed up inference time and reduce cost of GPT to 3.5 levels Presumably this would be a major part of the big material improvement we'd need to see.
ryandvm 3y ago
What do you mean by "not there yet"? I regularly use GPT-4 for things like "write exhaustive unit tests for the following function" and it gives fantastic results. They usually need fleshed out a little, sure, but in a few seconds I get something that would have take about $45 in billable human hours. I personally cannot wait to see the productivity gains that come from GPT-5.
- somewhereoutth 3y ago
  > They usually need fleshed out a little, sure,... > I personally cannot wait... My point exactly.
  - CuriouslyC 3y ago
    Would you be disappointed if someone came out with a magical new paintbrush that could paint any color in oil, acrylic or watercolor without actual paint because the paintbrush didn't levitate itself and do the painting for you? You can get red in the face at AI the disembodied futurist concept all you want, but the tools are the real deal - people are using image generation and chatbots today to be 50%+ more productive and produce better work. These tools require skill to use, they're not just plug-n-play, but that skill pays huge dividends, and if you don't learn it you're going to be left so far behind.
    - somewhereoutth 3y ago
      > people are using image generation and chatbots today to be 50%+ more productive and produce better work. I don't think we have evidence to support that.
      - CuriouslyC 3y ago
        https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id4573321.pdf https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id...
        3y ago

leroman 3y ago

GPT-4 is already hugely useful. If they are able to lower the cost of GPT-4 further, say 5x or 10x that in itself would be useful and huge.

consumer451 3y ago
Tangentially, I lowered my personal cost by more than 3x, and was able to share GPT-4 with my friends and family. I installed LibreChat on a Linode droplet, put $10 on my OpenAI account for API usage, and cancelled my ChatGPT subscription. Since neither I, nor my friends and family use ChatGPT a ton, I let them sign up on my server and the API costs are under $1 so far.
- infecto 3y ago
  I actually think this is the way. ChatGPT honestly is not that great. I would be happy to pay per token for ChatGPT and be able to have a little more control over the system messages.
  - CuriouslyC 3y ago
    If you regularly use GPT4 for coding this isn't so viable, it gets pricey with the amount of context you need and the default GPT4 chatbot is decent with code. The GPT API is best hit with questions that don't have a lot of setup, can leverage its excellent general knowledge, and can be answered succinctly.
    - infecto 3y ago
      I find it pretty viable for myself. I use this kind of setup within the IDE using API keys. I will flip between 3.5 and 4. The cost of 4 is minimal compared to my time.
- haswell 3y ago
  This sounds like a great idea and relevant to my current goals. I haven’t used LibreChat; how are you handling authentication?
  - consumer451 3y ago
    I just let them sign up with an email and password, then I turned off the ability to sign up once everyone had done that. LibreChat does support other options [0]. I configured the yaml file to use my own OpenAI API key for all users. [0] https://docs.librechat.ai/install/configuration/user_auth_system.html https://docs.librechat.ai/install/configuration/user_auth_sy... edit: oh, the only other configuration change that I made was to set the default OpenAI model to gpt-4-0125-preview.
    - haswell 3y ago
      Got it, thanks! I’ve been wanting to do some API work anyway, and this seems like a great way to consolidate everything.
infecto 3y ago
IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep with implementation on some of these models. If you could both speed up inference time and reduce cost of GPT to 3.5 levels, there are incredible amount of possibilities that open up that could actually help solve a lot of problems have had with trying to interface software to the real world (robotics). 4-Turbo is actually pretty amazing but its still a tad pricey for some tasks. Altman made a comment on Lex's podcast about compute and I think its true, essentially the world does not understand how much compute will be desired as the price of these things go down.
- lee-rhapsody 3y ago
  Agreed, the chat interface is quite limited compared to what's possible with the API. I recently wrote a script for my job at a publishing company that automatically writes social media promos for every new article published on our site by crawling the sitemap on a cron job. Still putting the finishing touches on it, but hoping to implement it soon. But with only the chat interface, our social media manager would have to find every new article in every discrete vertical on our site, copy all the URLs to a spreadsheet, then prompt ChatGPT for every article, wait for it to respond, then copy that response into the spreadsheet...
- somewhereoutth 3y ago
  > If you could both speed up inference time and reduce cost of GPT to 3.5 levels Presumably this would be a major part of the big material improvement we'd need to see.

ryandvm 3y ago

What do you mean by "not there yet"? I regularly use GPT-4 for things like "write exhaustive unit tests for the following function" and it gives fantastic results. They usually need fleshed out a little, sure, but in a few seconds I get something that would have take about $45 in billable human hours. I personally cannot wait to see the productivity gains that come from GPT-5.

somewhereoutth 3y ago
> They usually need fleshed out a little, sure,... > I personally cannot wait... My point exactly.
- CuriouslyC 3y ago
  Would you be disappointed if someone came out with a magical new paintbrush that could paint any color in oil, acrylic or watercolor without actual paint because the paintbrush didn't levitate itself and do the painting for you? You can get red in the face at AI the disembodied futurist concept all you want, but the tools are the real deal - people are using image generation and chatbots today to be 50%+ more productive and produce better work. These tools require skill to use, they're not just plug-n-play, but that skill pays huge dividends, and if you don't learn it you're going to be left so far behind.
  - somewhereoutth 3y ago
    > people are using image generation and chatbots today to be 50%+ more productive and produce better work. I don't think we have evidence to support that.
    - CuriouslyC 3y ago
      https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id4573321.pdf https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id...
      - 3y ago