7 ms·

GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using somet

GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof. TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind... And you can't even use Bard as a developer, there's no API! But GPT-4 is so, so much better. It's not clear to me that individual people doing LoRa at home is going to meaningfully close the gap in terms of generalized capability — at least, not faster than OpenAI itself improves its models. Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality. Edit: re: the complaints about Midjourney's UI being Discord — sure, that definitely constrains what you can do with it, but OpenAI's interface isn't Discord, it has an API. And you can fine-tune the GPT-3 models programmatically too, and although they haven't opened that up to GPT-4 yet, IME you can't fine-tune your way to GPT-4 quality anyway with anything. "There's no moat" and "OpenAI is irrelevant" feel like the cries of the company that's losing to OpenAI and wants to save face on the way out. Getting repeated generational improvements without the dataset size and compute scale of a dedicated, well-capitalized company is going to be very tough. As a somewhat similar data+compute problem, I can't think of an open-source project that effectively dethroned Google Search, for example... At least, not by being better at search (you can argue that maybe LLMs are dethroning Google, but on the other hand, it's not the open source models that are the best at that, it's closed-source GPT-4).

karmasimida 3y ago
GPT-4 is a must if tool using is your goal. GPT-3.5, I think it is mostly suitable for: 1. Quick documentation lookup for non-essential facts 2. Lightweight documents writing and rewriting 3. Translation Other use cases should go straightly to GPT-4
- feanaro 3y ago
  Ah, "tool using", the best of goals.
- biesnecker 3y ago
  I use GPT-3.5 for a lot of terminology lookup, and it's generally pretty great. "In the context of [field I'm ramping up in], what does X mean, and how is it different than Y" -- it's not as good as GPT4 but it emits so much quicker and it normally gets me where I needed to go.
- deleted 3y ago
  [deleted]
joshbert 3y ago
I don't feel sorry for Google, nor the big amounts of PR nonsense they're putting out there in order to try to spin their being too slow to move LLM tech to the side of the consumer. Get better or get out.
ineedasername 3y ago
Yes, I'd readily pay for GPT-4 access, though not the limited 25 requests per 3 hours version. I ponied up $20 for a month of usage to check it out, and it performs head & shoulders above 3.5 in its ability to comprehensively address more complex prompts and provide output that is more nuanced than ChatGPT. I'll also point out that paid api access to 3.5 (davinci-03) is frequently better than ChatGPT's use of 3.5. You get many fewer restrictions, and none the "awe shucks, I'm just a little 'ol LLM and so I couldn't possibly answer that". If you're frustrated by having to go to great lengths to prompt engineer and ask ChatGPT to "pretend" then it's worth it to pay for API access. I'm just frustrated that I can't use the GPT-4 API the same way yet (waitlist)
- reissbaker 3y ago
  I hear ya! I'm out here dying on the GPT-4 API waitlist too. I use gpt-3.5-turbo's API extensively, and occasionally copy my prompts into GPT-4's web UI and watch as it just flawlessly does all the things 3.5 struggles with. Very frustrating since I don't have GPT-4 API access, but also very, very impressive. It's not even remotely close. I pay the $20 for ChatGPT Plus (aka, GPT-4 web interface access); personally I find it useful enough to be worth paying for, even in its rate-limited state. It already replaces Google for anything complex for me. I wish I could pay for the API too, and use it in my projects.
  - moffkalast 3y ago
    GPT 4 really shows how absolutely terrible regular web search is at finding anything these days. Another complete embarrassment for Google. Often times it can just recite things from memory that Google can't even properly link to, and they've got a proper index to work from for fucks sake.
    - Joeri 3y ago
      The last time I tried to find a plumber in my local area through google I realized that the first three pages of results contained zero actual results. It was a mix of ads and seo spam from scammers. I ended up going to ddg and while there was plenty of seo spam there too, I found several good results on the first two pages. Google has the technology and talent to relaunch themselves in a leadership position, but the current executive team doesn’t seem to have what it takes. They’re custodians / accountants, running the company a bit like Microsoft in the Ballmer era. What google needs to do now is leap ahead, and I don’t see it happening without a leadership change.
spyckie2 3y ago
The question is how long will it take for open source to become just as good as GPT 4? If it is 3 years, then yes, this is copium. But if it is 1 year or less, then how much is google really missing out on? OpenAI spent 600m to improve GPT and made 200m from it and if costs dramatically fall for model development, it might be OpenAI that is shooting itself in the foot.
- p1esk 3y ago
  OpenAI spent 600m to improve GPT and made 200m from it How do you know?
cornel_io 3y ago
> And you can't even use Bard as a developer, there's no API! There is an API for the underlying model, it's just in alpha/beta/whatever they call limited invite-only release and you have to ask your devrel team to get access. I'm guessing we'll see better models very soon. Google is, as usual, playing catch-up, but I have no doubt once the machine gets cranking they'll be fully competitive, at least similar to how GCP is now a totally viable AWS alternative. They never lead the pack because they can't (lawyers, regulation, monopoly, etc), but they know how to commit and execute.
SanderNL 3y ago
SD and its configurability is miles and miles ahead of MJ. Sure if you want a fancy picture now it’s OK. How are you going to generate that same picture in another pose? Inpainting, outpainting.. I don’t even know where to begin. MJ is a toy compared to SD’s ecosystem.
whywhywhywhy 3y ago
>Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality. MJ only does one style, and you can emulate that just fine in SD if thats what you want.

karmasimida 3y ago

GPT-4 is a must if tool using is your goal. GPT-3.5, I think it is mostly suitable for: 1. Quick documentation lookup for non-essential facts 2. Lightweight documents writing and rewriting 3. Translation Other use cases should go straightly to GPT-4

feanaro 3y ago
Ah, "tool using", the best of goals.
biesnecker 3y ago
I use GPT-3.5 for a lot of terminology lookup, and it's generally pretty great. "In the context of [field I'm ramping up in], what does X mean, and how is it different than Y" -- it's not as good as GPT4 but it emits so much quicker and it normally gets me where I needed to go.
deleted 3y ago
[deleted]

joshbert 3y ago

I don't feel sorry for Google, nor the big amounts of PR nonsense they're putting out there in order to try to spin their being too slow to move LLM tech to the side of the consumer. Get better or get out.

ineedasername 3y ago

Yes, I'd readily pay for GPT-4 access, though not the limited 25 requests per 3 hours version. I ponied up $20 for a month of usage to check it out, and it performs head & shoulders above 3.5 in its ability to comprehensively address more complex prompts and provide output that is more nuanced than ChatGPT. I'll also point out that paid api access to 3.5 (davinci-03) is frequently better than ChatGPT's use of 3.5. You get many fewer restrictions, and none the "awe shucks, I'm just a little 'ol LLM and so I couldn't possibly answer that". If you're frustrated by having to go to great lengths to prompt engineer and ask ChatGPT to "pretend" then it's worth it to pay for API access. I'm just frustrated that I can't use the GPT-4 API the same way yet (waitlist)

reissbaker 3y ago
I hear ya! I'm out here dying on the GPT-4 API waitlist too. I use gpt-3.5-turbo's API extensively, and occasionally copy my prompts into GPT-4's web UI and watch as it just flawlessly does all the things 3.5 struggles with. Very frustrating since I don't have GPT-4 API access, but also very, very impressive. It's not even remotely close. I pay the $20 for ChatGPT Plus (aka, GPT-4 web interface access); personally I find it useful enough to be worth paying for, even in its rate-limited state. It already replaces Google for anything complex for me. I wish I could pay for the API too, and use it in my projects.
- moffkalast 3y ago
  GPT 4 really shows how absolutely terrible regular web search is at finding anything these days. Another complete embarrassment for Google. Often times it can just recite things from memory that Google can't even properly link to, and they've got a proper index to work from for fucks sake.
  - Joeri 3y ago
    The last time I tried to find a plumber in my local area through google I realized that the first three pages of results contained zero actual results. It was a mix of ads and seo spam from scammers. I ended up going to ddg and while there was plenty of seo spam there too, I found several good results on the first two pages. Google has the technology and talent to relaunch themselves in a leadership position, but the current executive team doesn’t seem to have what it takes. They’re custodians / accountants, running the company a bit like Microsoft in the Ballmer era. What google needs to do now is leap ahead, and I don’t see it happening without a leadership change.

spyckie2 3y ago

The question is how long will it take for open source to become just as good as GPT 4? If it is 3 years, then yes, this is copium. But if it is 1 year or less, then how much is google really missing out on? OpenAI spent 600m to improve GPT and made 200m from it and if costs dramatically fall for model development, it might be OpenAI that is shooting itself in the foot.

p1esk 3y ago
OpenAI spent 600m to improve GPT and made 200m from it How do you know?

cornel_io 3y ago

> And you can't even use Bard as a developer, there's no API! There is an API for the underlying model, it's just in alpha/beta/whatever they call limited invite-only release and you have to ask your devrel team to get access. I'm guessing we'll see better models very soon. Google is, as usual, playing catch-up, but I have no doubt once the machine gets cranking they'll be fully competitive, at least similar to how GCP is now a totally viable AWS alternative. They never lead the pack because they can't (lawyers, regulation, monopoly, etc), but they know how to commit and execute.

SanderNL 3y ago

SD and its configurability is miles and miles ahead of MJ. Sure if you want a fancy picture now it’s OK. How are you going to generate that same picture in another pose? Inpainting, outpainting.. I don’t even know where to begin. MJ is a toy compared to SD’s ecosystem.

whywhywhywhy 3y ago

>Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality. MJ only does one style, and you can emulate that just fine in SD if thats what you want.