4 ms·

So it appears that GPT-4-Turbo is indeed (at least marginally) smarter than the previous GPT-4, just as Altman claimed. Also, it's faster and cheaper, with a ma

So it appears that GPT-4-Turbo is indeed (at least marginally) smarter than the previous GPT-4, just as Altman claimed. Also, it's faster and cheaper, with a massive context window. Exciting!

doctoboggan 3y ago
Yes I was really worried this was going to be "better" as in faster and cheaper but not "better" as in smarter. I've been playing with it for the past day and haven't noticed it any smarter per se, but also haven't noticed it dummer either.
kridsdale3 3y ago
It's a big surprise. I assumed something would have to have been "lobotomized" to make this speed increase. I'd love to know what they did.
- bigyikes 3y ago
  Well, the knowledge cutoff is much more recent (I think sama said April 2023?), so having more, newer data might be a significant contributing factor.
Racing0461 3y ago
If this is the case, why use the word turbo in the name which has the baggage of faster but worse reasoning.
- jquery 3y ago
  Perhaps it may have worse reasoning for some tasks, so having "turbo" lets people know they can still try the old version if turbo doesn't work for them? Kind of like 3.5 vs 3.5 Turbo.
japhyr 3y ago
The increased context size will have the most significant impact on my work. That's where I run into limitations, when reviewing written work and code. I've been feeding written work into GPT in chunks, and I'm really happy to be able to feed in whole pieces. (I don't have it revise anything for me, I just have a specific prompt for exactly the kind of feedback I want on written work.) I tried Claude because of the larger context size, but I've been disappointed so far. I find Claude much more likely to just compliment my writing, whereas GPT will identify strengths and areas that could be improved.
- pests 3y ago
  Have you tried aider-chat? It does some interesting things with tree-sitter so it can give the LLM a context (files, classes, functions, parameters, etc) as well as certain full files. That way it has your entire codebase in API form and it can focus on the actual code you are looking at or editing.
berkut 3y ago
I haven't tried it yet, but people in the /r/chatgpt subreddit are claiming GPT-4-Turbo seems to have issues with understanding/remembering longer (say 100 lines) of code, whereas 3.5 and 4.0 seem to have handled things a bit better, implying that the context-window size isn't (currently) as large as claimed. Anyone else seeing any evidence of this?
- ramblerman 3y ago
  Some specialized subreddits can be incredibly useful. /r/chatgpt due to its popularity is not one of those. It's full of memes and people complaining its not as "good" as it was yesterday when it fails at completing their homework. I would take anything said there with a big grain of salt, and stick to benchmarks.
- bbahn 3y ago
  The context window IS longer, but it's less powerful. Obviously, they can't afford to have full transformer context over the entire context. That would be an impossibly large amount of ram. They're using some combination of sliding window/cyclical/or some other adjusted attention mechanism likely with some degree of summarization in some manner.

So it appears that GPT-4-Turbo is indeed (at least marginally) smarter than the previous GPT-4, just as Altman claimed. Also, it's faster and cheaper, with a massive context window. Exciting!

doctoboggan 3y ago

Yes I was really worried this was going to be "better" as in faster and cheaper but not "better" as in smarter. I've been playing with it for the past day and haven't noticed it any smarter per se, but also haven't noticed it dummer either.

kridsdale3 3y ago

It's a big surprise. I assumed something would have to have been "lobotomized" to make this speed increase. I'd love to know what they did.

bigyikes 3y ago
Well, the knowledge cutoff is much more recent (I think sama said April 2023?), so having more, newer data might be a significant contributing factor.

Racing0461 3y ago

If this is the case, why use the word turbo in the name which has the baggage of faster but worse reasoning.

jquery 3y ago
Perhaps it may have worse reasoning for some tasks, so having "turbo" lets people know they can still try the old version if turbo doesn't work for them? Kind of like 3.5 vs 3.5 Turbo.

japhyr 3y ago

The increased context size will have the most significant impact on my work. That's where I run into limitations, when reviewing written work and code. I've been feeding written work into GPT in chunks, and I'm really happy to be able to feed in whole pieces. (I don't have it revise anything for me, I just have a specific prompt for exactly the kind of feedback I want on written work.) I tried Claude because of the larger context size, but I've been disappointed so far. I find Claude much more likely to just compliment my writing, whereas GPT will identify strengths and areas that could be improved.

pests 3y ago
Have you tried aider-chat? It does some interesting things with tree-sitter so it can give the LLM a context (files, classes, functions, parameters, etc) as well as certain full files. That way it has your entire codebase in API form and it can focus on the actual code you are looking at or editing.

berkut 3y ago

I haven't tried it yet, but people in the /r/chatgpt subreddit are claiming GPT-4-Turbo seems to have issues with understanding/remembering longer (say 100 lines) of code, whereas 3.5 and 4.0 seem to have handled things a bit better, implying that the context-window size isn't (currently) as large as claimed. Anyone else seeing any evidence of this?

ramblerman 3y ago
Some specialized subreddits can be incredibly useful. /r/chatgpt due to its popularity is not one of those. It's full of memes and people complaining its not as "good" as it was yesterday when it fails at completing their homework. I would take anything said there with a big grain of salt, and stick to benchmarks.
bbahn 3y ago
The context window IS longer, but it's less powerful. Obviously, they can't afford to have full transformer context over the entire context. That would be an impossibly large amount of ram. They're using some combination of sliding window/cyclical/or some other adjusted attention mechanism likely with some degree of summarization in some manner.