4 ms·
I wonder what about this one gets the +0.5 to the name. IIRC the 2.0 model isn’t particularly old yet. Is it purely marketing, does it represent new model struc
by vineyardmike 2y ago
I wonder what about this one gets the +0.5 to the name. IIRC the 2.0 model isn’t particularly old yet. Is it purely marketing, does it represent new model structure, iteratively more training data over the base 2.0, new serving infrastructure, etc?
I’ve always found the use of the *.5 naming kinda silly when it became a thing. When OpenAI released 3.5, they said they already had 4 underway at the time, they were just tweaking 3 be better for ChatGPT. It felt like a scrappy startup name, and now it’s spread across the industry. Anthropic naming their models Sonnet 3, 3.5, 3.5 (new), 3.7 felt like the worst offender of this naming scheme.
I’m a much bigger fan of semver (not skipping to .5 though), date based (“Gemini Pro 2025”), or number + meaningful letter (eg 4o - “Omni”) for model names.
- forbiddenvoid 2y agoI would consider this a case of "expectation management"-based versioning. This is a release designed to keep Gemini in the news cycle, but it isn't a significant enough improvement to justify calling it Gemini 3.0.
- jstummbillig 2y agoI think it's reasonable. The development process is just not really comparable to other software engineering: It's fairly clear that currently nobody really has a good grasp on what a model will be while they are being trained. But they do have expectations. So you do the training, and then you assign the increment to align the two.
- 8n4vidtmkvmk 2y agoI figured you don't update the major unless you significantly change the... algorithm, for lack of a better word. At least I assume something major changed between how they trained ChatGPT 3 vs GPT 4, other than amount of data. But maybe I'm wrong.
- eru 2y agoThe number is purely for marketing. If you could get much better performance without changing the algorithm (eg just by scaling), you'd still bump the number.
- KoolKat23 2y agoFunnily enough, from early indications (user feedback) this new model would've been worthy of the 3.0 moniker, despite what the benchmarks say.
- falcor84 2y agoAgreed, can't everyone just use semantic versioning, with 0.1 increments for regular updates?
- laurentlb 2y agoRegarding semantic versioning: what would constitute a breaking change? I think it makes sense to increase the major / minor numbers based on the importance of the release, but this is not semver.
- falcor84 2y agoAs I see it, if it uses a similar training approach and is expected to be better in every regard, then it's a minor release. Whereas when they have a new approach and where there might be some tradeoffs (e.g. longer runtime), it should be a major change. Or if it is very significantly different, then it should be considered an entirely differently named model.
- morkalork 2y agoOr drop the pretext of version numbers entirely since they're meaningless here and go back to classics like Gemini Experience, Gemini: Millennium Edition or Gemini New Technology
- joaogui1 2y agoWould be confusing for non-tech people once you did x.9 -> x.10
- guelo 2y agoWhat would a major version bump look like for an llm?
- eru 2y agoGoing from English to Chinese, I guess? Because that would not be a compatible version for most previous users.
- Workaccount2 2y agoAt least for OpenAI, a .5 increment indicates a 10x increase in training compute. This so far seems to track for 3.5, 4, 4.5.
- aoeusnth1 2y agoI think it's because of the big jump in coding benchmarks. 74% on aider is just much, much better than before and worthy of a .5 upgrade.
- alphabetting 2y agoThe elo jump and big benchmark gains could be justification
- utopcell 2y agoIt may indicate a Tick-Tock [1] process. [1] https://en.wikipedia.org/wiki/Tick%E2%80%93tock_model https://en.wikipedia.org/wiki/Tick%E2%80%93tock_model