2 ms·
Your observations are right but pretty insane to consider them a pure PR company lol. They are making more frequent releases so yes the release-to-release quali
by aspenmartin 4mo ago
Your observations are right but pretty insane to consider them a pure PR company lol. They are making more frequent releases so yes the release-to-release quality is smaller but we’re still ascending quality and reliability curves the same way we have since GPT-3. You get a GPT4->5 leap every like 17 or 18 months I think it is
- kingkongjaffa 4mo agoThe gradient of improvement is absolutely not the same.
- aspenmartin 4mo agoIf anything its slightly higher. Feel free to provide any evidence to the contrary. ECI (good aggregate measure using IRT): https://epoch.ai/eci?view=graph&tab=release-date&subset-view=graph&subset-tab=Software+engineering https://epoch.ai/eci?view=graph&tab=release-date&subset-view... METR time horizon (now topped out): https://metr.org/time-horizons/ https://metr.org/time-horizons/
- WASDx 4mo agoI like this one, although its data seem to overlap with ECI. https://artificialanalysis.ai/trends https://artificialanalysis.ai/trends