2 ms·

Just regarding "frontier model", it is most definitely not just a marketing term. It was borrowed from the financial concept of "efficient frontier", i.e. the b

Just regarding "frontier model", it is most definitely not just a marketing term. It was borrowed from the financial concept of "efficient frontier", i.e. the best performance at a particular cost, though in the context of things like "pacing the frontier" it refers to the highest capability models. ECI (Epoch Capability Index) is probably the most commonly used metric of overall capability: https://epoch.ai/eci?view=graph&tab=release-date https://epoch.ai/eci?view=graph&tab=release-date "What this has to do with anything anyway" is that pushing the boundaries on AI capabilities has required tons of compute (and money). Open weight models exist and are useful but they always trail in capabilities and they can't be used as evidence to think somehow you can push the boundaries of LLMs without a shit ton of money.

jamienk 7d ago
"can't be used as evidence" << the whole field is only a few yrs old and is based on academic maths from decades ago. A lot of progress is in harnesses and distillation and weird research that requires creativity. The "Frontier" always tries to frame things in terms of "the frontier" in order to limit how we look at things. We don't know the true costs of pre- and post- training, running data-centers, hiring people, making business deals, etc., so even the (maybe) quantifiable question of cost vs capabilities is not at all clear. The efficiency of businesses ultimately gets measured by their ability to maximize a commercial return; but this is NOT a good metric for other important things, including maybe "how good is this"? (Where "good" is open to debate but maybe not quantifiable.) It had been true that the "best" operating systems required a lot of business money. And encyclopedias. And ... (list some other good historical examples here!) ... Are Linux and Wikipedia the "efficient frontier"? In a sense they are both infinite performance at zero cost. Hard cash outlays sometimes stop being the determinant for high-effort endeavors. I remember in the 1990s Encyclopedia Britannica's CDs and MSN (Microsoft Network(tm)) among many others argued that the web would never be able to match the curated, bespoke, expensive products they offered. It was IMPOSSIBLE for that crap to be matched! It is a POLITICAL question.

jamienk 7d ago

"can't be used as evidence" << the whole field is only a few yrs old and is based on academic maths from decades ago. A lot of progress is in harnesses and distillation and weird research that requires creativity. The "Frontier" always tries to frame things in terms of "the frontier" in order to limit how we look at things. We don't know the true costs of pre- and post- training, running data-centers, hiring people, making business deals, etc., so even the (maybe) quantifiable question of cost vs capabilities is not at all clear. The efficiency of businesses ultimately gets measured by their ability to maximize a commercial return; but this is NOT a good metric for other important things, including maybe "how good is this"? (Where "good" is open to debate but maybe not quantifiable.) It had been true that the "best" operating systems required a lot of business money. And encyclopedias. And ... (list some other good historical examples here!) ... Are Linux and Wikipedia the "efficient frontier"? In a sense they are both infinite performance at zero cost. Hard cash outlays sometimes stop being the determinant for high-effort endeavors. I remember in the 1990s Encyclopedia Britannica's CDs and MSN (Microsoft Network(tm)) among many others argued that the web would never be able to match the curated, bespoke, expensive products they offered. It was IMPOSSIBLE for that crap to be matched! It is a POLITICAL question.