3 ms·
There are an enormous number of tasks that can get by on good enough. If you need image recognition, and a 30B model saturates the use case with 100% accuracy,
by adam_arthur 16d ago
There are an enormous number of tasks that can get by on good enough.
If you need image recognition, and a 30B model saturates the use case with 100% accuracy, you absolutely wouldn't continue to use the next frontier model as they come out.
And I'd argue most economically meaningful tasks will be saturated by cheaper models than those requiring frontier.
Think about what today's models can do with pretty close to 100% accuracy, and then consider that they will be orders of magnitudes cheaper over the years.
5.6 Sol can already obviate tons of labor, and why would you pay 2x or more for no meaningful gain?
The relative gap between frontier and non frontier also continues to shrink, so it's not like you take a meaningful performance loss by rewinding to models from 3-6 months ago. And soon that gap will expand to 12-24 months.
I get the impression the majority of people on here only think about coding, which net net will be a tiny volume of overall AI use in the end.
- bilater 16d agoThat can all be true but the frontier models will still have a huge market. You're thinking of all tasks as a fixed pie. The top 1% of intelligence opens up a whole new pie, stuff nobody does today because it's too expensive: daily cancer scans instead of one every few years, asteroid mining missions that need ten thousand PhD-hours of planning, custom drugs designed for your specific tumor, a personal lawyer and doctor for every person on earth, auditing every line of code in every bank and hospital continuously and so on.
- michaellee8 16d agoyea you see no body are vibecoding games before opus 5 and astra, after they are released games basically got commoditized
- adam_arthur 16d agoIf the output is commoditized, how much can you afford to pay for the input?
- sampullman 16d agoIs this sarcasm? I assume most studios are integrating AI into their workflows, but I still haven't seen a single vibecoded game that looks interesting.
- chasd00 16d agoI was yelled at repeatedly here that AI is not used in gaming.
- adam_arthur 16d agoYes, agree that token consumption will increase exponentially for the next while. Disagree that the frontier model is where the economic gains will be realized. The smaller the relative gap between frontier and non-frontier/open weights, the less pricing power. This gap has shown only to shrink over time, not expand. Businesses will pay more for frontier, but not meaningfully more to justify the economics. It's always going to be a low margin business, perhaps outside of cyber security, warfare/intelligence and perhaps drug discovery. Though the expensive and time consuming part of drugs is doing the trials and getting approval, not coming up with ideas
- bilater 16d agohow much would you pay for a prompt that could cure cancer? if you're a pharma company you would pay millions to get there days faster than your competitor. as intelligence rises the marginal value it can deliver rises with it.
- philipkglass 16d agoSomething like curing cancer (more realistically, curing a specific kind of cancer) has to interact with much slower real-world processes. The most expensive part of drug development is Phase 3 clinical trials in humans. Even the smartest model in the world can't accelerate that meaningfully. Even much earlier when drugs are just testing in cell cultures, it's a lot slower to run lab tests than to run software tests or mathematical proof checkers. Or to put it another way, there's enough natural variation in real-world bottlenecks that no pharma company can assume they'll beat competitors to market by using a smarter model. A really smart model could significantly improve the pharma business if it could identify promising approaches to cancer treatment that are less likely to fail in clinical trials, but I don't think that the frontier labs have data to make that work yet. Much of the biomedical literature is poorly reproducible ("replication crisis") and much of the drug-development-specific data is proprietary, never published in the first place. I do have hopes that general laboratory automation will go faster with LLM assistance, even if all the LLM does is write Python glue scripts to enable custom workflows and instrument integrations.
- 16d ago
- throwup238 16d ago> daily cancer scans instead of one every few years Silly nitpick: the reason we don’t do daily cancer scans isn’t the cost, it’s the false positive rate. Invasive procedures like biopsies come with complications like infections that happen at a higher rate and do more damage than the cancer that doesn’t even exist. This dilemma is pervasive in medicine, because our tests aren’t perfect but the thing they’re testing for is rare.
- sacred_numbers 16d agoThe whole paradigm changes, though, when you can do daily cancer scans. You don't get a biopsy when the scan shows a lump. You get a biopsy after a couple weeks of daily scans showing the lump growing. Plus, having all the data from the daily scans improves your testing accuracy so false positives and negatives are more rare.
- throwup238 16d agoThe errors are correlated, not random. Lumps are usually benign cysts. If you start cutting people open for every cyst on a scan, you’d kill a lot more people than you’d save. This isn’t something you can solve with more scans because the tests test for data that is indistinguishable. They look the same on a scan, there’s an overlap in the assay with some random protein with the same binding sites that is only present in 1% of the population, the coding gene in one person gets repeated in a noncoding region in another, and so on. The “more data” that works is a doctor applying professional judgement (which they’re also famously bad at because biology is a fickle mistress).
- mhluongo 16d agoMore data (daily scans) can mean we get better at medicine, though, and more accurate. You're assuming "daily cancer scans" look just like they do today, rather than eg unobtrusive devices in our daily lives that measure changes over time. The frequent "muh false positives" comment we hear from doctors appears to be a lack of imagination?
- done_lurking 16d agoWouldn't daily cancer scans give you cancer from all the radiation? I thought that's why the doc hides behind a wall when giving an X-ray.