4 ms·
Designing AI for Disruptive Science
- vivid242 8mo agoI wasn’t aware of the map empire, thank you! Taking away some complexity comes at a price, and for some people, it’s hard to see that it outweighs the practicality.
- ortusdux 8mo agoReminds me of the coastline paradox - https://en.wikipedia.org/wiki/Coastline_paradox https://en.wikipedia.org/wiki/Coastline_paradox
- boulos 8mo agoPlease don't editorialize titles unless they're clearly clickbait. "Designing AI for Disruptive Science" is a bit market-ey, but "AI Risks 'Hypernormal' Science" is just a trimmed section heading "Current AI Training Risks Hypernormal Science".
- bananaflag 8mo agoI find it funny how people are so concerned that AI cannot innovate, that AI coding agents only give the most bland solutions to any problem etc. when the next step in OpenAI's 5 stages to AGI is literally called "Innovators".
- jacquesm 8mo agoIt's marketing.
- munk-a 8mo agoDo you mean to say my current AI workflow doesn't involve secret agents running around Bond-style sabotaging those that'd impede my efforts to build a super secret RSS forwarder that pig-lantinifies the text before sending it to my client?
- thegrim33 8mo agoMy two step plan is to go to sleep and then wake up the next day and be a billionaire. Surely because that's my stated next step that means when I wake up tomorrow I'll be rich.
- jacquesm 8mo agoAt no risk to myself I will try your plan and if it doesn't work you owe me your billion.
- cogman10 8mo agoThe article presumes that the models we have today describing everything could still be subject to a major paradigm shift. Maybe they could be, but it seems pretty unlikely. The edges of a lot of scientific understanding are now past practical applicability. The edges are essentially models of things impossible to test. In fact, relativity was only recently fully backed up with experimental data.
- tech_ken 8mo agoI don't think paradigm shifts have to be 'better' in some march-toward-progress sense, they can be lateral or even regressive in that way and still lead to longer-horizon improvements. I think also what's practically applicable changes constantly. Perhaps we're truly at the End of Science, but empirically we've been wrong every other time we've said that. My money is that there's more race to run.
- cogman10 8mo ago> I don't think paradigm shifts have to be 'better' But they do. Paradigm shifts happen because the new paradigm explains the unexplained and importantly also covers the old model. If prior data is unexplained with a paradigm shift, the shift will never be adopted. > Perhaps we're truly at the End of Science Who said that? Just because the core of our current models seem pretty rock steady doesn't mean there's not more science. It simply means that we can mostly just expect refining rather than radical discovery. There will be sub-paradigm shifts, but there's likely not going to be major "relativity" moments from here on out.
- paganel 8mo ago> It simply means that we can mostly just expect refining rather The practical issue is if there will enough funds for just "refining", instead of "paradigm shifts", which I understand as new and "exciting" discoveries. I'm not a scientist, of course, this is just my layman's understanding.
- tech_ken 8mo ago
- edwardsrobbie 8mo ago[flagged]
- tech_ken 8mo agoMy hot take is that mathematical and scientific 'soundness' is ultimately more of an aesthetic preference than an objective quality of reality. Good science makes sense to humans, and 'what makes sense' is ultimately what fits satisfyingly in your brain. There's nothing inherently wrong with an enormous epicycle model of reality from the perspective of the God of Math; so long as your formal system is consistent and expressive enough to represent everything then meh, it's a model. But the model that humans want to elevate to canonical status has far stricter requirements, and ultimately it's the one which the majority of sufficiently credentialed tastemakers decide is 'best'. Parsimony works well in physics where you have closed form expressions for all your stuff, but the biology cases are so much messier because it turns out that sometimes reality isn't parsimonious. All this to say that good science is a matter of taste, and while AI can gist the broad strokes of taste I've yet to see it take on the role of genuine tastemaker.
- ArRENCEAI 8mo agoWhat's more alarming isn't that AI is limited to existing domain data, it's that when people push it to deviate outside those known data points it confidently hallucinates nonsense.
- duskwuff 8mo agoSo much nonsense. If I had a nickel for every AI-poisoned "researcher" I'd seen with a preprint full of nonsense buzzwords like "quantum fractal holographic resonance matrix"... well, I wouldn't be rich, but I'd probably at least have enough to buy a coffee.
- underlipton 8mo agoAnd many of today's publicly-accessible platforms are designed to steer results away from nonsense back to... too much sense. "Hypernormal" is a good word for it. I've spent a lot of time prompt-yelling, "Please make something as weird as I'm imagining, stop veering back to what normal people want to see/read."
- Nevermark 8mo agoModels are not trained to self-evaluate. Or only as an afterthought during tuning. So they are poor at it. It isn’t mysterious. Humans are trained incrementally, educated informally and formally, and along the way tested by context and classroom. Evaluated by people in their circle and strangers. The training to evaluate ourselves is near constant. Even then, many people habitually believe they understand things they clearly don’t. And can even be hostile to feedback. Many forms of hallucination are canonized, or socially encouraged at varying demographic scales. While others are idiosyncratic. Once self-examination is a first-class part of models training process, I expect they will respond like they have to being trained on vast troves of information, which is far excel human beings. But, until then, their poor self-assessment isn’t mysterious, it is a mundane result of being trained not to do that. Or only as a tuning after thought. Not as different from humans as we might like to think. And models are noticeably improving on this measure. Humans in my experiencing may be regressing.
- piker 8mo agoI got stuck for a minute on the caption "Harry Beck’s 1933 map of the London Underground" to: https://substackcdn.com/image/fetch/$s_!VsWm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f4bd2cb-dcb6-46e1-b1fd-6078e95e35bd_5350x3746.jpeg https://substackcdn.com/image/fetch/$s_!VsWm!,f_auto,q_auto:... which contains Heathrow Terminals 1, 2, 3, 4 & 5 on the Picadilly line. For about 15 seconds I imagined a world where Heathrow has had 5 terminals since 1933, then I read the map itself: "Recreated by Arthurs D". Phew. Awesome example of improving information conveyance through abstractions though!
- drtgh 8mo ago> AI could repeat this pattern at a larger scale — generating faster results within the existing paradigm, while the structural conditions for disruptive science remain unchanged or worsen. Worsen. LLMs discard/loses and mixes data on their statistical "compression" to create their vectorial database model. Across the time, successive feed back will be homologous to create a jpg image sourcing a jpg image that was created from another jpg image, through this "gaussian" loop. Those faster (but worst) results will degrade real valuable data and science at a speed/rate that will statistically discard good done science on a regular basis, systematically. IMHO.
- whattheheckheck 8mo agoCan you explain further how we can prevent this
- bibimsz 8mo agoThe piano accompaniment and human narration are a nice touch.
- jikojkb 8mo ago[flagged]
- liuliu 8mo agoI don't think this argument is wrong. But also debatable. At the end of the day, we are talking about the manifold of the reality (as compressed by LLM through language abstraction). It is remain to be seen if supervised fine-tuning on the best human can produce would nag the model enough to generate surprising findings. We know the pre-trained models do tend to revert to mean, but I don't think that's enough to say SFT / RL models will do the same, although some might argue RL only sharpens the distribution, even for that, I am skeptical about that paper.
- irdc 8mo agoIt's been shown in other fields that training models on the output of other models produces subtly broken models, not a flattening to the statistical mean. Why would science be different?
- deleted 8mo ago[deleted]
- pevansgreenwood 8mo ago[dead]
- OhMeadhbh 8mo agomeh. I would be happier with this article if it demonstrated familiarity with the source material. "Del Rigor en la Ciencia" (On Exactitude in Science) was Borges (hilarious) investigation into Korzybski's General Semantics, a fact that was surprisingly absent from the text. Borges implied "the map is not the territory," but Korzybski actually came out in said it a decade or so before Borges wrote the story in question. Understanding the themes of Borges story is greatly informed by a passing familiarity with Korzybski's work. WorldCat tells me the 4th edition of "Science and Sanity" is widely held by libraries, and if you're interested in the assertions made in this article, you might enjoy reading (at least parts of) it. https://search.worldcat.org/title/369632 https://search.worldcat.org/title/369632 The author completely missed the point Borges (and Korzybski) made about the utility of maps. Maps (according to both) are abstractions which allow the user to ignore irrelevant aspects of reality so other, more interesting facets come into sharper resolve. This might be why Beck's London Tube map is so well regarded. It allows the user to easily ignore aspects that are not germane to the task of deciding where and when to get on and off the tube. But is a scientific paradigm like a map? Certainly it is an abstraction, if we take Kuhn's definition. If you're interested, I can recommend both "The Structure of Scientific Revolutions" and "The Essential Tension : Selected Studies in Scientific Tradition and Change" by Kuhn. https://search.worldcat.org/title/4660423077 https://search.worldcat.org/title/4660423077 https://search.worldcat.org/title/3034084 https://search.worldcat.org/title/3034084 Calling scientific paradigms maps isn't wrong, per se, but it does create more of a meta-metaphor, and a weak one at that. Also. No. Maxwell did not replace a patchwork of equations with four short ones. That was Heaviside. https://en.wikipedia.org/wiki/Oliver_Heaviside https://en.wikipedia.org/wiki/Oliver_Heaviside Something we don't mention in polite society these days is that Maxwell proposed electromagnetic waves as propagating through an aether: https://en.wikisource.org/wiki/A_Treatise_on_Electricity_and_Magnetism/Part_IV/Chapter_XX https://en.wikisource.org/wiki/A_Treatise_on_Electricity_and... If you're going to talk about new paradigms, Maxwell is a great example, but his story is not complete without mentioning Heaviside, Michelson and Morley. Also... I bristle at the phrase "Hypernormal Science." It's also introduced without definition or reference. Collins, et al describe it as distinct from (though seemingly related to) the word "Hypernormal" as coined by Yurchak in "Everything Was Forever Until It Was No More." https://direct.mit.edu/posc/article-abstract/31/2/262/112751/Hypernormal-Science-and-its-Significance https://direct.mit.edu/posc/article-abstract/31/2/262/112751... https://search.worldcat.org/title/1572419463 https://search.worldcat.org/title/1572419463 Or if you're short on time, you can get an entertaining (though not as enlightening) description from Adam Curtis' 2016 documentary HyperNormalization. You won't come away from it with a better understanding of AI, General Semantics or Popperian falsifiability, but it has a striking visual style and a very good soundtrack. And may lead to a better understanding of "hypernormal science." https://youtu.be/u1Tp-ryQPFI https://youtu.be/u1Tp-ryQPFI And getting back to the Michelson-Morley experiment. The author talks about how their results did not cause the scientific establishment to abandon the concept of luminiferous aether. Certainly there is conservatism in science. Gigging science-monkeys tend to want to see interesting results replicated. And this was one of the issues with the MM experiment. It took a while to replicate. We're MUCH better at replicating it these days and I would guess that thousands (maybe hundreds) of physics undergrads did this very task last year. But we've had over a century of pedagogical experience w/ this experiment. We know how to structure it to get the results we want. This was not the case in the late 1800s and in fact, several early attempts to replicate the experiment suggested the existence of an aether which was drifting slowly towards Cleveland. And what does it say that heat flow, fluid flow, diffusion and electrostatics share equations? Does it say there's something fundamental in reality? Or does it say there's something fundamental in the way we model reality? That being said... I think the author has hit upon something here... people are often wary of evidence which contradicts experience, even when that evidence (and not experience) is more correct. But each of the examples he provides glosses over the process by which new paradigms overrode the old. I deeply appreciate the author avoiding slavish fealty to fashionable AI trends. He probably could have gone further to describe more representational weakness of ESM3 and GNoME. I fear, however, he has missed the point. It's less interesting to describe the messy ways in which AI fails than to describe the messy ways in which humans succeed. The process by which paradigms shift is messy, social and fundamentally human. It often has more to do with qualitative explanations than quantitative science. Science, as a human endeavor, is very much a story-telling exercise.
- pu_pe 8mo agoAs another commenter mentioned, the point of the story from Borges is that a perfectly detailed map is rather useless, because you need abstraction (it's a repeating theme in some other stories from him like the Library of Babel, and Funes the Memorious). LLMs are likely already able to exhaust the conceptual space for any given field, but some judgement is still going to be required about what to pursue. In biology and other fields this problem is even bigger because experimentation is so difficult and expensive. The process of judgement and resource allocation will still be human for quite a while, but it's quite likely some humans will outsource their responsibility to AI to cut corners.