10 ms·
Accelerating scientific breakthroughs with an AI co-scientist
- Jimmc414 2y agopaper: https://storage.googleapis.com/coscientist_paper/ai_coscientist.pdf https://storage.googleapis.com/coscientist_paper/ai_coscient...
- mnky9800n 2y agoTbh I don’t see why I would use this. I don’t need an ai to connect across ideas or come up with new hypothesis. I need it to write lots of data pipeline code to take data that is organized by project, each in a unique way, each with its own set of multimodal data plus metadata all stored in long form documents with no regular formatting, and normalize it all into a giant database. I need it to write and test a data pipeline to detect events both in amplitude space and frequency space in acoustic data. I need it to test out front ends for these data analysis backends so i can play with the data. Like I think this is domain specific. Probably drug discovery requires testing tons of variables one by one iterating through the values available. But that’s not true for my research. But not everything is for everybody and that’s okay.
- not_kurt_godel 2y agoAgreed - AI that could take care of this sort of cross-system complexity and automation in a reliable way would be actually useful. Unfortunately I've yet to use an AI that can reliably handle even moderately complex text parsing in a single file more easily than if I'd just done it myself from the start.
- mnky9800n 2y agoYes. It’s very frustrating. Like there is a great need for a kind of data pipeline test suite where you can iterate through lots of different options and play around with different data manipulations so a single person can do it. Because it’s not worth it to really build it if it doesn’t work. There needs to be one of these astronomer/dagster/apache airflow/azure ml tools that are quick and dirty to try things out. Maybe I’m just naive and they exist and I’ve had my nose in Jupyter notebooks. But I really feel hindered these days in my ability to prototype complex data pipelines myself while also considering all of the other parts of the science.
- knowaveragejoe 2y agoThis reminds me of a paper: "The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators" https://arxiv.org/abs/2407.11004 https://arxiv.org/abs/2407.11004 In essence, LLMs are quite good at writing the code to properly parse large amounts of unstructured text, rather than what a lot of people seem to be doing which is just shoveling data into an LLM's API and asking for transformations back.
- SubiculumCode 2y agoThe doing tends to be the hard part. Every scientist has 1000 idea for every one they get a chance to pursue. That said, I requested early access.
- deleted 2y ago[deleted]
- parineum 2y ago> I don’t need an ai to connect across ideas or come up with new hypothesis. This feels like hubris to me. The idea here isn't to assist you with menial tasks, the idea is to give you an AI generalist that might ne able to alert you to things outside of your field that may be related to your work. It's not going to reduce your workload, in fact, it'll probably increase it but the result should be better science. I have a lot more faith in this use of LLMs than I do for it to do actual work. This would just guide you to speak with another expert in a different field and then you take it from there. > In many fields, this presents a breadth and depth conundrum, since it is challenging to navigate the rapid growth in the rate of scientific publications while integrating insights from unfamiliar domains.
- anothermathbozo 2y agoImagine someone can do the things you can’t do and needs help doing the things you can.
- tippytippytango 2y agoExactly, they want to automate the most rewarding part that we don’t need help with… plus I don’t believe they’ve solved the problem of LLMs generating trite ideas.
- trilobyte 2y agoSounds like the message artists were giving when generative AI started blowing up.
- Der_Einzige 2y agoThis ludditism shit is the death drive externalized. You'd forsake an amazing future based on copes like the precautionary principle or worse yet, a belief that work is good and people must be forced into it. The tears of butthurt scientists, or artists who are automated out of existence because they refused to leverage or use AI systems to enhance themselves will be delicious. The only reason that these companies aren't infinitely better than what Aaron Swartz tried to do was that they haven't open accessed everything. Deepseek is pretty close (sans the exact dataset), and so is Mistral and apparently Meta? Y'all talked real big about loving "actual" communism until it came for your intellectual property, now you all act like copyright trolls. Fuck that!
- mnky9800n 2y agoYou sure are argumentative for someone who believes they are so correct. In any case, I don’t think I’m a Luddite. I use many ai tools in my research including for idea generation. So far i have not found it to be very useful. Moreover the things it could be useful for such as automated data pipeline generation it doesn’t do. I could imagine a series of agents where one designs pipelines and one fills in the codes, etc per node in the pipeline but so far I didn’t see anything like that. If you have some kind of constructive recommendations in that direction I’m happy to hear them.
- YeGoblynQueenne 2y agoAll very valid criticism. My question is, when are you going to make it work?
- eamag 2y agoI think you're just not the target audience. If AI can come up with some good ideas and then split it into tasks some of them an undergrad can do - it can speed up the global research speed by involving more people in useful science
- coliveira 2y agoIn science, having ideas is not the limiting factor. They're just automating the wrong thing. I want to have ideas and ask the machine to test for me, not the other way around.
- eamag 2y agoThe difference is the complexity of ideas. There are straightforward ideas anyone can test and improve, and there are ideas where only PhDs in CERN can test
- JW_00000 2y agoI don't think that's really right. E.g. what makes finding the Higgs boson difficult is that you need to build a really large collider, not coming up with the idea, which could be done 50 years earlier. Admittedly the Higgs boson is still a "complex idea", but the bottleneck still was the actual testing.
- webmaven 2y agoIf I understand what's been published about this, it isn't just ideation, but also critiquing and ranking them, to select the few most worth pursuing. Choosing a hypothesis to test is actually a hard problem, and one that a lot of humans do poorly, with significant impact on their subsequent career. From what I have seen as an outsider to academia, many of the people who choose good hypotheses for their dissertation describe it as having been lucky.
- kuhewa 2y agoI bet all of these researchers involved had a long list of candidates they'd like to test and have a very good idea what the lowest hanging fruit are, sometimes for more interesting reasons than 'it was used successfully as an inhibitor for X and hasn't been tried yet in this context' — not that that isn't a perfectly good reason. I don't think ideas are the limiting factor. The reason attention was paid to this particular candidate is because google put money down.
- deleted 2y ago[deleted]
- crypto420 2y agoI'm not sure if people here even read the entirety of the article. From the article: > We applied the AI co-scientist to assist with the prediction of drug repurposing opportunities and, with our partners, validated predictions through computational biology, expert clinician feedback, and in vitro experiments. > Notably, the AI co-scientist proposed novel repurposing candidates for acute myeloid leukemia (AML). Subsequent experiments validated these proposals, confirming that the suggested drugs inhibit tumor viability at clinically relevant concentrations in multiple AML cell lines. and, > For this test, expert researchers instructed the AI co-scientist to explore a topic that had already been subject to novel discovery in their group, but had not yet been revealed in the public domain, namely, to explain how capsid-forming phage-inducible chromosomal islands (cf-PICIs) exist across multiple bacterial species. The AI co-scientist system independently proposed that cf-PICIs interact with diverse phage tails to expand their host range. This in silico discovery, which had been experimentally validated in the original novel laboratory experiments performed prior to use of the AI co-scientist system, are described in co-timed manuscripts (1, 2) with our collaborators at the Fleming Initiative and Imperial College London. This illustrates the value of the AI co-scientist system as an assistive technology, as it was able to leverage decades of research comprising all prior open access literature on this topic. The model was able to come up with new scientific hypotheses that were tested to be correct in the lab, which is quite significant.
- preston4tw 2y agoThis is one thing I've been wondering about AI: will its broad training enable it to uncover previously covered connections between areas the way multi-disciplinary people tend to, or will it still miss them because it's still limited to its training corpus and can't really infer. If it ends up being more the case that AI can help us discover new stuff, that's very optimistic.
- semi-extrinsic 2y agoIn some sense, AI should be the most capable at doing this within math. Literally the entire domain in its entirety can be tokenized. There are no experiments required to verify anything, just theorem-lemma-proof ad nauseam. Doing this like in this test, it's very tricky to rule out the hypothesis that the AI is just combining statements from the Discussion / Future Outlook sections of some previous work in the field.
- ACV001 2y agoJust as the invention of writing degraded human memory (before that they memorized whole stories, poems), with the advent of AI, humans will degrade their thinking skills and knowledge in general.
- deleted 2y ago[deleted]
- azinman2 2y agoIt seems in general we’re heading toward’s Minsky’s society of minds concept. I know OpenAI is wanting to collapse all their models into a single omni model that can do it all, but I wonder if under the hood it’d just be about routing. It’d make sense to me for agents to specialize in certain tool calls, ways of thinking, etc that as a conceptual framework/scaffolding provides a useful direction.
- yjftsjthsd-h 2y agoIsn't that kinda the idea of Mixture of Experts?
- mythrwy 2y agoI wonder if OpenAI might be routing already based on speed of some "O1" responses I receive. It does make sense.
- willy_k 2y agoAlso, for some more complex questions I’ve noticed that it doesn’t expose its reasoning. Specifically, yesterday I asked it to perform a search algorithm provided a picture of a grid, and it reasoned for 1-2 minutes but didn’t show any of it (neither in real time nor afterwords), whereas for simpler questions I’ve asked it the reasoning is provided as well. Not sure what this means, but it suggests some type of different treatment based on complexity.
- funnyAI 2y ago"conceptual framework" can actually be another generalist model. Splitting model also comes with some advantages. Like easy separate tuning and replacements. Easy scaling by simply duplicating heavily used model on new hardware.
- ThouYS 2y agoI guess we do live in the fast take off world
- celltalk 2y ago“Drug repurposing for AML” lol As a person who is literally doing his PhD on AML by implementing molecular subtyping, and ex-vivo drug predictions. I find this super random. I would truly suggest our pipeline instead of random drug repurposing :) https://celvox.co/solutions/seAMLess https://celvox.co/solutions/seAMLess edit: Btw we’re looking for ways to fund/commercialize our pipeline. You could contact us through the site if you’re interested!
- heyoni 2y agoCan you explain what you mean by subtyping and if/how it negates the usefulness of repurposing (if that’s what you meant to say). Wouldn’t subtyping complement a drug repurposing screen by allowing the scientist to test compounds against a subset of a disease? And drug repurposing is also used for conditions with no known molecular basis like autism. You’re not suggesting its usefulness is limited in those cases right?
- celltalk 2y agoSure. There are studies like BEAT-AML which tests selected drugs’ responses on primary AML material. So, not on a cell-line but on true patient data. Combining this information with molecular measurements, you can actually say something about which drugs would be useful for a subset of the patients. However, this is still not how you treat a patient. There are standard practices in the clinic. Usually the first line treatment is induction chemo with hypomethylating agents (except elderly who might not be eligible for such a treatment). Otherwise the options are still very limited, the “best” drug in the field so far is a drug called Venetoclax, but more things are coming up such as immuno-therapy etc. It’s a very complex domain, so drug repurposing on an AML cell line is not a wow moment for me.
- ttpphd 2y agoIt's almost like scientists are doing something more than a random search over language.
- celltalk 2y ago
- waynenilsen 2y agoit seems that humans may become the hands of the AI before the robots are ready mechanical turk, but for biology
- akomtu 2y agoThat's the Quake version of the machine civilization: machines make the decisions, but use chunks of humans to improve their unholy machinery. The alternative Doom version is the opposite: humans make the decisions, but they are blended in an unholy way into the machines.
- tsumnia 2y agoNow do the Warhammer 40k version :D
- quinnjh 2y agoThe market seems excited to charge in whatever direction the weathervane has last been pointing, regardless of the real outcomes of running in that direction. Hopefully I’m wrong, but it reminds me very much of this study (I’ll quote a paraphrase) “A groundbreaking new study of over 1,000 scientists at a major U.S. materials science firm reveals a disturbing paradox: When paired with AI systems, top researchers become extraordinarily more productive – and extraordinarily less satisfied with their work. The numbers tell a stark story: AI assistance helped scientists discover 44% more materials and increased patent filings by 39%. But here's the twist: 82% of these same scientists reported feeling less fulfilled in their jobs.” Quote from https://futureofbeinghuman.com/p/is-ai-poised-to-suck-the-soul-out-of-science https://futureofbeinghuman.com/p/is-ai-poised-to-suck-the-so... Referencing this study https://aidantr.github.io/files/AI_innovation.pdf https://aidantr.github.io/files/AI_innovation.pdf
- yodon 2y agoAs a dev, I have the same experience. AI chat is a massive productivity enhancer, but, when coding via prompts, I'm not able to hit the super satisfying developer flow state that I get into via normal coding. Copilot is less of a productivity boost, but also less of a flow state blocker.
- sanderjd 2y agoYep! I think these tools are incredibly useful, but I think they're basically changing all our jobs to be more like what product managers do, having ideas for what we want to achieve, but farming out a significant chunk of the work rather than doing it ourselves. And that's fine, I find it very hard to argue that it's a bad thing. But there's a reason that we aren't all product managers already. Programming is fun, and I do experience it as a loss to find myself doing less of it myself.
- pradn 2y agoThere is some queasy feeling of fake-ness when auto-completing so much code. It feels like you're doing something wrong. But these are all based on my experience coding for half my life. AI-native devs will probably feel differently.
- m3kw9 2y agoI really would like to see a genuine breakthru amongst all this talk about AI doing that
- stanford_labrat 2y agoSo I'm a biomedical scientist (in training I suppose...I'm in my 3rd year of a Genetics PhD) and I have seen this trend a couple of times now where AI developers tout that AI will accelerate biomedical discovery through a very specific argument that AI will be smarter and generate better hypotheses than humans. For example in this Google essay they make the claim that CRISPR was a transdisciplinary endeavor, "which combined expertise ranging from microbiology to genetics to molecular biology" and this is the basis of their argument that an AI co-scientist will be better able to integrate multiple fields at once to generate novel and better hypothesis. For one, what they fail to understand as computer scientists (I suspect due to not being intimately familiar with biomedical research) is that microbio/genetics/mol bio are closer linked than you may expect as a lay person. There is no large leap between microbiology and genetics that would slow down someone like Doudna or even myself - I use techniques from multiple domains in my daily work. These all fall under the general broad domain of what I'll call "cellular/micro biology". As another example, Dario Amodei from Claude also wrote something similar in his essay Machines of Loving Grace that the limiting factor in biomedical is a lack of "talented, creative researchers" for which AI could fill the gap[1]. The problem with both of these ideas is that they misunderstand the rate-limiting factor in biomedical research. Which to them is a lack of good ideas. And this is very much not the case. Biologists have tons of good ideas. The rate limiting step is testing all these good ideas with sufficient rigor to either continue exploring that particular hypothesis or whether to abandon the project for something else. From my own work, the hypothesis driving my thesis I came up with over the course of a month or two. The actual amount of work prescribed by my thesis committee to fully explore whether or not it was correct? 3 years or so worth of work. Good ideas are cheap in this field. Overall I think these views stem from field specific nuances that don't necessarily translate. I'm not a computer scientist, but I imagine that in computer science the rate limiting factor is not actually testing out hypothesis but generating good ones. It's not like the code you write will take multiple months to run before you get an answer to your question (maybe it will? I'm not educated enough about this to make a hard claim. In biology, it is very common for one experiment to take multiple months before you know the answer to your question or even if the experiment failed and you have to do it again). But happy to hear from a CS PhD or researcher about this. All this being said I am a big fan of AI. I try and use ChatGPT all the time, I ask it research questions, ask it to search the literature and summarize findings, etc. I even used it literally yesterday to make a deep dive into a somewhat unfamiliar branch of developmental biology more easy (and I was very satisfied with the result). But for scientific design, hypothesis generation? At the moment, useless. AI and other LLMs at this point are a very powerful version of google and code writer. And it's not even correct 30% of the time to boot so you have to be extremely careful when using it. I do think that wasting less time exploring hypotheses that are incorrect or bad is a good thing. But the problem here is that we can pretty easily identify good and bad hypotheses already. We don't need AI for that, what takes time is the actual amount of testing of these hypotheses that slows down research. Oh and politics, which I doubt AI can magic away for us. [1] https://darioamodei.com/machines-of-loving-grace#1-biology-and-health https://darioamodei.com/machines-of-loving-grace#1-biology-a...
- bjarlsson 2y agoThis is marketing material from Google and people are accepting the premises uncritically.
- anothermathbozo 2y agoAlmost this entire thread is criticism
- writeslowly 2y agoI recently ran across this toaster-in-dishwasher article [1] again and was disappointed that the LLMs I have access to could replicate the "hairdryer-in-aquarium" breakthrough (or the toaster-in-dishwasher scenario, although I haven't explored it as much), which has made me a bit skeptical of the ability of LLMs to do novel research. Maybe the new OpenAI research AI is smart enough to figure it out? [1] https://jdstillwater.blogspot.com/2012/05/i-put-toaster-in-dishwasher.html https://jdstillwater.blogspot.com/2012/05/i-put-toaster-in-d...
- BriggyDwiggs42 2y agoDo you mean they sided with the incorrect common wisdom all the people in the article were using?
- hinkley 2y agoI am generally down on AI these days but I still remember using Eliza for the first time. I think I could accept an AI prompting me instead of the other way around. Something to ask you a checklist of problems and how you will address them. I’d also love to have someone apply AI techniques to property based testing. The process of narrowing down from 2^32 inputs to six interesting ones works better if it’s faster.
- confused_boner 2y agoAI prompting us sounds interesting
- lorepieri 2y agoCheck Manna.
- insane_dreamer 2y agoSeems like the primary value-add is to speed up the literature review process during the hypothesis formulation process.
- deleted 2y ago[deleted]
- dr_kretyn 2y agoInteresting set of comments. Personally - fantastic! It's a co-scientist and not a "scientist". There's enormous value in reviewing work and ranking "what" might provide some interesting output. A lot of ideas are not even considered because they aren't "common" because their components are expensive. If there's a "reasonable expectation" then there's a lower risk of failure. I'm not a scientist "anymore" but I'd love to play with this and see what odd combinations might potentially produce.
- cratermoon 2y agomeanwhile: OpenAI Deep Research - Six Strange Failures: <https://futuresearch.ai/oaidr-feb-2025 https://futuresearch.ai/oaidr-feb-2025>
- curtisszmania 2y ago[dead]
- anu7df 2y agoOh great.. This looks exactly like some PhD advisors I've heard of. Creating a list of "ideas" and having their lab monkey PhD students work on it. Surefire way to kill joy of discovery and passion for science. Nice going google :). Also, while validating with "in silico" discovery I would like it to be double blind. If I know the idea and its final outcome the prompts I give are vastly different from if I did not.
- inglor_cz 2y agoLoosely connected: I read recently about an Austrian pilot project in Wien, where they analyze patients' tumors and let AI suggest treatment. They had some spectacular successes, where the AI would recommend drugs that aren't normally used for that type of cancer, but when deployed, worked well.
- pazimzadeh 2y agoI don't think generating hypotheses is where AI is useful, I think it's more useful for doing back of napkin (or more serious) calculations, helping to find protocols, sourcing literature, etc. Grunt work basically. Generating hypotheses is the fun, exciting part that I doubt scientists want to outsource to AI.
- inductive_magic 2y ago> I don't think generating hypotheses is where AI is useful, > Generating hypotheses is the fun, exciting part that I doubt scientists want to outsource to AI The latter doesn’t imply the former
- ZeroGravitas 2y agoI read the scientist quote in a newspaper article first and the surprise seemed to hinge on his entire team working on the problem for a decade and not publishing anything in a way that AI could gobble it up (which seemed strange to me) and no other human researcher working on the problem over that decade-ish timespan publishing anything suggesting the same idea. Which seems a hard thing to disprove. In which case, if some rival of his had done the same search a month earlier, could he have claimed the priority? And would the question of whether the idea had leaked then been a bit more salient to him. (Though it seems the decade of work might be the important bit, not the general idea).
- jjk166 2y agoThis is in line with how I've been using AI in my workflow recently. I give it a summary of my findings thus far and ask it to suggest explanations and recommend further tests I should conduct. About 70% of its ideas are dumb and sometimes I need to give it a little extra prompting, but it does spit out ideas that hadn't occurred to me which make sense. Obviously it's not going to replace a knowledgeable human, but as a tool to assist that human it has outperformed some very expensive PhD level consultants.