4 ms·

Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analy

Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collaborators. This could stimulate interdisciplinary work and foster innovation at the boundaries between fields. LBD systems can also identify “blind spots” in a given field, and even predict future discoveries and who will make them. I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? Does that do anything for us? > The second area is “robot scientists”, also known as “self-driving labs”. These are robotic systems that use AI to form new hypotheses, based on analysis of existing data and literature, and then test those hypotheses by performing hundreds or thousands of experiments, in fields including systems biology and materials science. Unlike human scientists, robots are less attached to previous results, less driven by bias—and, crucially, easy to replicate. They could scale up experimental research, develop unexpected theories and explore avenues that human investigators might not have considered. This sounds basically of just asking a GPT to suggest ideas for experiments? Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster. EDIT: Relevant XKCD https://xkcd.com/2341/ https://xkcd.com/2341/

jdeaton 3y ago
> This sounds basically of just asking a GPT to suggest ideas for experiments? I think actually the second idea extends quite a bit further than that. The idea is that you enable the agent to interact with an entire collection of laboratory equipment through tool-use. The agent not only generates hypotheses and designs experiments to test them, but then also actually executes the experiments through tool-use and iterates.
- rozal 3y ago
  [dead]
m3kw9 3y ago
Game developers should be hired to make an UI that allows scientists to accept “quests” and will show the public who accepted what and is collaborating with who on which problems with rankings. The reason to use game is to make it more engaging for everyone and also an incentive to go thru with it when in public
- gustavus 3y ago
  I'm now getting an idea for a dystopian cyberpunk novel about a world in which everyone is addicted to an awesome game that they think they are obsessed with that provides gift cards and credits that allow you to purchase real things, but it turns out it's all actually secretly controlled by a massive shadow government/corporation and everything everyone thinks they are doing in the "game" is actually having massive real world impacts.... Hey hollywood if your writers are still on strike HMU.
  - pixl97 3y ago
    Enders Capitalism?
  - samr71 3y ago
    Just formalize it and make it a corporation. Perhaps companies should be designed explicitly as games these days.
vharuck 3y ago
>Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster. This matches what I was told earlier this year when looking into what LLMs could do. I work in public health, and asked staff from a large registry what they'd like to me try. I expected something like generating reports from aggregate data (with some of that automatic exploration mentioned in the article). What they really wanted was: 1. A nice chat bot to answer questions from data submitters about reporting policies. 2. A tool to ingest federal grant announcements and filter down to those the registry could apply for. #1 is useful because that's a lot of their job. Getting people to send accurate data in the correct format on time is exhausting. #2 helps with everything, because it could mean hiring an additional staff member. That extra person can write reports and apply for grants, clean data, answer calls, and cover when somebody's out sick. Humans are still really useful.
eli_gottlieb 3y ago
>The second area is “robot scientists”, also known as “self-driving labs”. These are robotic systems that use AI to form new hypotheses, based on analysis of existing data and literature, and then test those hypotheses by performing hundreds or thousands of experiments, in fields including systems biology and materials science. Unlike human scientists, robots are less attached to previous results, less driven by bias—and, crucially, easy to replicate. This is half a real insight and half total bullshit. Honestly, what we need robotics and AI for in laboratory science is just old-fashioned standardization and labor-saving.
RandomLensman 3y ago
Both things sound like massive false positive generators when used at scale. Also, where do those robots come from that can do all these experiments beyond "trivial" setups? The "robot scientists" sound very sci-fi outside of standardized high volume automation.
- randcraw 3y ago
  Yeah, I've found that generating (scientific) hypotheses is easy. But generating more than combinatoric variants of existing hypotheses is hard, and harder still is to propose an insightful interpretation of results or a creative model for a mechanism of action that better explains the observed outcomes. I see nothing in today's 'deep AI' that addresses these desiderata. And I don't see the current AI strategy of accumulating only existing knowledge as the means to those ends either. Optimization of learning can only learn more facts or do it faster, not think with more creativity or innovation. Memory is but a small part of genius.
deleted 3y ago
[deleted]
ben_w 3y ago
> I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? Does that do anything for us? I think the problems with replication are a separate axis to the problem that there is too much being published for anyone to actually read. AI in general — never mind LLMs, even the much ones running search engines — help with the content overload. > This sounds basically of just asking a GPT to suggest ideas for experiments? I think the car analogy here is that if what you've suggesting was google maps, what the article is suggesting is Level 5 autonomy with no steering wheel and an opaque wall instead of a windscreen. I have absolutely no idea how hard such a level of automation might be to actually implement, not least because of Moravec's paradox: https://en.wikipedia.org/wiki/Moravec's_paradox https://en.wikipedia.org/wiki/Moravec's_paradox
pocw 3y ago
I've seen a scientific researcher do literature review manually. Search, refine, print, read, highlight, collate. Lots of time can be saved by automating those steps (and many researchers don't enjoy it so their job satisfaction could be increased). Also the resulting output could be improved if the researcher had a well structured summary to use as the foundation of their outline. Improve search with semantic search (search by concept not keyword) Improve refinement by preprocessing and summarizing Don't print, display clean and concise data. Summarize, cite and display. https://studyrecon.ai https://studyrecon.ai This stops short of literature based discovery, you have to bring your own research question. We've also had some luck finding a gap in existing research. We did a POC where we scraped pubmed and graphed study results by concept. We then used the graphed concepts to explore the conceptual space. It seems that vitamin D protects against cancer and heart disease. It seems that vitamin D supplementation protects against cancer but not heart disease. Is this because of some previously unknown effect of sun exposure (the primary natural source of vitamin D) or is it just that people with adequate vitamin D go outside a lot more and therefore also get more exercise? Don't know, would love to read the paper if someone studies it ;- )
amelius 3y ago
> Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster. Yeah, and then the funding agencies summarize the applications by running them through an LLM.
c7b 3y ago
> I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? It might be worse than that, not only might ChatGPT uncritically accept authors' claims or reduce the effort to produce low-quality research - given the current state of the art, there seems to be no guarantee that LLMs won't claim things about a paper that was never even said.

jdeaton 3y ago

> This sounds basically of just asking a GPT to suggest ideas for experiments? I think actually the second idea extends quite a bit further than that. The idea is that you enable the agent to interact with an entire collection of laboratory equipment through tool-use. The agent not only generates hypotheses and designs experiments to test them, but then also actually executes the experiments through tool-use and iterates.

rozal 3y ago
[dead]

m3kw9 3y ago

Game developers should be hired to make an UI that allows scientists to accept “quests” and will show the public who accepted what and is collaborating with who on which problems with rankings. The reason to use game is to make it more engaging for everyone and also an incentive to go thru with it when in public

gustavus 3y ago
I'm now getting an idea for a dystopian cyberpunk novel about a world in which everyone is addicted to an awesome game that they think they are obsessed with that provides gift cards and credits that allow you to purchase real things, but it turns out it's all actually secretly controlled by a massive shadow government/corporation and everything everyone thinks they are doing in the "game" is actually having massive real world impacts.... Hey hollywood if your writers are still on strike HMU.
- pixl97 3y ago
  Enders Capitalism?
- samr71 3y ago
  Just formalize it and make it a corporation. Perhaps companies should be designed explicitly as games these days.

vharuck 3y ago

>Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster. This matches what I was told earlier this year when looking into what LLMs could do. I work in public health, and asked staff from a large registry what they'd like to me try. I expected something like generating reports from aggregate data (with some of that automatic exploration mentioned in the article). What they really wanted was: 1. A nice chat bot to answer questions from data submitters about reporting policies. 2. A tool to ingest federal grant announcements and filter down to those the registry could apply for. #1 is useful because that's a lot of their job. Getting people to send accurate data in the correct format on time is exhausting. #2 helps with everything, because it could mean hiring an additional staff member. That extra person can write reports and apply for grants, clean data, answer calls, and cover when somebody's out sick. Humans are still really useful.

eli_gottlieb 3y ago

>The second area is “robot scientists”, also known as “self-driving labs”. These are robotic systems that use AI to form new hypotheses, based on analysis of existing data and literature, and then test those hypotheses by performing hundreds or thousands of experiments, in fields including systems biology and materials science. Unlike human scientists, robots are less attached to previous results, less driven by bias—and, crucially, easy to replicate. This is half a real insight and half total bullshit. Honestly, what we need robotics and AI for in laboratory science is just old-fashioned standardization and labor-saving.

RandomLensman 3y ago

Both things sound like massive false positive generators when used at scale. Also, where do those robots come from that can do all these experiments beyond "trivial" setups? The "robot scientists" sound very sci-fi outside of standardized high volume automation.

randcraw 3y ago
Yeah, I've found that generating (scientific) hypotheses is easy. But generating more than combinatoric variants of existing hypotheses is hard, and harder still is to propose an insightful interpretation of results or a creative model for a mechanism of action that better explains the observed outcomes. I see nothing in today's 'deep AI' that addresses these desiderata. And I don't see the current AI strategy of accumulating only existing knowledge as the means to those ends either. Optimization of learning can only learn more facts or do it faster, not think with more creativity or innovation. Memory is but a small part of genius.

deleted 3y ago

[deleted]

ben_w 3y ago

> I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? Does that do anything for us? I think the problems with replication are a separate axis to the problem that there is too much being published for anyone to actually read. AI in general — never mind LLMs, even the much ones running search engines — help with the content overload. > This sounds basically of just asking a GPT to suggest ideas for experiments? I think the car analogy here is that if what you've suggesting was google maps, what the article is suggesting is Level 5 autonomy with no steering wheel and an opaque wall instead of a windscreen. I have absolutely no idea how hard such a level of automation might be to actually implement, not least because of Moravec's paradox: https://en.wikipedia.org/wiki/Moravec's_paradox https://en.wikipedia.org/wiki/Moravec's_paradox

pocw 3y ago

I've seen a scientific researcher do literature review manually. Search, refine, print, read, highlight, collate. Lots of time can be saved by automating those steps (and many researchers don't enjoy it so their job satisfaction could be increased). Also the resulting output could be improved if the researcher had a well structured summary to use as the foundation of their outline. Improve search with semantic search (search by concept not keyword) Improve refinement by preprocessing and summarizing Don't print, display clean and concise data. Summarize, cite and display. https://studyrecon.ai https://studyrecon.ai This stops short of literature based discovery, you have to bring your own research question. We've also had some luck finding a gap in existing research. We did a POC where we scraped pubmed and graphed study results by concept. We then used the graphed concepts to explore the conceptual space. It seems that vitamin D protects against cancer and heart disease. It seems that vitamin D supplementation protects against cancer but not heart disease. Is this because of some previously unknown effect of sun exposure (the primary natural source of vitamin D) or is it just that people with adequate vitamin D go outside a lot more and therefore also get more exercise? Don't know, would love to read the paper if someone studies it ;- )

amelius 3y ago

> Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster. Yeah, and then the funding agencies summarize the applications by running them through an LLM.

c7b 3y ago

> I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? It might be worse than that, not only might ChatGPT uncritically accept authors' claims or reduce the effort to produce low-quality research - given the current state of the art, there seems to be no guarantee that LLMs won't claim things about a paper that was never even said.