6 ms·

The Humans Building AI Scientists

71 points by surprisetalk 2y ago

ludicrousdispla 2y ago
>> I’m optimistic that an AI scientist will help with reproducibility overall. Did you do the experiment that you said you did? Did you record all the variables in a way that you can report it in the way you did it? Interesting to think of the potential long term impact for science. Reminds me of the reform in the early 20th century that focused on ensuring the contents of canned goods matched their labeled ingredients.
- analog31 2y ago
  That's still a thing, in the drug industry.
plaidfuji 2y ago
> Rodriques: Many people assume we’re focused on wet lab automation. There are certainly opportunities there and we are exploring them, but the biggest opportunities are actually on the cognitive side. Wet lab automation is very difficult and capital intensive. And once you build your lab you are constraining yourself to answering questions within a certain domain for which you have the relevant sample prep and characterization equipment. Your equipment in essence defines your design space, and thus your potential solution space, which places a bound on your TAM. So of course automating the thinking part of science is more approachable with current AI - but is that what people want? It’s certainly an attractive proposition for management: automate away the highly-paid SMEs and turn R&D into more of a factory environment with replaceable lab techs.. but actually implementing this depends on where the power lies in an org. My theory is that in many orgs the “cognitive” folks hold the true power (= the unwritten expertise about what works and what doesn’t, when to work around your existing setup, how much to trust each number an instrument produces). They’ll resist this change to their last breath. You may gain some short-term efficiency by accelerating the experiments of today, but in the long run you lose the expertise to break out of local minima imposed by your equipment and training data. Or to think about this another way, imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful?
- deleted 2y ago
  [deleted]
- owenpalmer 2y ago
  > imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful? A lot of scientific knowledge cannot be communicated through papers. This is especially true in wet labs, where there's no procedural standardization. Keoni Gandall wrote an excellent post on this topic as it applies to synthetic bio [0]. I've experienced this first hand as a student participating in chemistry labs. Even when you're given a step by step procedure, it's impossible to predict the spacial logistics and inefficiencies you'll run into when you actually try to execute the procedure, regardless of your analytical preparation. The other type of knowledge that is rarely communicated through papers is the informal exploratory thought process of the researcher, and their embarrassing failures/mistakes. If I have my own lab someday, I think it would be cool if everyone wore bodycams, showing their first person view. By publishing the raw footage with the paper/code, hopefully this would help with reproducibility. [0] https://keonigandall.com/posts/too_much_tacit.html https://keonigandall.com/posts/too_much_tacit.html
- the_snooze 2y ago
  >Or to think about this another way, imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful? For anyone who thinks this is sufficient, be aware that papers only tell you what's successful (for some definition of successful). Being part of a research community gives you access to the other side: hallway conversations at conferences and informal collaborative networks are far more candid, where people will tell you what they've tried and didn't work, or what resources they need for an ambitious study that's just out of reach for their current budget. This is also where a lot of new ideas and collaborations come from, where people come together with matching problems and solutions to new interesting questions. I'm not sure how an AI is supposed to help with this, as research is ultimately a very social activity from my perspective.
light_hue_1 2y ago
I'm so incredibly tired of all of the BS claims. (I'm an AI/ML researcher) > has enabled open-source LLMs “to exceed human-level performance on two more of the lab-bench tasks: doing scientific literature research and reasoning about DNA constructs” with only “modest compute budgets.” No. They did not. They just ran a crappy experiment and came up with an absurd result. As a community we need to invest much more effort into benchmarking as a science. Our space is full of garbage claims like this and it isn't doing us any favors. Eventually the hype will die down and people will realize that a lot of the claims were obvious falsehoods. Then we'll all get collectively punished for it.
- jaggederest 2y ago
  I'm reminded of the time I saw some A/B test results that didn't make much sense, but were highly significant[1] I asked how many A/B tests they were running... hundreds. Overlapping. At least they had a holdout group (that they mostly ignored, which indicated that all the A/B tests more or less made no difference) [1] P < 0.001 with a large effect size. No, your a/b test probably didn't break the laws of economics - you probably messed up your data.
  - MostlyStable 2y ago
    If they were running many concurrent, overlapping A/B tests, then they didn't necessarily mess up their data. You are likely to get that result, honestly and truthfully, purely by chance, if you run enough tests. Unless the "running concurrent tests and not correcting your significance level", is what you meant by messing up their data, in which case yeah.
    - jaggederest 2y ago
      Yes precisely, the latter. I know there are techniques to disentangle that kind of thing but neither I nor they had them readily to hand.
- kordlessagain 2y ago
  The problem is with the reporting, if you want to call it that. This is more like a promotional piece. They write something someone said, and assume it’s true or useful to get attention. There’s not even a name attributed to the article.
aithrowawaycomm 2y ago
I will never stop being amazed at AI folks' childish views of animal cognition: > A lot of your tools reference crows. What’s up with that? > White: When I got started in this space around October 2022, I was red-teaming with GPT4. Around the same time, a paper called “Language Models are Stochastic Parrots” was circulating, and people were debating whether these models were just regurgitating their training data or truly reasoning. The analogy is appealing, and parrots are definitely known for mimicking speech. But what we saw was that pairing these language models with external tools made them much more accurate — a bit like crows, which can use tools to solve puzzles. > In the work that led to ChemCrow,1 for instance, we found that giving the large language model access to calculators or chemistry software made its answers much better. So we kind of retconned a little bit to make “Crows” be agents that can interact with tools using natural language. This is incredibly insulting to crows, who can spontaneously create tools and use bizarre man-made tools with no training. And when crows use tools for problem solving in the lab, the tools are not "solve the problem for me" like a calculator, they require much more creative thinking. What White really means - whether he knows it or not - is that crows are known for being intelligent and he wants to use this for marketing purposes. I don't think anyone alive today will live to see an AI as smart as a crow, in no small part because AI researchers and investors refuse to take animal intelligence seriously.
- mistrial9 2y ago
  yes agree and more.. it is detached thinking from the actual real world creatures called Crow. add some pep to the situation - Crows lineage is from an age before the rise of Humans.
- odyssey7 2y ago
  Western awareness still not recovered from Descartes.
  - _se 2y ago
    Nothing Western about this type of naivety. You'll find this all across the world at present.
dr_dshiv 2y ago
There are massive opportunities for accelerating science through AI-human collaboration. However, it requires new management and a new set of standards. If AI can write a paper that was good for publication a year ago, what does that mean for science? It’s really unclear. Im particularly interested in AI assisted education research. I think we need to keep an eye on empirical methods for developing smarter humans.
uptownfunk 2y ago
I think more reliable in Silico experimentation will yield much better results in the long run but is probably akin to a spacex or Tesla type of investment and 1-2 oom more compute intensive.
deleted 2y ago
[deleted]
admiralrohan 2y ago
How is this different in technical standpoint from the AI agents used for marketing use cases? Is this really some out of the blue use case as this article made it look like?

71 points by surprisetalk 2y ago

>> I’m optimistic that an AI scientist will help with reproducibility overall. Did you do the experiment that you said you did? Did you record all the variables in a way that you can report it in the way you did it? Interesting to think of the potential long term impact for science. Reminds me of the reform in the early 20th century that focused on ensuring the contents of canned goods matched their labeled ingredients.

analog31 2y ago
That's still a thing, in the drug industry.

plaidfuji 2y ago

> Rodriques: Many people assume we’re focused on wet lab automation. There are certainly opportunities there and we are exploring them, but the biggest opportunities are actually on the cognitive side. Wet lab automation is very difficult and capital intensive. And once you build your lab you are constraining yourself to answering questions within a certain domain for which you have the relevant sample prep and characterization equipment. Your equipment in essence defines your design space, and thus your potential solution space, which places a bound on your TAM. So of course automating the thinking part of science is more approachable with current AI - but is that what people want? It’s certainly an attractive proposition for management: automate away the highly-paid SMEs and turn R&D into more of a factory environment with replaceable lab techs.. but actually implementing this depends on where the power lies in an org. My theory is that in many orgs the “cognitive” folks hold the true power (= the unwritten expertise about what works and what doesn’t, when to work around your existing setup, how much to trust each number an instrument produces). They’ll resist this change to their last breath. You may gain some short-term efficiency by accelerating the experiments of today, but in the long run you lose the expertise to break out of local minima imposed by your equipment and training data. Or to think about this another way, imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful?

deleted 2y ago
[deleted]
owenpalmer 2y ago
> imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful? A lot of scientific knowledge cannot be communicated through papers. This is especially true in wet labs, where there's no procedural standardization. Keoni Gandall wrote an excellent post on this topic as it applies to synthetic bio [0]. I've experienced this first hand as a student participating in chemistry labs. Even when you're given a step by step procedure, it's impossible to predict the spacial logistics and inefficiencies you'll run into when you actually try to execute the procedure, regardless of your analytical preparation. The other type of knowledge that is rarely communicated through papers is the informal exploratory thought process of the researcher, and their embarrassing failures/mistakes. If I have my own lab someday, I think it would be cool if everyone wore bodycams, showing their first person view. By publishing the raw footage with the paper/code, hopefully this would help with reproducibility. [0] https://keonigandall.com/posts/too_much_tacit.html https://keonigandall.com/posts/too_much_tacit.html
the_snooze 2y ago
>Or to think about this another way, imagine a PhD student who was never allowed to talk to other people, attend conferences etc., and could only read papers and try things in lab. But they can read papers extremely fast. Would they be successful? For anyone who thinks this is sufficient, be aware that papers only tell you what's successful (for some definition of successful). Being part of a research community gives you access to the other side: hallway conversations at conferences and informal collaborative networks are far more candid, where people will tell you what they've tried and didn't work, or what resources they need for an ambitious study that's just out of reach for their current budget. This is also where a lot of new ideas and collaborations come from, where people come together with matching problems and solutions to new interesting questions. I'm not sure how an AI is supposed to help with this, as research is ultimately a very social activity from my perspective.

light_hue_1 2y ago

I'm so incredibly tired of all of the BS claims. (I'm an AI/ML researcher) > has enabled open-source LLMs “to exceed human-level performance on two more of the lab-bench tasks: doing scientific literature research and reasoning about DNA constructs” with only “modest compute budgets.” No. They did not. They just ran a crappy experiment and came up with an absurd result. As a community we need to invest much more effort into benchmarking as a science. Our space is full of garbage claims like this and it isn't doing us any favors. Eventually the hype will die down and people will realize that a lot of the claims were obvious falsehoods. Then we'll all get collectively punished for it.

jaggederest 2y ago
I'm reminded of the time I saw some A/B test results that didn't make much sense, but were highly significant[1] I asked how many A/B tests they were running... hundreds. Overlapping. At least they had a holdout group (that they mostly ignored, which indicated that all the A/B tests more or less made no difference) [1] P < 0.001 with a large effect size. No, your a/b test probably didn't break the laws of economics - you probably messed up your data.
- MostlyStable 2y ago
  If they were running many concurrent, overlapping A/B tests, then they didn't necessarily mess up their data. You are likely to get that result, honestly and truthfully, purely by chance, if you run enough tests. Unless the "running concurrent tests and not correcting your significance level", is what you meant by messing up their data, in which case yeah.
  - jaggederest 2y ago
    Yes precisely, the latter. I know there are techniques to disentangle that kind of thing but neither I nor they had them readily to hand.
kordlessagain 2y ago
The problem is with the reporting, if you want to call it that. This is more like a promotional piece. They write something someone said, and assume it’s true or useful to get attention. There’s not even a name attributed to the article.

aithrowawaycomm 2y ago

I will never stop being amazed at AI folks' childish views of animal cognition: > A lot of your tools reference crows. What’s up with that? > White: When I got started in this space around October 2022, I was red-teaming with GPT4. Around the same time, a paper called “Language Models are Stochastic Parrots” was circulating, and people were debating whether these models were just regurgitating their training data or truly reasoning. The analogy is appealing, and parrots are definitely known for mimicking speech. But what we saw was that pairing these language models with external tools made them much more accurate — a bit like crows, which can use tools to solve puzzles. > In the work that led to ChemCrow,1 for instance, we found that giving the large language model access to calculators or chemistry software made its answers much better. So we kind of retconned a little bit to make “Crows” be agents that can interact with tools using natural language. This is incredibly insulting to crows, who can spontaneously create tools and use bizarre man-made tools with no training. And when crows use tools for problem solving in the lab, the tools are not "solve the problem for me" like a calculator, they require much more creative thinking. What White really means - whether he knows it or not - is that crows are known for being intelligent and he wants to use this for marketing purposes. I don't think anyone alive today will live to see an AI as smart as a crow, in no small part because AI researchers and investors refuse to take animal intelligence seriously.

mistrial9 2y ago
yes agree and more.. it is detached thinking from the actual real world creatures called Crow. add some pep to the situation - Crows lineage is from an age before the rise of Humans.
odyssey7 2y ago
Western awareness still not recovered from Descartes.
- _se 2y ago
  Nothing Western about this type of naivety. You'll find this all across the world at present.

dr_dshiv 2y ago

There are massive opportunities for accelerating science through AI-human collaboration. However, it requires new management and a new set of standards. If AI can write a paper that was good for publication a year ago, what does that mean for science? It’s really unclear. Im particularly interested in AI assisted education research. I think we need to keep an eye on empirical methods for developing smarter humans.

uptownfunk 2y ago

I think more reliable in Silico experimentation will yield much better results in the long run but is probably akin to a spacex or Tesla type of investment and 1-2 oom more compute intensive.

deleted 2y ago

[deleted]

admiralrohan 2y ago

How is this different in technical standpoint from the AI agents used for marketing use cases? Is this really some out of the blue use case as this article made it look like?