6 ms·

Artificial Intelligence, Scientific Discovery, and Product Innovation [pdf]

180 points by therabbithole 2y ago

newyankee 2y ago
Well I hope it works well and fast enough. I cannot wait for my 10k cycles, 300 Wh/kg batteries. 35% efficiency solar modules in market at cheap prices and plenty of nanotech breakthroughs that were promised yet we are still waiting on
11101010001100 2y ago
Any idea if the points raised here https://pubs.acs.org/doi/10.1021/acs.chemmater.4c00643 https://pubs.acs.org/doi/10.1021/acs.chemmater.4c00643 were considered in the analysis?
bbor 2y ago
Well damn, that’s a lot more specific and empirical than I was expecting given the title. Fascinating stuff, talk about a useful setup for studying the issue! “AI is useless to many but invaluable to some” (as mentioned in the abstract) is a great counterpoint to anti-AI luddites. No offense to any luddites on here ofc, the luddites were pretty darn woke for their time, all things considered
youoy 2y ago
From the conclusions: > I find that AI substantially boosts materials discovery, leading to an increase in patent filing and a rise in downstream product innovation. However, the technology is effective only when paired with sufficiently skilled scientists. I can see the point here. Today I was exploring the possibility of some new algorithm. I asked Claude to generate some part which is well know (but there are not a lot of examples on the internet) and it hallucinated some function. In spite of being bad, it was sufficiently close to the solution that I could myself "rehallucinate it" from my side, and turn it into a creative solution. Of course, the hallucination would have been useless if I was not already an expert in the field.
- darepublic 2y ago
  I find proofreading the code gen ai less satisfying than writing it myself though it does depend on the nature of the function. Migrating mindless mapping type functions to autocomplete is nice
  - mkatx 2y ago
    This is one big point I've subscribed to, I'd rather write the code and understand it that way, than read and try to understand code I did not write. Also, I think it would be faster to write my own than try to fully understand others (LLM) code. I have developed my own ways of ensuring certain aspects of the code, like security, organization, and speed. Trying to knead out how those things are addressed in code I didn't write takes me longer. Edit; spelling
- vatys 2y ago
  I wonder if the next generation of experts will be held back by use of AI tools. Having learned things “the hard way” without AI tools may allow better judgement of these semi-reliable outputs. A younger generation growing up in this era would not yet have that experience and may be more accepting of AI generated results.
  - mkatx 2y ago
    Yeah, as a cs student, some professors allow use of LLM's because it is what will be a part of the job going forward. I get that, and I use them for learning, as opposed to internet searches, but I still manually write my code and fully understand it, cause I don't wanna miss out on those lessons. Otherwise I might not be able to verify an LLM's output.
    - daveguy 2y ago
      Excellent approach. You will be leagues ahead of someone who relies on LLM alone.
      - 2y ago
slopeloaf 2y ago
“Survey evidence reveals that these gains come at a cost, however, as 82% of scientists report reduced satisfaction with their work due to decreased creativity and skill underutilization.” What an interesting finding and not what I was expecting. Is this an issue with the UX/tooling? Could we alleviate this with an interface that still incorporates the joy of problem solving. I haven’t seen any research that Copilot and similar tools for programmers have a similar reduction in satisfaction. Likely with how much the tools feel like an extension of traditional auto complete, and you still spend a lot of time “programming”. You haven’t abandoned your core skill. Related: I often find myself disabling copilot when I have a fun problem I want the satisfaction of solving myself.
- gmaster1440 2y ago
  AI appears to have automated aspects of the job scientists found most intellectually satisfying. - Reduced creativity and ideation work (dropping from 39% to 16% of time) - Increased focus on evaluating AI suggestions (rising to 40% of time) - Feelings of skill underutilization
- sourcepluck 2y ago
  > Related: I often find myself disabling copilot when I have a fun problem I want the satisfaction of solving myself. The way things seem to be going, I'd be worried management will find a way to monitor and try cut out this "security risk" in the coming months and years.
- dennisy 2y ago
  I feel if people are finding programming as creative and interesting with AI as without there is a chance they actually prefer product management? Half statement, half question… I have personally stopped using AI assistance in programming as I felt it was making my mind lazy, and I stopped learning.
  - aerhardt 2y ago
    The thing I like the most about AI coding is how it lowers the threshold of energy and motivation needed to start a task. Being able to write a detailed spec of what I want, or even discussing an attack plan (for high-level architecture or solution design) and getting an initial draft is game-changing for me. I usually take it from there, because as far as I can tell, it sucks after that point anyway.
uxhacker 2y ago
It’s interesting to see how this research emphasizes the continued need for human expertise, even in the era of advanced AI. It highlights that while AI can significantly boost productivity, the value of human judgment and domain knowledge remains crucial.
- nyrikki 2y ago
  Even Warren McCulloch and Walter Pitts were the two who originally modeled neurons with OR statements, realized it wasn't sufficient for a full replacement. Biological neurons have many features like active dendritic compartmentalization that perceptrons cannot duplicate. They are different with different advantages and limitations. We have also known about the specification and frame problems for a long time also. Note that part of the reason for the split between the symbolic camp and statistical camp in the 90s was due to more practical models being possible with existential quantification. There have been several papers on HN talking about a shift to universal quantification to get around limitations lately. Unfortunately discussions about the limits of first order logic have historical challenges and adding in the limits of fragments of first order logic like grounding are compounded upon those challenges with cognitive dissonance. While understanding the abilities of multi level perceptrons is challenging, there is a path of realizing the implications of an individual perceptron as a choice function that is useful for me. The same limits that have been known for decades still hold in the general case for those who can figure a way to control their own cognitive dissonance, but they are just lenses. As an industry we need to find ways to avoid the traps of the Brouwer–Hilbert controversy and unsettled questions and opaque definitions about the nature of intelligence to fully exploit the advantages. Hopefully experience will tempor the fear and enthusiasm for AGI that has made it challenging to discuss the power and constraints of ML. I know that even discussing dropping the a priori assumption of LEM with my brother who has a PhD in complex analysis is challenging. But the platonic ideals simply don't hold for non-trivial properties, and no matter if we are using ML or BoG Sat, the hard problems are too high in the polynomial hierarchy to make that assumption.
gmaster1440 2y ago
How generalizable are these findings given the rapid pace of AI advancement? The paper studies a snapshot in time with current AI capabilities, but the relationship between human expertise and AI could look very different with more advanced models. I would love to have seen the paper: - Examine how the human-AI relationship evolved as the AI system improved during the study period - Theorize more explicitly about which aspects of human judgment might be more vs less persistent - Consider how their findings might change with more capable AI systems
Animats 2y ago
"The tool automates a majority of “idea generation” tasks, reallocating scientists to the new task of evaluating model-suggested candidate compounds. In the absence of AI, researchers devote nearly half their time to conceptualizing potential materials. This falls to less than 16% after the tool’s introduction. Meanwhile, time spent assessing candidate materials increases by 74%" So the AI is in charge, and mostly needs a bunch of lab assistants. "Machines should think. People should work." - not a joke any more.
caycep 2y ago
would there be a difference in accuracy of the statement if you replace AI w/ "data science and statistical models"?
iimaginary 2y ago
Conclusion: Augmented Intelligence is more useful than Artificial Intelligence.
lysecret 2y ago
Interesting, a large US company with over 1000 materials scientists (there can only be a handful of those) introduced a cutting-edge AI tool and decided to make a study out of it / randomize it and gave all the credentials to some econ PHD student. Would love to know more about how this came to be. Also, why his PHD supervisor didn't get a co-author, never seen that. I'm always slightly suspicious of these very strong results without any public data / way to reproduce it. We essentially have to believe 1 guys word.

180 points by therabbithole 2y ago

Well I hope it works well and fast enough. I cannot wait for my 10k cycles, 300 Wh/kg batteries. 35% efficiency solar modules in market at cheap prices and plenty of nanotech breakthroughs that were promised yet we are still waiting on

11101010001100 2y ago

Any idea if the points raised here https://pubs.acs.org/doi/10.1021/acs.chemmater.4c00643 https://pubs.acs.org/doi/10.1021/acs.chemmater.4c00643 were considered in the analysis?

bbor 2y ago

Well damn, that’s a lot more specific and empirical than I was expecting given the title. Fascinating stuff, talk about a useful setup for studying the issue! “AI is useless to many but invaluable to some” (as mentioned in the abstract) is a great counterpoint to anti-AI luddites. No offense to any luddites on here ofc, the luddites were pretty darn woke for their time, all things considered

youoy 2y ago

From the conclusions: > I find that AI substantially boosts materials discovery, leading to an increase in patent filing and a rise in downstream product innovation. However, the technology is effective only when paired with sufficiently skilled scientists. I can see the point here. Today I was exploring the possibility of some new algorithm. I asked Claude to generate some part which is well know (but there are not a lot of examples on the internet) and it hallucinated some function. In spite of being bad, it was sufficiently close to the solution that I could myself "rehallucinate it" from my side, and turn it into a creative solution. Of course, the hallucination would have been useless if I was not already an expert in the field.

darepublic 2y ago
I find proofreading the code gen ai less satisfying than writing it myself though it does depend on the nature of the function. Migrating mindless mapping type functions to autocomplete is nice
- mkatx 2y ago
  This is one big point I've subscribed to, I'd rather write the code and understand it that way, than read and try to understand code I did not write. Also, I think it would be faster to write my own than try to fully understand others (LLM) code. I have developed my own ways of ensuring certain aspects of the code, like security, organization, and speed. Trying to knead out how those things are addressed in code I didn't write takes me longer. Edit; spelling
vatys 2y ago
I wonder if the next generation of experts will be held back by use of AI tools. Having learned things “the hard way” without AI tools may allow better judgement of these semi-reliable outputs. A younger generation growing up in this era would not yet have that experience and may be more accepting of AI generated results.
- mkatx 2y ago
  Yeah, as a cs student, some professors allow use of LLM's because it is what will be a part of the job going forward. I get that, and I use them for learning, as opposed to internet searches, but I still manually write my code and fully understand it, cause I don't wanna miss out on those lessons. Otherwise I might not be able to verify an LLM's output.
  - daveguy 2y ago
    Excellent approach. You will be leagues ahead of someone who relies on LLM alone.
    - 2y ago

slopeloaf 2y ago

“Survey evidence reveals that these gains come at a cost, however, as 82% of scientists report reduced satisfaction with their work due to decreased creativity and skill underutilization.” What an interesting finding and not what I was expecting. Is this an issue with the UX/tooling? Could we alleviate this with an interface that still incorporates the joy of problem solving. I haven’t seen any research that Copilot and similar tools for programmers have a similar reduction in satisfaction. Likely with how much the tools feel like an extension of traditional auto complete, and you still spend a lot of time “programming”. You haven’t abandoned your core skill. Related: I often find myself disabling copilot when I have a fun problem I want the satisfaction of solving myself.

gmaster1440 2y ago
AI appears to have automated aspects of the job scientists found most intellectually satisfying. - Reduced creativity and ideation work (dropping from 39% to 16% of time) - Increased focus on evaluating AI suggestions (rising to 40% of time) - Feelings of skill underutilization
sourcepluck 2y ago
> Related: I often find myself disabling copilot when I have a fun problem I want the satisfaction of solving myself. The way things seem to be going, I'd be worried management will find a way to monitor and try cut out this "security risk" in the coming months and years.
dennisy 2y ago
I feel if people are finding programming as creative and interesting with AI as without there is a chance they actually prefer product management? Half statement, half question… I have personally stopped using AI assistance in programming as I felt it was making my mind lazy, and I stopped learning.
- aerhardt 2y ago
  The thing I like the most about AI coding is how it lowers the threshold of energy and motivation needed to start a task. Being able to write a detailed spec of what I want, or even discussing an attack plan (for high-level architecture or solution design) and getting an initial draft is game-changing for me. I usually take it from there, because as far as I can tell, it sucks after that point anyway.

uxhacker 2y ago

It’s interesting to see how this research emphasizes the continued need for human expertise, even in the era of advanced AI. It highlights that while AI can significantly boost productivity, the value of human judgment and domain knowledge remains crucial.

nyrikki 2y ago
Even Warren McCulloch and Walter Pitts were the two who originally modeled neurons with OR statements, realized it wasn't sufficient for a full replacement. Biological neurons have many features like active dendritic compartmentalization that perceptrons cannot duplicate. They are different with different advantages and limitations. We have also known about the specification and frame problems for a long time also. Note that part of the reason for the split between the symbolic camp and statistical camp in the 90s was due to more practical models being possible with existential quantification. There have been several papers on HN talking about a shift to universal quantification to get around limitations lately. Unfortunately discussions about the limits of first order logic have historical challenges and adding in the limits of fragments of first order logic like grounding are compounded upon those challenges with cognitive dissonance. While understanding the abilities of multi level perceptrons is challenging, there is a path of realizing the implications of an individual perceptron as a choice function that is useful for me. The same limits that have been known for decades still hold in the general case for those who can figure a way to control their own cognitive dissonance, but they are just lenses. As an industry we need to find ways to avoid the traps of the Brouwer–Hilbert controversy and unsettled questions and opaque definitions about the nature of intelligence to fully exploit the advantages. Hopefully experience will tempor the fear and enthusiasm for AGI that has made it challenging to discuss the power and constraints of ML. I know that even discussing dropping the a priori assumption of LEM with my brother who has a PhD in complex analysis is challenging. But the platonic ideals simply don't hold for non-trivial properties, and no matter if we are using ML or BoG Sat, the hard problems are too high in the polynomial hierarchy to make that assumption.

gmaster1440 2y ago

How generalizable are these findings given the rapid pace of AI advancement? The paper studies a snapshot in time with current AI capabilities, but the relationship between human expertise and AI could look very different with more advanced models. I would love to have seen the paper: - Examine how the human-AI relationship evolved as the AI system improved during the study period - Theorize more explicitly about which aspects of human judgment might be more vs less persistent - Consider how their findings might change with more capable AI systems

Animats 2y ago

"The tool automates a majority of “idea generation” tasks, reallocating scientists to the new task of evaluating model-suggested candidate compounds. In the absence of AI, researchers devote nearly half their time to conceptualizing potential materials. This falls to less than 16% after the tool’s introduction. Meanwhile, time spent assessing candidate materials increases by 74%" So the AI is in charge, and mostly needs a bunch of lab assistants. "Machines should think. People should work." - not a joke any more.

caycep 2y ago

would there be a difference in accuracy of the statement if you replace AI w/ "data science and statistical models"?

iimaginary 2y ago

Conclusion: Augmented Intelligence is more useful than Artificial Intelligence.

lysecret 2y ago

Interesting, a large US company with over 1000 materials scientists (there can only be a handful of those) introduced a cutting-edge AI tool and decided to make a study out of it / randomize it and gave all the credentials to some econ PHD student. Would love to know more about how this came to be. Also, why his PHD supervisor didn't get a co-author, never seen that. I'm always slightly suspicious of these very strong results without any public data / way to reproduce it. We essentially have to believe 1 guys word.