6 ms·

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a littl

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).

xiphias2 4y ago
I guess it will get more interesting for your work when it integrates with BioTech startup apis as plugins (I imagine not too cheap ones)
KidComputer 4y ago
This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process. Maybe you can ask GPT for some good starting points.
- lend000 4y ago
  Sure, but this is a glass half empty isolated scenario that could be more than offset by the positives. For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
  - lanstin 4y ago
    I don’t understand why people think this information, to solve biology, is out there in the linguisticly expressed training data we have. Our knowledge of biology is pretty small, it because we haven’t put it all together but because there are vast swaths of stuff we have no idea about or ideas opposite to the truth (evidence, every time we get mechanical data about some biological system, the data contradict some big belief; how many human genes? 100k up until the day we sequenced it and it was 30k. Information flow in the cell, dna to protein only, unidirectional, till we undercover reverse transcription and now proteonomics, methylation factors, etc. etc. once we stop discovering new planets with each better telescope, then maybe we can master orbital dynamics. And this knowledge is not linguistic, it is more practical knowledge. I doubt it is just a matter of combining all the stuff we have tried in disparate experiments, but it is a matter of sharpening and refined our models and tools to confirm the models. Real8ty doesn’t care what we think and say, and mastering what humans think and say is a long way from mastering the molecules that make humans up.
    - sjducb 4y ago
      Ive had this chat with engineers too many times. They're used to systems where we know 99% of everything that matters. They don't believe that we only know 0.001% of biology.
      - mchaver 4y ago
        There's a certain hubris in many engineers and software developers because we are used to having a lot of control over the systems we work on. It can be intoxicating, but then we assume that applies to other areas of knowledge and study. ChatGPT is really cool because it offers a new way to fetch data from the body of internet knowledge. It is impressive because it can remix it the knowledge really fast (give X in the style of Y with constraints Z). It functions as StackOverflow without condescending remarks. It can build models of knowledge based on the data set and use it to give interpretations of new knowledge based on that model and may have emergent properties. It is not yet exploring or experiencing the physical world like humans so that makes it hard to do empirical studies. Maybe one day these systems can, but it not in their current forms.
dekhn 4y ago
My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous. For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly the primary molecular structure by which heredity is transferred. The experiments had the advantage of being fairly easy to understand, easy to replicate, and fairly convincing. There are probably similar simple experiments that can be easily reproduced widely that would resolve any number of interesting questions outstanding in the field. For example, I'd like to see better ways of demonstrating the causal nature of the genome on the heredity of height, or answering a few important open questions in biology. Right now discovery science is a chaotic, expensive, stochastic process which fails the vast majority of the time and even when it succeeds, usually only makes small incremental discoveries or slightly reduces the ambiguity of experiment's results. Most of the ttime is spent simply mastering boring technical details like how to eliminate variables (Jacob and Monod made their early discoveries in gene regulation because they were just a bit better at maintaining sterile cultures than their competitors, which allowed them to conceive of good if obvious hypotheses quickly, and verify them.
- maxlamb 4y ago
  By that definition of AGI, not even most scientists are generally intelligent.
  - yosito 4y ago
    Speaking from personal experience of a career in science, this is true.
- justrealist 4y ago
  At least recognize that the definition of AGI is moving from the previous goalpost of "passable human-level intelligence" to "superhuman at all things at once".
  - 4y ago
throwaway292939 4y ago
My take is that for GPT-4, it has mastery of existing knowledge. I'm not sure how it would be able to push new boundaries.

xiphias2 4y ago

I guess it will get more interesting for your work when it integrates with BioTech startup apis as plugins (I imagine not too cheap ones)

KidComputer 4y ago

This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process. Maybe you can ask GPT for some good starting points.

lend000 4y ago
Sure, but this is a glass half empty isolated scenario that could be more than offset by the positives. For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
- lanstin 4y ago
  I don’t understand why people think this information, to solve biology, is out there in the linguisticly expressed training data we have. Our knowledge of biology is pretty small, it because we haven’t put it all together but because there are vast swaths of stuff we have no idea about or ideas opposite to the truth (evidence, every time we get mechanical data about some biological system, the data contradict some big belief; how many human genes? 100k up until the day we sequenced it and it was 30k. Information flow in the cell, dna to protein only, unidirectional, till we undercover reverse transcription and now proteonomics, methylation factors, etc. etc. once we stop discovering new planets with each better telescope, then maybe we can master orbital dynamics. And this knowledge is not linguistic, it is more practical knowledge. I doubt it is just a matter of combining all the stuff we have tried in disparate experiments, but it is a matter of sharpening and refined our models and tools to confirm the models. Real8ty doesn’t care what we think and say, and mastering what humans think and say is a long way from mastering the molecules that make humans up.
  - sjducb 4y ago
    Ive had this chat with engineers too many times. They're used to systems where we know 99% of everything that matters. They don't believe that we only know 0.001% of biology.
    - mchaver 4y ago
      There's a certain hubris in many engineers and software developers because we are used to having a lot of control over the systems we work on. It can be intoxicating, but then we assume that applies to other areas of knowledge and study. ChatGPT is really cool because it offers a new way to fetch data from the body of internet knowledge. It is impressive because it can remix it the knowledge really fast (give X in the style of Y with constraints Z). It functions as StackOverflow without condescending remarks. It can build models of knowledge based on the data set and use it to give interpretations of new knowledge based on that model and may have emergent properties. It is not yet exploring or experiencing the physical world like humans so that makes it hard to do empirical studies. Maybe one day these systems can, but it not in their current forms.

dekhn 4y ago

My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous. For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly the primary molecular structure by which heredity is transferred. The experiments had the advantage of being fairly easy to understand, easy to replicate, and fairly convincing. There are probably similar simple experiments that can be easily reproduced widely that would resolve any number of interesting questions outstanding in the field. For example, I'd like to see better ways of demonstrating the causal nature of the genome on the heredity of height, or answering a few important open questions in biology. Right now discovery science is a chaotic, expensive, stochastic process which fails the vast majority of the time and even when it succeeds, usually only makes small incremental discoveries or slightly reduces the ambiguity of experiment's results. Most of the ttime is spent simply mastering boring technical details like how to eliminate variables (Jacob and Monod made their early discoveries in gene regulation because they were just a bit better at maintaining sterile cultures than their competitors, which allowed them to conceive of good if obvious hypotheses quickly, and verify them.

maxlamb 4y ago
By that definition of AGI, not even most scientists are generally intelligent.
- yosito 4y ago
  Speaking from personal experience of a career in science, this is true.
justrealist 4y ago
At least recognize that the definition of AGI is moving from the previous goalpost of "passable human-level intelligence" to "superhuman at all things at once".
- 4y ago

throwaway292939 4y ago

My take is that for GPT-4, it has mastery of existing knowledge. I'm not sure how it would be able to push new boundaries.