5 ms·

> Critically, this hypothesis was unique to the research team and had not been published anywhere else. Nobody in the team had shared their findings This seem

> Critically, this hypothesis was unique to the research team and had not been published anywhere else. Nobody in the team had shared their findings This seems like the most important detail, but it also seems impossible to verify if this was actually the case. What are the chances that this AI spat out a totally unique hypothesis that has absolutely no corollaries in the training data, that also happens to be the pet hypothesis of this particular research team? I'm open to being convinced, but I'm skeptical.

mtrovo 2y ago
Define "totally unique hypothesis" in this context. If the training data contains studies with paths like A -> B and C -> D -> E, and the AI independently generates a proof linking B -> C, effectively creating a path from A -> E, is that original enough? At some point, I think we're going to run out of definitions for what makes human intelligence unique. > It also seems impossible to verify if this was actually the case. If this is a thinking model, you could always debug the raw output of the model's internal reasoning when it was generating an answer. If the agent took 48 hours to respond and we had no idea what it was doing that whole time, that would be the real surprise to me, especially since Google is only releasing this in a closed beta for now.
TrackerFF 2y ago
Could some of the scientists have saved their data in the google cloud, say using google drive? And then some internal google crawler went through, and indexed those files? I don't know that their policy says about that, or if it is even something they do...at least not publicly.
miyuru 2y ago
> "I wrote an email to Google to say, 'you have access to my computer, is that right?'", he added. sounds extra fishy, since google does not provide email support normally.
- Jimmc414 2y ago
  "Scientists who are part of our Trusted Tester Program will have early access to AI co-scientist" https://blog.google/feed/google-research-ai-co-scientist/ https://blog.google/feed/google-research-ai-co-scientist/
- bArray 2y ago
  Exactly my thought, probably the least likely part of the whole thing. He emailed Google and they replied. Not only that, he asked a question they would really rather not answer.
bArray 2y ago
Google openly train stuff based on your email, they used is specifically to train Smart Compose, but maybe other stuff too. He likely uses multiple Google products. Draft papers in Google Drive perhaps? These LLM models are essentially trying to produce material that sounds correct, perhaps the hypothesis was a relatively obvious question with the right domain knowledge. Additionally, he may not have been the first to ask the question. It's entirely possible that the AI chewed up and spat out some domain knowledge from a foreign research group outside of his wheelhouse. This kind of stuff happens all the time. I personally have accidentally reinvented things without prior knowledge of them. Many years ago in University I remember deriving a PID controller without being aware of what one was. I probably got enough clues from other people/media that were aware of them, that bridging that final gap was made easier.
- Jabbles 2y ago
  > We do not use your Workspace data to train or improve the underlying generative AI and large language models that power Gemini, Search, and other systems outside of Workspace without permission. https://support.google.com/meet/answer/14615114?hl=en#:~:text=We%20do%20not%20use%20your%20Workspace%20data%20to%20train%20or%20improve%20the%20underlying%20generative%20AI%20and%20large%20language%20models%20that%20power%20Gemini%2C%20Search%2C%20and%20other%20systems%20outside%20of%20Workspace%20without%20permission. https://support.google.com/meet/answer/14615114?hl=en#:~:tex... You may not believe them, but I challenge your description of it as "openly".
  - stonogo 2y ago
    Yeah, "Workspace data." If you don't think a scientist has copies of all his/her stuff in a personal Google account you've never met a scientist.
    - cwillu 2y ago
      The second half of the sentence is a bit awkward, but the first bit is pretty clear: “We do not use your Workspace data”
card_zero 2y ago
I wonder about a "clever Hans" effect, where they unwittingly suggest their discovery in their prompt. Also whether they got paid.
- cwillu 2y ago
  “ "It's not just that the top hypothesis they provide was the right one," he said. "It's that they provide another four, and all of them made sense. And for one of them, we never thought about it, and we're now working on that."”
  - card_zero 2y ago
    I wonder whether that one is really any good.
- graeme 2y ago
  May well be but in this case it would be a two way clever Hans which is very promising.

mtrovo 2y ago

Define "totally unique hypothesis" in this context. If the training data contains studies with paths like A -> B and C -> D -> E, and the AI independently generates a proof linking B -> C, effectively creating a path from A -> E, is that original enough? At some point, I think we're going to run out of definitions for what makes human intelligence unique. > It also seems impossible to verify if this was actually the case. If this is a thinking model, you could always debug the raw output of the model's internal reasoning when it was generating an answer. If the agent took 48 hours to respond and we had no idea what it was doing that whole time, that would be the real surprise to me, especially since Google is only releasing this in a closed beta for now.

TrackerFF 2y ago

Could some of the scientists have saved their data in the google cloud, say using google drive? And then some internal google crawler went through, and indexed those files? I don't know that their policy says about that, or if it is even something they do...at least not publicly.

miyuru 2y ago

> "I wrote an email to Google to say, 'you have access to my computer, is that right?'", he added. sounds extra fishy, since google does not provide email support normally.

Jimmc414 2y ago
"Scientists who are part of our Trusted Tester Program will have early access to AI co-scientist" https://blog.google/feed/google-research-ai-co-scientist/ https://blog.google/feed/google-research-ai-co-scientist/
bArray 2y ago
Exactly my thought, probably the least likely part of the whole thing. He emailed Google and they replied. Not only that, he asked a question they would really rather not answer.

bArray 2y ago

Google openly train stuff based on your email, they used is specifically to train Smart Compose, but maybe other stuff too. He likely uses multiple Google products. Draft papers in Google Drive perhaps? These LLM models are essentially trying to produce material that sounds correct, perhaps the hypothesis was a relatively obvious question with the right domain knowledge. Additionally, he may not have been the first to ask the question. It's entirely possible that the AI chewed up and spat out some domain knowledge from a foreign research group outside of his wheelhouse. This kind of stuff happens all the time. I personally have accidentally reinvented things without prior knowledge of them. Many years ago in University I remember deriving a PID controller without being aware of what one was. I probably got enough clues from other people/media that were aware of them, that bridging that final gap was made easier.

Jabbles 2y ago
> We do not use your Workspace data to train or improve the underlying generative AI and large language models that power Gemini, Search, and other systems outside of Workspace without permission. https://support.google.com/meet/answer/14615114?hl=en#:~:text=We%20do%20not%20use%20your%20Workspace%20data%20to%20train%20or%20improve%20the%20underlying%20generative%20AI%20and%20large%20language%20models%20that%20power%20Gemini%2C%20Search%2C%20and%20other%20systems%20outside%20of%20Workspace%20without%20permission. https://support.google.com/meet/answer/14615114?hl=en#:~:tex... You may not believe them, but I challenge your description of it as "openly".
- stonogo 2y ago
  Yeah, "Workspace data." If you don't think a scientist has copies of all his/her stuff in a personal Google account you've never met a scientist.
  - cwillu 2y ago
    The second half of the sentence is a bit awkward, but the first bit is pretty clear: “We do not use your Workspace data”

card_zero 2y ago

I wonder about a "clever Hans" effect, where they unwittingly suggest their discovery in their prompt. Also whether they got paid.

cwillu 2y ago
“ "It's not just that the top hypothesis they provide was the right one," he said. "It's that they provide another four, and all of them made sense. And for one of them, we never thought about it, and we're now working on that."”
- card_zero 2y ago
  I wonder whether that one is really any good.
graeme 2y ago
May well be but in this case it would be a two way clever Hans which is very promising.