4 ms·

A friend of mine who is in "serious" AI research keeps telling me not to get too excited about Google's AI stuff, as they do release a lot of papers, but these

A friend of mine who is in "serious" AI research keeps telling me not to get too excited about Google's AI stuff, as they do release a lot of papers, but these are essentially worthless and non-reproduceable as they hide raw data. He suspects half of them or more are completely made up, as they do not align well with what else happens in the field. I can't verify that, but whenever the "AI" behind my Google Home gets a bit more stupid (happens once a week nowadays, with it 'forgetting' even the few things that worked before), I am inclined to agree.

swyx 4y ago
agree that they are nonreproduceable, but they are probably not worthless in that i'm sure they have very extensive review processes and theyve said that they actually put models like T5 into production on search. there's no bigger skin in the game than that. > they do not align well with what else happens in the field i'm interested in this comment though. havent transformers changed the field? what is this referring to? mind getting your friend to elaborate?
- visarga 4y ago
  > they actually put models like T5 into production on search Google search does not show signs of using a decent language model. If they did, then questions like this should work: > "What is the world record for crossing the English Channel entirely on foot?" This is the quote it gives on top of all results: > And it was a Towson University graduate to do it the fastest. Two weeks ago, Nik Haynes '00 became a world-record holder by backstroking the English Channel in an astounding 12 hours, 52 minutes. Haynes bested the mark set by Tina Neill in 2005 of 13 hours, 22 minutes.Aug 27, 2020 And the rest of the page is anything but crossing on foot. Ok, so it missed the core semantics of the question completely. In reality there have been crossings through the Channel Tunnel. And at some point in the distant past the sea level was low enough people could cross on foot. This kind of problem happens in many searches - if you search topic X which is not very popular, and there is a topic Y close to X that is very popular, Google will reply to Y instead of X. Topic blindness.
netheril96 4y ago
> as they do release a lot of papers, but these are essentially worthless and non-reproduceable as they hide raw data. Well, I know at least that the transformers from Google are widely used in industry outside Google.
- visarga 4y ago
  The authors of that paper have long left Google and are on a bunch of startups. So you could say they already lost that advantage.

swyx 4y ago

agree that they are nonreproduceable, but they are probably not worthless in that i'm sure they have very extensive review processes and theyve said that they actually put models like T5 into production on search. there's no bigger skin in the game than that. > they do not align well with what else happens in the field i'm interested in this comment though. havent transformers changed the field? what is this referring to? mind getting your friend to elaborate?

visarga 4y ago
> they actually put models like T5 into production on search Google search does not show signs of using a decent language model. If they did, then questions like this should work: > "What is the world record for crossing the English Channel entirely on foot?" This is the quote it gives on top of all results: > And it was a Towson University graduate to do it the fastest. Two weeks ago, Nik Haynes '00 became a world-record holder by backstroking the English Channel in an astounding 12 hours, 52 minutes. Haynes bested the mark set by Tina Neill in 2005 of 13 hours, 22 minutes.Aug 27, 2020 And the rest of the page is anything but crossing on foot. Ok, so it missed the core semantics of the question completely. In reality there have been crossings through the Channel Tunnel. And at some point in the distant past the sea level was low enough people could cross on foot. This kind of problem happens in many searches - if you search topic X which is not very popular, and there is a topic Y close to X that is very popular, Google will reply to Y instead of X. Topic blindness.

netheril96 4y ago

> as they do release a lot of papers, but these are essentially worthless and non-reproduceable as they hide raw data. Well, I know at least that the transformers from Google are widely used in industry outside Google.

visarga 4y ago
The authors of that paper have long left Google and are on a bunch of startups. So you could say they already lost that advantage.