3 ms·

Question for any Googlers in the thread - do folks speak up if they see flaws in the methodology or approach of this research or is the pressure from the top so

by tmpz22 2y ago

Question for any Googlers in the thread - do folks speak up if they see flaws in the methodology or approach of this research or is the pressure from the top so strong on this initiative that people hush up?

rishabhjain1198 2y ago
No pressure from the top. Methodologies are pretty solid. The internal dev tooling at Google is quite far ahead of what's available on the market rn.
barrkel 2y ago
The code completion is quite smart and one of the bigger advantages Google has now is the monorepo and the knowhow to put together a pipeline of continuous tuning of models to keep them up to date. The pressure, such that it is, is killing funding for the custom extension for IntelliJ that made it possible to use it with the internal repo. Cider doesn't have the code manipulation featureset that IntelliJ has, but it's making up for that with deeper AI integration.
DannyBee 2y ago
Fwiw, ai code completion initiatives at Google well predate current hype (ie overall research path, etc, was started a long time ago, though obviously modified as capability has changed). So this particular thing was a very well established program, path, and methodology by the time AI hype came. Whether that is good or bad I won't express an opinion, but it might mean you get a different answer to your question for this particular thing.
UncleMeat 2y ago
There were already teams building ML-based code completion, code suggestions, and code repair before LLMs blew up a couple years ago. So the principle of it isn't driven by AI hype. Yes, there are oodles of people complaining about AI overuse and there is a massive diversity of opinion about these tools being used for coding, testing, LSCs, etc. I've seen every opinion from "this is absolute garbage" to "this is utter magic" and everything in between. I personally think that the AI suggestions in code review are pretty uniformly awful and a lot of people disable that feature. The team that owns the feature tracks metrics on disabling rates. I also have found the AI code completion while actually writing code to be pretty good. I also think that the "% of characters written by AI" is a pretty bad metric to chase (and I'm stunned it is so high). Plenty of people, including fairly senior people, have expressed concern with this metric. I also know that relevant teams are tracking other stuff like rollback rates to establish metrics around quality. There is definitely pressure to use AI as much as reasonably possible and I think that at the VP and SVP level it is getting unreasonable, but at the director and below level I've found that people are largely reasonable about where to deploy AI, where to experiment with AI, and where to AI to fuck off.

by tmpz22 2y ago

rishabhjain1198 2y ago

No pressure from the top. Methodologies are pretty solid. The internal dev tooling at Google is quite far ahead of what's available on the market rn.

barrkel 2y ago

The code completion is quite smart and one of the bigger advantages Google has now is the monorepo and the knowhow to put together a pipeline of continuous tuning of models to keep them up to date. The pressure, such that it is, is killing funding for the custom extension for IntelliJ that made it possible to use it with the internal repo. Cider doesn't have the code manipulation featureset that IntelliJ has, but it's making up for that with deeper AI integration.

DannyBee 2y ago

Fwiw, ai code completion initiatives at Google well predate current hype (ie overall research path, etc, was started a long time ago, though obviously modified as capability has changed). So this particular thing was a very well established program, path, and methodology by the time AI hype came. Whether that is good or bad I won't express an opinion, but it might mean you get a different answer to your question for this particular thing.

UncleMeat 2y ago

There were already teams building ML-based code completion, code suggestions, and code repair before LLMs blew up a couple years ago. So the principle of it isn't driven by AI hype. Yes, there are oodles of people complaining about AI overuse and there is a massive diversity of opinion about these tools being used for coding, testing, LSCs, etc. I've seen every opinion from "this is absolute garbage" to "this is utter magic" and everything in between. I personally think that the AI suggestions in code review are pretty uniformly awful and a lot of people disable that feature. The team that owns the feature tracks metrics on disabling rates. I also have found the AI code completion while actually writing code to be pretty good. I also think that the "% of characters written by AI" is a pretty bad metric to chase (and I'm stunned it is so high). Plenty of people, including fairly senior people, have expressed concern with this metric. I also know that relevant teams are tracking other stuff like rollback rates to establish metrics around quality. There is definitely pressure to use AI as much as reasonably possible and I think that at the VP and SVP level it is getting unreasonable, but at the director and below level I've found that people are largely reasonable about where to deploy AI, where to experiment with AI, and where to AI to fuck off.