16 ms·

I work for Google, and I just got done with my work day. I was just writing I guess what you'd call "AI generated code." But the code completion engine is basi

• insane_dreamer 2y ago
  also useful for writing unit tests, comments, descriptions, so if you count all of that as code, together with boilerplate stuff, then yeah, it could add up to 25%.
• ImaCake 2y ago
  This autocomplete seems about on par with github copilot. Do you also get options for prompting it on specific chunks of code and performing specific actions such as writing docs or editing existing code? All things that come standard with gh copilot now.
• hackerknew 2y ago
  I wondered if this the real context. i.e. They are just referring to code-completion as AI-generated code. But, the article seems like it is referring to more than that?
• ghostpepper 2y ago
  Does it make you 25% more productive?
  • vundercind 2y ago
    Between the fraction of my time I spend actually writing code, and how much of the typing time I’m using to think anyway, I dunno how much of an increase in my overall productivity could realistically be achieved by something that just helped me type the code in faster. Probably not 25% no matter how fast it made that part. 5% is maybe possible, for something that made that part like 2-3x faster, but much more than that and it’d run up against a wall and stop speeding things up.
    • imchillyb 2y ago
      I imagine that those who cherished the written word thought similar thoughts when the printing press was invented, when the typewriter was invented, and before excel took over bookkeeping. My productivity isn't so much enhanced. It's only 1%... 2%... 5%... globally, for each employee. Have you ever dabbled with, mucked around in, a command line? Autocomplete functions there save millions of man-hour-typing-units per year. Something to think about. A single employee, in a single task, for a single location may not equal much gained productivity, but companies now think on much larger scales than a single office location.
      • moron4hire 2y ago
        This is a fallacy because there is no way to add up 1% savings across 100 employees into an extra full time employee. Work gets scheduled on short time frames. 5% savings isn't enough to change the schedule for any one person. At most, it gives me time to grab an extra coffee. I can't string together "foregone extra coffees" into "more tasks/days in the schedule".
        • robertlagrant 2y ago
          This. I had the same conversation years ago with someone who said "imagine if Windows booted 30s faster, all the productivity gains across the world!" And I said the same thing you did: people turn their computer on and then make a cup of tea. Now making a kettle faster? That might actually be something.
  • rustcleaner 2y ago
    If 25% of code was AI-written, wouldn't it be a 33[.333...]% increase in productivity?
    • card_zero 2y ago
      Not if there was also an 8.333̅% increase in slacking off. Wait, no. That should be based on how much slacking off Google employees do ordinarily, an unknown quantity.
      • saagarjha 2y ago
        You can just check Memegen traffic to figure that one out.
    • PeterStuer 2y ago
      It is not a direct correlation. I might write 80% of the lines of code in a week, then spend the next 6 months on the remaining 20%. If the AI was mostly helpfull in that first week, overall productivity gain would be very low.
    • vundercind 2y ago
      Who spends 100% of their time actually typing code? It’s probably closer to 10% than 100%, especially at big companies. One thing I would love to see is reports of benefits from various tools coming with one’s typing ability in WPM. I’d also like to see that on posts where people express a preference for “a quick call” or stopping by your desk rather than posting what they want in chat. I have some hypotheses I’d like to test out.
• alxjrvs 2y ago
  In my day to day, this still remains the main way I interact with AI coding tools. I regularly describe it as "The best snippet tool I've ever used (because it plays horseshoes)".
  • tomcam 2y ago
    Horseshoes? As in “close enough”?
    • ttul 2y ago
      Or, as in, “Ouch, man! You hit my foot!”
      • goykasi 2y ago
        As long as hand grenades arent introduced, I could live with that.
        • DanHulton 2y ago
          Honestly, I don't think "close only count in horseshoes, hand grenades, and production code" will ever catch on...
          • alxjrvs 2y ago
            This is why I frame it as a "snippets" plugin, rather than a Code generation tool. I would be very confused if someone told me that they uncritically used the generated code from a snippet program with no manual input or understanding, and I feel the same with Copilot. At best, it suggests an auto-complete that I read and interpret before accepting. The closest I come to "code generation" is during test writing, where occasionally I will let the description generate some setup, but only in tests where there are a broad number of examples to follow, and I am still going to end up re-writing a decent chunk of it based on personal example. I would not "let it write the test suite for me" and then trust the green, and I suspect that would easily fail code review (though it would be an interesting experiment...). Obviously your comment as a good goof and well made, but it does speak to a little bit of the disconnect between what is being touted as an "AI coding tool" and how I, a person who makes react native apps to pay my rent, actually use the dang thing (i.e., "A pretty good snippets plugin"). Is My code 'AI generated'? I wouldn't call it that, but who can say definitively? We're in a fun new semantic world now.
• Galatians4_16 2y ago
  Kerry said hi
• nycdatasci 2y ago
  This is a great anecdote. SOTA models will not provide “engineering” per se, but they will easily double productivity of a product manager that is exploring new product ideas or technologies. They are much more than intelligent auto-complete. I have done more with side projects in the last year than I did in the preceding decade.
  • llm_trw 2y ago
    One of my friends put it best: I just did a months worth of experimentation in two hours.
    • Sateeshm 2y ago
      I find this hard to believe. Can someone give me an example of something that takes months that AI can correctly do in hours?
      • jvanveen 2y ago
        Not hours; but days instead of months: porting around 30k lines of legacy livescript project to typescript. Most of the work is in tweaking a prompt for Claude (using Aider) so the porting process is done correctly.
        • cdchn 2y ago
          Thankfully it seems like AI is best at automating the most tedious and arguably most useless endeavor in software engineering- rewriting perfectly good code in whatever the language du jour is.
          • disgruntledphd2 2y ago
            Again, what AI is good at shows the revealed preferences of the training data, so it does make sense that it would excel at pointless rewrites.
          • 2y ago
• napierzaza 2y ago
  [dead]
• afro88 2y ago
  So more or less on par with continue.dev using a local starcoder2:3b model
• OnionBlender 2y ago
  Do people find these AI auto complete things helpful? I was trying the XCode one and it kept suggesting API calls that don't exist. I spent more time fixing its errors than I would have spent typing the correct API call.
  • DecoySalamander 2y ago
    I was evaluating it for a month and caught myself regularly switching to an IDE with non-AI intellisense because I wanted code that actually works.
  • mu53 2y ago
    I find the simpler engines work better. I want the end of the line completed with focus on context from the working code base, and I don't want an entire 5 line function completed with incomplete requirements. It is really impressive when it implements a 5 line function correctly, but its like hitting the lottery
  • skybrian 2y ago
    I often delete large chunks of it unread if it doesn't do what I expected. It's much like copy and paste; deleting code doesn't take long.
    • card_zero 2y ago
      So your test is "seems to work"?
      • skybrian 2y ago
        No, what I meant is that, much like when copying code, I only keep the generated source code if it's written the way I would write it. (By "unread" I meant that I don't look very closely before deleting if it looks weird.) And then write tests. Or perhaps I wrote the test first.
        • card_zero 2y ago
• atoav 2y ago
  So this is basically the google CEO saying "a quarter of our terminal inputs is written by a glorified tab completion"?
  • asdfman123 2y ago
    Yes. Most AI hype is this bad. They have to justify the valuations.
    • remus 2y ago
      "tab completion good enough to write 25% of code" feels like a pretty good hit rate to me! Especially when you consider that a good chink of the other 75% is going to be the complex, detailed stuff where you probably want someone thinking about it fairly carefully.
      • Maxion 2y ago
        For me it's really goddam satisfying having good autocomplete, especially when you are just writing boilerplate lines of code to get the code into a state where you actually get to work on the fun stuff (ther harder problems).
        • amelius 2y ago
          Also if your code gets sent to someone else's cloud?
          • mewpmewp2 2y ago
            Have you ever had your code repository hosted by Github, Bitbucket, Gitlab or similar? If so, all your code is sent to cloud.
            • amelius 2y ago
              Answer: yes, some code. But other code I and my company like to keep private.
• simplyluke 2y ago
  This is exactly how I’ve used copilot for over a year now. It’s really helpful! Especially with repetitive code. Certainly worth what my employer pays for it. The general public has a very different idea of that though and I frequently meet people very surprised the entire profession hasn’t been automated yet based on headlines like this.
  • arisAlexis 2y ago
    Because you are using it like that doesn't mean that it can't be used for the whole stack and on its own and the public including laymen such as the Nvidia CEO and Sam think that yes, we (I'm a dev) will be replaced. Plan accordingly my friend.
    • robertlagrant 2y ago
      > Because you are using it like that doesn't mean that it can't be used for the whole stack Well no, but we have no evidence it can be used for the whole stack, whatever that means.
      • arisAlexis 2y ago
        Even last year's gpt4 could make a whole iphone app from scratch for someone that doesn't know how to code. You can find videos online. I think you are applying the ostrich method which is understandable. We need to adapt.
        • papichulo2023 2y ago
          Complexity increase over time. I can create new features in minutes for my new selfhosted projects, equivalent work on my entreprise work takes days...
          • arisAlexis 2y ago
            New Gemini has millions of context windows. Think big and project 1-2 years
• cryptica 2y ago
  This is my experience as well. LLMs are great to boost productivity, especially in the hands of senior engineers who have a deep understanding of what they're doing because they know what questions to ask, they know when it's safe to use AI-generated code and they know what issues to look for. In the hands of a junior, AI can create a false sense of confidence and it acts as a technical debt and security flaw multiplier. We should bring back the title "Software engineer" instead of "Software developer." Many people from other engineering professions look down on software engineers as "Not real engineers" but that's because they have the same perspective on coding as typical management types have. They think all code is equal, it's unavoidable spaghetti. They think software design and architecture doesn't matter. The problems a software engineer faces when building a software system are the same kinds of problems that a mechanical or electrical engineer faces when building any engine or system. It's about weighing up trade-offs and making a large number of nuanced technical decisions to ultimately meet operational requirements in the most efficient, cost-effective way possible.
• hgomersall 2y ago
  Before I go and rip out and replace my development workflow, is it notably better than auto complete suggestions from CoC in neovim (with say, rust-analyzer)? I'm generally pretty impressed how quickly it gives me the right function call or whatever, or it's the one of the top few.
  • Leherenn 2y ago
    It's more than choosing the right function call, it goes further than that. If your code has patterns, it recognises and suggests them. For instance, one I find very useful is that we have this pattern of checking the result of a function call, logging the error and returning, or whatever. So now, every time you have `result = foo()`, it will auto suggest `if (!result) log_error...` with a generally very good error message. Very basic, but damn convenient. The more patterns you use, the more helpful it becomes.
• karmasimida 2y ago
  I can totally see it. It is actually a testament that, part of Google's code are ... kinda formulaic to some degree. Prior to the LLM take over, we already heard praise how Google's code search works wonder in helping its engineer writing code, LLM just brought that experience to next level.
• davedx 2y ago
  I'm working on a CRM with a flexible data model, and ChatGPT has written most of the code. I don't use the IDE integrations because I find them too "low level" - I work with GPT more in a sort of "pair programming" session: I give it high level, focused tasks with bits of low level detail if necessary; I paste code back and forth; and I let it develop new features or do refactorings. This workflow is not perfect but I am definitely building out all the core features way faster than if I wrote the code myself, and the code is in quite a good state. Quite often I do some bits of cleanup, refactorings, making sure typings are complete myself, then update ChatGPT with what the code now looks like. I think what people miss is there are dozens of different ways to apply AI to your day-to-day as a software engineer. It also helps with thinking things through, architecture, describing best practices.
  • littlestymaar 2y ago
    I share your sentiment, I've written three apps where I've used language models extensively (a different one for each: ChatGPT, Mixtral and Llama-70B) and while I agree that they where immensely helpful in terms of velocity, there are a bunch of caveats: - it only works well when you write code from scratch, context length is too short to be really helpful for working on existing codebase. - the output code is pretty much always broken in some way, and you need to be accustomed to doing code reviews to use them effectively. If you trust the output and had to debug it later it would be a painfully slow process. Also, I didn't really noticed a significant difference in code quality, even the best model (GPT-4) write code that doesn't work, and I find it much more efficient to use open models on Groq due to the really fast inference. Looking at ChatGPT slowly typing is really annoying (I didn't test o1 and I have no interest in doing so because of its very low throughput).
    • davedx 2y ago
      > context length is too short to be really helpful for working on existing codebase. This is kind of true, my approach is I spend a fairly large amount of time copy-pasting code from relevant modules back and forth into ChatGPT so it has enough context to make the correct changes. Most changes I need to make don't need more than 2-3 modules though. > the output code is pretty much always broken in some way, and you need to be accustomed to doing code reviews to use them effectively. I think this really depends on what you're building. Making a CRM is a very well trodden path so I think that helps? But even when it came to asking ChatGPT to design and implement a flexible data model it did a very good job. Most of the code it's written has worked well. I'd say maybe 60-70% of the code it writes I don't have to touch at all. The slow typing is definitely a hindrance! Sometimes when it's a big change I lose focus and alt-tab away, like I used to do when building large C++ codebases or waiting for big test suites to run. So that aspect saps productivity. Conversely though I don't want to use a faster model that might give me inferior results.
• NotAnOtter 2y ago
  I also work at google (until last Friday). Agree with what you said. My thoughts are 1. This quote is clearly meant to exaggerate reality, and they are likely including things like fully automated CL/PR's which have been around for a decade as "AI generated". 2. I stated before that if a team of 10 is equally as productive as a team of 8 utilizing things like copilot, it's fair to say "AI replaced 2 engineers", in my opinion. More importantly, Tech leaders would be making this claim if it were true. Copilot and it's clones have been around long enough know for the evidence to be in, and no one is stating "we've replaced X% of our workforce with AI" - therefor my claim is (by 'denying the consequent'), using copilot does not materially accelerate development.
  • onion2k 2y ago
    no one is stating "we've replaced X% of our workforce with AI" That's only worth doing if you're trying to cut costs though. If the company has unmet ambitions there's no reason to shrink the headcount from 10 to 8 and have the same amount of output when you can keep 10 people and have the output of 12 by leveraging AI.
    • hyperpape 2y ago
      Almost all the big tech companies have had layoffs over the past several years. I think it’s safe to say cost cutting is very much part of their goal.
      • deleted 2y ago
        [deleted]
      • lupire 2y ago
        But the specific roles being laid off are arbitrary, and the overall goal headcount reduction is driven by macroeconomics factors (I'm being generous there), not based on new efficiencies. Note the difft between "cost cutting" (do less, to lower cost) and "efficiency" (do same, but with less cost)
      • theptip 2y ago
• nlehuen 2y ago
  I also work at Google and I agree with the general sentiment that AI completion is not doing engineering per se, simply because writing code is just a small part of engineering. However in my experience the system is much more powerful than you described. Maybe this is because I'm mostly writing C++ for which there is a much bigger training corpus than JavaScript. One thing the system is already pretty good at is writing entire short functions from a comment. The trick is not to write: function getAc... But instead: // This function smargls the bleurgh // by flooming the trux. function getAc... This way the completion goes much farther and the quality improves a lot. Essentially, use comments as the prompt to generate large chunks of code, instead of giving minimum context to the system, which limits it to single line completion.
  • miki123211 2y ago
    This is a good idea even outside of Google, with tools like copilot and such. Often when I don't know exactly what function / sequence of functions I need to achieve a particular outcome, I put in a comment describing what I want to do, and Copilot does the rest. I then remove the comment once I make sure that the generated code actually works. I find it a lot less flow-breaking than stackoverflow or even asking an LLM. It doesn't work all of the time, and sometimes you do have to Google still, but for the cases it does work for, it's pretty nice.
    • Aachen 2y ago
      Why remove the comment that summarises the intent for humans? The compiler will ignore your comment anyway, so it's only there for the next human who comes along and will help them understand the code
      • cryptonym 2y ago
        Next human will put the code in a prompt and ask what it does. Chinese Whispers.
        • Aachen 2y ago
• heresie-dabord 2y ago
  "Our overhyped Autocomplete Implementation (A.I.) is completing 25% of our lines of code so well that we need to fund nuclear reactors to power the server farms."
• znpy 2y ago
  > If I'm writing "function getAc..." it's smart enough to complete to "function getActionHandler()", and maybe suggest the correct arguments and a decent jsdoc comment. I really mean no offense, but your example doesn't sound much different from what old IDEs (say, Netbeans) used to do 15 years ago. I could design a Swing ui and it would generate the code and if I wanted to override a method it would generate a decent boilerplate boilerplate (a getter, like in your example) along with usual comments and definitely correct parameters list (with correct types). Is this "AI Code" thing something that appears new because at some point we abandoned IDEs with very strong intellisense (etc) ?
  • hoveringhen 2y ago
    This video is a pretty good one on how it works in practice: https://storage.googleapis.com/gweb-research2023-media/media/AIforSE-ExampleIDE3.mp4 https://storage.googleapis.com/gweb-research2023-media/media...
• red_admiral 2y ago
  A few years ago we called that IntelliSense, right? I remember many years ago as a Java developer, Netbeans could do such things as complete `psvm` to "public static void main() {...}", or if you had a field "private String name;" you could press some key combination and it would generate you the getter and setter, complete with javadoc which was mandatory at that place because apparently you need "Returns the name.\n @return The name." on a method called getName() in case you wondered what it was for.
  • _heimdall 2y ago
    Yes, Copilot and other LLM coding tools are just a (much) better version of IntelliSense.
    • snowe2010 2y ago
      Much worse imo.
      • _heimdall 2y ago
        That could be too. I don't use LLMs so I'm just giving it the benefit of the doubt based on other commentors here.
  • skydhash 2y ago
    Most jetbrains IDEs come with those snippets and if you’re using IDEA, the code will be 50%+ generated by the IDE.
  • rty32 2y ago
    I think most people define "Intellisense" as "IDE suggestions based on static anaysis results". Sometimes it blends a bit of heuristics/usage statistics as added feature depending on the tool. They are mostly deterministic, based on actual AST of your code, and never hallucinates. They may not be helpful but can never be wrong. On the other hand, LLMs are completely different -- based on machine learning and everything is random and about statistics. It depends on training data and context. It is more useful but make a ton of mistakes.
• aforty 2y ago
  We have this at our company too. I guess it’s useful but doesn’t really have a whole lot of time.
• markstos 2y ago
  Which editor is Google's AI code completion integrated with? VS Code?
  • hoveringhen 2y ago
    Yeah
• jb1991 2y ago
  Long before this current AI hype cycle, we’ve had excellent code completion in editors for decades. So I guess by that definition, we’ve all been writing AI assisted code for a very long time.
  • fhd2 2y ago
    I'd say so, and it's a bit misleading to leave that out. Code generation is almost as old as computing. So far, most of it happened to be deterministic.
  • player1234 2y ago
    Yeah but it didn't cost trillions and needed its own nuclear power plant. Noone disputes that llm/ai is cool/can be helpful but at what cost, where is the roi?
• grecy 2y ago
  I'm confused, I've been doing similar tab completion for function names in eclipse since about 2003...
• bambax 2y ago
  Thank you for this comment. So the code written in this manner isn't really "created by AI"; AI is just a nice additional feature of an editor. I wonder if the enormous hype around AI is a good or bad thing; it's obviously both but will the good win out the bad, or will the disappointment eventually be so overwhelming as to extinguish any enthusiasm.
• theodric 2y ago
  I wonder how this works with IP rights in the USA. Like, is `function getAc` eligible for copyright protection, but `tionHandler()` isn't? After all, [1] [1] https://www.reuters.com/legal/ai-generated-art-cannot-receive-copyrights-us-court-says-2023-08-21/ https://www.reuters.com/legal/ai-generated-art-cannot-receiv...
• awkward 2y ago
  Stuff that works well with AI seems to correlate pretty well with high churn changes. I've had good luck using AI to port large numbers of features from version A to version B, or getting code with a a lot of dependencies under mocked unit tests. It's easy to see that adding up quickly to represent large percentages of the codebase by line, but it's not feature development or solving hard problems.
  • blindhippo 2y ago
    Same things I use it for as well - crap like "update this class to use JDK21" or "re-implement this client to use AWS SDKv2" or whatever. And it works maybe... 80% of the way and I spend all my time fixing the remaining 20%. Anecdotally I don't "feel" like this really accelerates me or reduces the time it would take me to do the change if I just implemented the translation manually.
    • awkward 2y ago
      Amazon is publicly claiming that they have saved hundreds of millions on jvm upgrades using AI, so while it feels trivial - because before that work would end up in the "just don't do it" pile - it's a relevant use case.
• josh_carterPDX 2y ago
  My first reaction to the title was, "That explains why things are broken." but this explanation makes so much sense. Thanks for clarifying. But yeah, I wish the new version of Chrome worked better. ¯\_(ツ)_/¯
• jszymborski 2y ago
  Sounds like the JetBrains new local AI autocomplete. If it's anything like that, it's honestly my ideal application of generative deep learning.
• prismatic-david 2y ago
  This is pretty much what I've found with Copilot as well. It's like a slightly smarter autocomplete in most cases. Copilot tends toward being a little eager sometimes, but it's easy enough to just ignore the suggestions when it starts going down a weird path.
• peepee1982 2y ago
  That's what I thought. In recent weeks, most of the code I’ve written has been AI-generated. But it was mostly JSDoc comments, type checking (I'm writing JavaScript), abstracting code if I see that I'm repeating myself a little too often, etc. All things that I would consider tedious housekeeping, but nothing that needs serious reasoning. It's basically a glorified LSP.
  • inanepenguin 2y ago
    I know you're not saying anything revolutionary but this is the best succinct yet fair description of these tools that I've seen. They're not worthless but they're not job destroying.
    • peepee1982 2y ago
      You're right, it's not revolutionary at all. But I'm glad you liked my summary!
• segasaturn 2y ago
  How do you square this comment with the one right below it[1], which explicitly confirms the statement that Google is using GenAI via Gemini to write code? Lots of mixed signals coming from the Googlers here. 1: https://news.ycombinator.com/item?id=41992028 https://news.ycombinator.com/item?id=41992028