8 ms·
New Capabilities for GPT-3: Edit and Insert
- Smaug123 5y agoI find the autocompletion for "improve the runtime complexity of the [Fibonacci] function" at the top excruciating. Surely Codex has seen verbatim the two-argument form many times?
- zackees 5y agoThe open ai project CoPilot is just absolutely amazing. I would say that it increases my productivity by 5x because it eliminates a lot of going to stack overflow to find the answer.
- jarbus 5y agoDoes anyone actually use Copilot for their work? I can't imagine it's anywhere near as reliable as OpenAI claims. I'd imagine a user would spend more time fixing mistakes or re-trying with different queries than they'd actually save.
- unwoundmouse 5y agoI use copilot, it's much more useful than you'd expect. Really helpful for places where you would normally need to record a small macro, copilot can infer the completions easily
- drusepth 5y agoI've used it briefly in someone else's IDE (who swears by it) and it blew me away. It pretty much removed ever having to google syntax or snippets from SO in a language I wasn't totally familiar with (python).
- armchairhacker 5y agoAt first I thought copilot would be pretty useless until I actually tried it. It turns out that a lot of code is boilerplate and the same simple patterns, even with abstraction. Copilot is not particularly genius, but it fills in simple patterns (e.g. do the same for the Y-axis that you did for the X-axis), and autocompletes typical utility functions (e.g. add 2 2d positions, shuffle an array, setTimeout promise, etc. which I have to write functions for because they are not in the JavaScript standard library.) These may seem like odd scenarios but there are actually a lot of them.
- gavinray 5y agoYeah. I thought the same thing at first, had it for ~2 months in early access and never turned it on. Now I can't live without it.
- rictic 5y agoI use it all the time, it's significantly improved my productivity. It's a little like pair programming with an incredibly eager junior developer who has read a lot of the documentation of every popular API in the world. I need to review the code it produces, but it's very fast, and its suggestions are usually great. It's annoying when I know exactly what I want to write, and most helpful when I'm unsure (either because I'm trying things out, or if I'm using a new API or a language I'm rusty at).
- Traster 5y agoThis seems like a Google problem. Google was really helpful when it had a tonne of organic content that it could systematically steal from, plagiarise, and rip off to the point of devaluing the entire internet. The result with google was absolute centralization of that content and therefore a withering of the organic content. I wonder if we're going to see the same thing with these AI tools. It's fine to learn from 100,000 developers all writing code. But if you steal their IP, rip off their designs and plagiarise their work to build your tool what you end up with is a tool that is basically just learning from itself. No one really writes basic code anymore, but as a result there's no source for the AI to learn from. In other words, it centralizes knowledge but doesn't advance knowledge, and in the process it devlues anyone else advancing knowledge.
- jjwiseman 5y agoI use it constantly and hope I never have to code without it (or something better) again. It does a good job writing the kind of boring code I don't want to write, and it generally seems to include fewer errors than I write in my first drafts of code. More than once I ignored its suggestion and wrote my own version, only to later realize its version was more correct and efficient. It does a good (sometimes incredible) job of even handling pretty specialized subject matter, and of using the context of other code and comments you've written to suggest exactly what you need next.
- amelius 5y ago> It does a good job writing the kind of boring code I don't want to write (...) If only we had programming languages that didn't force us to write boring code in the first place ...
- deleted 5y ago[deleted]
- jjwiseman 5y agoMaybe once you eliminate one level of "boring" code, that just means parts of the next-higher level of code become rote and boring. It reminds me a little of Richard Gabriel's reply to Guy Steele, when Steele said something like "Lisp doesn't need design patterns; it has macros." Gabriel said "That just moves the patterns up a level of abstraction." (I probably remember that story all wrong. But I like it anyway!)
- eru 5y agoOne of the problems with (classic) Lisp macros is that they aren't first class, ie you can pass them around like you can do with functions (or numbers etc).
- deleted 5y ago[deleted]
- mark_l_watson 5y ago
- powersnail 5y agoOne thing that it’s really good at is writing boilerplate-y code. For example, web scrapers. It can even read the function’s name and deduce some proper variable names, or use variable names to deduce whether I want a list of elements or one element. Not 100% correct, of course, but good enough if you treat it like an advanced snippet manager.
- davidbarker 5y agoI’ve been using it every day for a few months (for Typescript/React), and it still astounds me. I can write a comment outlining what I want a function to do, and 90% of the time it will generate the code I need (or very close with a couple of small tweaks needed). Coincidentally, my Stack Overflow visits have decreased by approximately 90%.
- nlh 5y agoI was going to write exactly this and I'm glad you beat me to it (and that someone else is experiencing this benefit). It's obviously not a replacement for a human writing a big program (yet...), but it does a damn fine job of giving you a shell of a function based on a comment. PS - it is also AMAZING with CSS and saves so much time on easy things that I just haven't memorized. "Style the LI so there are no bullet points" and boom...'list-style: none;'
- davidbarker 5y agoGreat to hear we’re having similar experiences. It’s also quite impressive how Copilot will learn from any patterns you’ve typed on previous lines — so when I’m using a certain color name in a variable, it knows that I likely want to use it in subsequent code. Or, more concretely, when using tokens for colors (e.g. blue.50 for light blue, blue.900 for dark blue) it can figure out that I probably want my background to be an x.50 colour and my text to be, say, x.700. So cool!
- deleted 5y ago[deleted]
- IanCal 5y agoIt really is surprisingly good. I use it. It's quick to scan and ignore things that aren't right, and it's either completely right or close enough that it definitely feels like a timesaver. The best parts are where it's doing something long-winded but fairly straight forward (e.g. assigning variables). But it has moments of shocking ability with more complex things.
- superasn 5y agoBig supporter of Copilot. I am still amazed how good it is and I feel it's getting better and better. So much boilerplate code gone. Also it really gives you a boost in confidence when the A.I writes the same code you're thinking. I feel so lucky having to see these amazing developments in A.I and V.R. recently.
- layer8 5y ago> boost in confidence when the A.I writes the same code you're thinking. I would have expected the demonstration that your work is easily replaced by an AI to have the opposite effect. :)
- karmasimida 5y agoFor boilerplate code, Copilot gets the job done. But you need to set the right expectations that it is not magic, which requires some tuning.
- nyanpasu64 5y agoJust wanted to have a less glowing counterpoint to the other claims. I've used Copilot a bit, and found that the automatic completion was frequently interrupting my train of thought, making it harder to concentrate ("intrusive thoughts as a service"). I preferred only triggering completions upon pressing a keystroke, so I choose when to take a shortcut and ask to have code generated. I found it very helpful for generating boilerplate code, and debug logging I shouldn't have to think too hard about. Also it sometimes gave me clever ideas better than I thought of myself (like Rust code matching on a HashMap's Entry). Nonetheless I felt uneasy because I noticed myself getting too "lazy", not thinking about what code I want written before asking for help. In the end, aside from boilerplate, I spend most of my time in Qt Creator (which doesn't have a Copilot plugin) rather than VS Code, so I mostly stopped using Copilot anyway.
- rjh29 5y agoI often have to turn it off. It's very useful when I know what needs to be typed but am too lazy. It's also sometimes amazing useful at coding stuff from a comment, and like you say it can teach you new idioms. It would be a great way of learning a new language from examples. When I'm problem solving or not sure how to code something though, its constant suggestions are just noise, and then I turn it off.
- zoba 5y agoIn my experience copilot is amazing. It prompted the (joke) thought that perhaps it is making me less productive because of how often I end up sitting back and marveling at how amazing it is. I really can’t believe how good it is.
- muzani 5y agoPersonally using Codex, not Copilot, but similar engine. It's really good for boilerplate. Things like TDD tests where I'm just modifying a few parameters. You can get it to write functions like "parse this DateTime object into a format like Tuesday, 15 May 2020". It's a useful lookup too. Like often I just want to extract a variable from a List and would spend 15 mins looking up the docs or sifting Stack Overflow. Codex is more faster and accurate. With GPT-3 it's garbage in, garbage out. You have to invest a few days in learning the prompts that work.
- morgante 5y agoWhat's your workflow for using Codex directly? Are you copying over to the playground or using a custom extension?
- muzani 5y agoYeah, just copying to a preset on the playground. Whenever I try to reach out to Stack Overflow or get stuck on documentation, I'd open up Codex. It's possible to set up an API to it, but it's not yet at the point where it's worth doing. I think it's possible to get Codex to write the script, now that I think about it.
- csomar 5y agoI use it with Rust, and my only grief is that it only takes the current file as context. It is dumb but very good at doing repeating things, or being a very smart auto-complete. For writing tests, it is actually quite decent.
- justinlloyd 5y agoI've tried Copilot, Tabnine, and a couple of the others out there. I find them to be an annoying Clippy-like companion that interrupts my train of thought and introduces a whole separate class of programming bugs: "autocomplete errors" which are like copy & paste errors, but written by another programmer. I like my auto-complete functionality built in to Jetbrains and Visual Studio for hinting at variable names, function parameters, classes, imports, and so forth. But the boiler-plate code that "assistant AI programmers" provide are not worth the effort at this time and do not live up to the hype. I think they help less experienced developers out, but the kind of code I am writing won't usually be found in Copilot or Tabnine. I could see the appeal if all I was doing was churning out boilerplate CRUD app code all day long, which honestly, I would outsource to Craigslist for thirty bucks an hour. Or just hand off to my assistant.
- zackees 5y ago
- deutschew 5y agoI'm on track to surpass 1 million euros in salary working for 4 different companies thanks to Copilot so yes, this has been a huge boon for me.
- surrTurr 5y agoIt’s too good. I found myself getting very lazy with coming up solutions on my own, it felt like my problem solving capabilities declined. IMO you should install, and then disable it. Only use it sparingly, when time is very limited or to write tests/docs.
- BeFlatXIII 5y agoHow polyglot is Copilot? I see plenty of positive results in the replies for (Java|Type)Script and Python, but how does it fare with a Lisp-like language or Prolog (for examples)?
- minimaxir 5y agoI did a few nonscientific tests: https://twitter.com/minimaxir/status/1503822287903985664 https://twitter.com/minimaxir/status/1503822287903985664 Both the edit endpoint (used in those tweets) and the insert endpoint have mixed performance and tend to go off-the-rails often, especially compared to the new "Instruct" models which do a much better job, although expensive while these new endpoints are in a free beta. The coding endpoints are slightly better but a more narrow domain. In all cases I recommend looking at the docs for examples.
- drusepth 5y agoReally, really happy they're enabling the edits endpoint as a free-to-use beta for now. In my experience, GPT-3 works really well but is pretty prohibitively expensive unless you're optimizing for profitable tasks. Offering even limited-time free use (and unlimited tokens) is a nice nudge toward the "open" in OpenAI. Also, the edit/insert endpoints specifically should hypothetically help a lot with plot divergence, which has been a huge problem trying to generate long-form works with standard completions, even with top-down outline expansion strategies, scene transition metadata, etc. Excited to see what the millions of "AI word processors" that've sprung up over the past year actually do with it, besides the obvious.
- deutschew 5y agoif you are already using copilot, do we get these new feature automatically?
- samwillis 5y agoI’m on the copilot wait list, super excited to give it a go. Cheeky question, does anyone know of a way to get it sooner?
- davidbarker 5y agoTry emailing the GitHub CEO. I did, regarding the Codespaces waitlist, and he sent me three riddles to answer correctly before I could get access. (I realise this sounds like I’m making it up, but I promise this is a real story. It was quite fun.)
- falcor84 5y agoI'm now curious to know whether Copilot could correctly answer those riddles
- davidbarker 5y agoBrilliant thought. I’ll have to dig the emails out and test it.
- jhgb 5y agoSo...what is your favourite color?
- andybak 5y agoI used it today for some javascript DOM manipulation (in which I'm a bit rusty). It definitely saved me time. But it wasn't a panacea. It I didn't have a good eye and some background understanding then I wouldn't have got working code. But it definitely saved me time. It's won't write your software for you. It's more like autocomplete but much, much better.
- andybak 5y agoI just gave it the opening paragraph of Pride & Prejudice in edit mode and the instruction to "rewrite it as a rap": When Jane saw mister Bingley, She knew that he was fine, But it took her a while to realize He wanted to be more than friends, Then she met Fitzwilliam Darcy, And she thought he had a lot of money, But it took her a while to realize He didn't want to talk about money, She was just a young girl, But she knew what was going on, She knew who she wanted to be with, But she also had her doubts, She knew he was great, But she also thought he was a snob, She was afraid that, He was going to break her heart. She loved him, But didn't know if she could trust him, So she decided to wait, Until his feelings became clear. Then all of a sudden, She realized, That she had been waiting all this time, And that she was in love. So she decided to go after him, And she got to him just in time, And they lived happily ever after. It's a terrible rap but a fairly impressive summary of the book (from someone that's admittedly never read it) Odd that it took "rewrite (the opening paragraph) as a rap" to mean "summarise the whole book".
- snek_case 5y agoIt might have memorized a summary of the book from the data it's been trained on. GPT-3 is impressive, but it's kind of hard to control exactly what it's going to do. It recognized what you input was from pride and prejudice, remembered a summary of the book from somewhere, and turned that into a rap, which is both impressive and not really what you asked for.
- oofbey 5y agoDeep learning theory has shown that these huge neural networks can memorize basically any dataset they're given. The useful ones find ways to interpolate everything they've memorized to new contexts.
- arjvik 5y agoNot that it's easy to quantify, but do we know if the interpolation tends to happen at training time or inference time?
- BurningFrog 5y agoIs there a way I as a random civilian can play around with GPT-3 online?
- teaearlgraycold 5y agoWe evaluated the suffix capabilities (insert) at CopyAI and determined it's not ready for us to use in production. It's a great idea and I think they'll improve it with time. But it's not up to the quality bar of Davinci completions.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- upperb0ound 5y agoI work at OpenAI. Since Insert is still in beta we really appreciate the feedback! We've found that it's easier to get Insert to work in production settings for code than for natural language. For natural language which I think you are more interested in, we documented some best practices that can help the users succeed here: https://beta.openai.com/docs/guides/completion/inserting-text https://beta.openai.com/docs/guides/completion/inserting-tex... I'd especially recommend trying our new and improved best_of feature.
- teaearlgraycold 5y agoThanks a bunch! Absolutely not trying to downplay the massive undertaking that went into the suffix parameter. And it should have gone without saying that expectations should be tempered given it's still a beta feature.
- spupe 5y agoAs usual, they give us a press release and two or three examples, but no real data or systematic analysis about its consistency, which is the real problem with any of these models. I'm excited about the potential of things like this, I just don't like this wish-wash way of presenting a technical tool.
- rosenjcb 5y ago"OpenAI" except everything is closed research. Why did they even pick this name?
- momenti 5y agoTheir initial intention was possibly genuine, but there are strong incentives for secrecy (trade secrets), in addition to the U.S. military most likely having an interest in suppressing information to avoid someone else creating AI first.
- 0ldskool 5y agothe U.S. military most likely having an interest in suppressing information to avoid someone else creating AI first. someone else creating ai first? You make it sound like they are trying to create general ai skynet... there are strong incentives for secrecy (trade secrets) they do not have the right to call themselves "open" if they are focusing on secrecy. They are trying to paint themselves in a positive opensource light when really they are no different then any other corporate overlord hiding in the shadows. Open implies developed in the open, free for use + modification, and/or key details NOT being hidden from the public. Simply sharing details of things created or research done is no different than what researchers at any other ABC corporation/organization do.
- yoyopa 5y agomaybe fix the racism in your current ai before adding more stuff to it?
- lelag 5y agoI've been playing with the edit beta mode in the playground using the alpha text-davinci-edit-001 model but it does not appear to be free as the article states: all the calls made to the edit endpoints were charged to my account. As larger requests (ex: spellcheck or translate 1000 words) can cost 20c+ each and the model often requires to be ran 2 or 3 times before giving an acceptable answer on more complex tasks (ex: translation), that makes for a very expensive tool if you want to edit a significant amount of text. I wonder if the free usage only apply to the Codex engine and not the GPT-3.
- wrongraspberry 5y agoI've tried using the new codex engine (code-davicii-002) and found that it costs money. OpenAI's playground says the codex is free but that doesn't seem to be the case right now. I'm not sure if this is a bug or intentional.
- npew 5y agoI work at OpenAI. This was unfortunately a bug, and we've fixed it since.