10 ms·

I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And i

I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And it was basically useless when trying to deal with anything non-trivial or anything to do with SQL (even when I frontloaded it with my entire schema). It was much less effort to just write everything myself because I actually know what I want to write and fixing my own mistakes was easier than fixing the bot’s. I weep for the juniors that will be absolutely crushed by this garbage.

oldpersonintx 3y ago
[dead]
teaearlgraycold 3y ago
Really? I've been doing web dev as a hobby for 20 years and professionally for 6 or 7 years. It's super helpful for me giving how much boilerplate there is to write and how much of the training set is for websites. Any time I try to get it to write non-trivial SQL or TypeScript types it fails hard. But it's still a nice time saver for writing tests, request handling, React boilerplate, etc.
- louthy 3y ago
  This is the problem. As programmers we should be focusing effort on reducing boilerplate, so that it’s never needed again. Instead, we’ve created a boilerplate generator.
  - CuriouslyC 3y ago
    The challenge there is that reducing boilerplate comes with opinionated magic, which rubs a lot of people the wrong way. So the creators add configs and code level config, and pretty soon you're back in boilerplate hell.
  - teaearlgraycold 3y ago
    Nope. The wrong abstraction is too risky vs. having no abstraction. AI can let us move towards an abstraction-free mode of coding. In the future you’ll tell the AI “add a 60 second cache to every source of X data” and it’ll scour your source code for matching code and you’ll review the diffs. C programmers would review LLVM IR diffs. Web devs will review whatever compilation target they’re using. We use high level languages because they improve reading comprehension and save us time when writing. Having a copilot take on a major role allows us to fundamentally rethink programming.
    - norir 3y ago
      I find it unlikely that we would be asking the ai for a very specific caching design. If the optimists are right, the ai should be able to do its own analysis and automatically design an architecture that is superior to what a human would be able to do. A job spent mostly reviewing ai generated diffs sounds like a level of hell beyond even Dante's imagination.
    - deleted 3y ago
      [deleted]
    - louthy 3y ago
      > We use high level languages because they improve reading comprehension and save us time when writing. Which completely contradicts your earlier point. Why not just get copilot write assembly for you? Or, spit out raw machine code? Oh, that’s right, because you need to check that it hasn’t fucked something up. Which, when there’s a ton of boilerplate, is hard. It’s arguable that programming language evolution stopped around the time Java was released (barring a few sprinkles here and there, like async/await, affine/linear types, etc.) We haven’t had a major leap in language power for decades (not like the leap from assembly to procedural languages) - I believe it’s because Java popularised evolution through libraries - and for a long time that was fine, even if it did lead to language evolution stagnating. But now we’ve hit a complexity threshold that demands an abstraction leap, but instead of looking for that abstraction leap we’re getting a computer to generate boilerplate for us, hoping it will dig us out of the complexity hole. We’re still ways off having a computer maintain a complex code base over many years. So humans still have to do that. It’d be much easier if we remove the incidental complexity. “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger W. Dijkstra
  - deleted 3y ago
    [deleted]
  - Aeolun 3y ago
    I’m starting to lean in the direction that trading some boiletplate for flexibility is a good deal.
Syntaf 3y ago
As with most things in life, moderation is key. I find co-pilot primarily useful as an auto-complete tool to save keystrokes when writing predictable context driven code. Writing an enum class in one window? Co-pilot can use that context to auto complete usage in other windows. Writing a unit test suite? Co-pilot can scaffold your next test case for you with a simple tab keystroke. Especially in the case of dynamic languages, co-pilot nicely compliments your intellisense
- thehours 3y ago
  My experience is similar. This week I created a few DBT models and Co-Pilot saved a *ton* of keystrokes on the YML portion. It needed some hand-holding in the early parts, but it was so satisfying to tab autocomplete entire blocks of descriptions and tests once it picked up context with my preferences.
- choilive 3y ago
  Yep, treat copilot like a really good autocomplete system and its wonderful. Saves a lot of time and typing even in languages that aren't known for having a lot of boilerplate. Treat copilot to solve actual problems (y'know, the kind of stuff you are presumably paid to solve), and it falls completely flat.
  - craig 3y ago
    I've found it's autocomplete really bad. It often hallucinates import paths and other illegal operations. It's not clear to me why it can't leverage the LSP better.
  - Aeolun 3y ago
    It’s also nice for solving small things like ‘euclidian distance’
  - deleted 3y ago
    [deleted]
ben_w 3y ago
> I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And it was basically useless when trying to deal with anything non-trivial or anything to do with SQL (even when I frontloaded it with my entire schema). Good to know, that means I'm still economically useful. I'm using ChatGPT rather than Copilot, and I'm surprised by how much it can do, but even so I wouldn't call it "good code" — I use it for JavaScript, because while I can (mostly) read JS code, I've spent the last 14 years doing iOS professionally and therefore don't know what's considered best practice in browser-land. Nevertheless, even though (usually) I get working code, I can also spot it producing bad choices and (what seems like) oddities. > I weep for the juniors that will be absolutely crushed by this garbage. Indeed. You avoid the two usual mistakes I see with current AI, either thinking it's already game over for us or that it's a nothing-burger. For the latter, I normally have to roll out a quote I can't remember well enough to google, that's something along the lines of "your dog is juggling, filing taxes, and baking a cake, and rather than be impressed it can do any of those things, you're complaining it drops some balls, misses some figures, and the cake recipe leaves a lot to be desired".
- varispeed 3y ago
  > I'm surprised by how much it can do, but even so I wouldn't call it "good code" You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver.
  - ben_w 3y ago
    > You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver. Yes, I've even done that to see how well it works (my experience is that half the time it fixes the code, the other half it claims to but doesn't) — but I can do that because I can recognise that it's bad code. I don't expect a junior to notice bad code, and therefore I don't expect them to ask for a fix. (I've also spent 6 months in a recent job just fixing code written by work experience students).
l5870uoo9y 3y ago
This is the problem with using AI for generating SQL statements; it doesn't know the semantics of the your database schema. If you are still open for a solution, I recently deployed a solution[1] that combines AI, db schema and simple way train AI to know your database schema. Essentially, you "like" correct (or manually corrected) generations and a vectorized version is stored and used in future similar generations. An example could be tell which table or foreign key is preferred for a specific query or that is should wrap columns in quotes. From my preliminary tests it works well. I was able to consistently make it use correct tables, foreign keys and quotes on table/column name for case-sensitivity using only a couple of trainings. Will open a public API for that soon too. [1]: https://www.sqlai.ai/ https://www.sqlai.ai/
- tilwidnk 3y ago
  It's "a public API", not "an public API", because of the consonant rule. I really worry that there are people out there who will anxiously mangle their company's data thinking what is being called AI, which doesn't exist yet, will save the day.
  - l5870uoo9y 3y ago
    Use it as a tool not a replacement. However it does do things well even without much additional information like fixing SQL statement[1]. That being said it is consistently improving, GPT-3.5 to GPT-4 was a major upgrade. [1]: https://www.sqlai.ai/snippets/clroq0qn9001xqzqeidtm4jgx https://www.sqlai.ai/snippets/clroq0qn9001xqzqeidtm4jgx
- CrimsonRain 3y ago
  Just copy paste the db/table schema in a comment at start of your file. Nothing else needed.
godelski 3y ago
I have similar sentiment and looking at how mixed takes are, I think it depends on what you do. I write a lot of research code so I think it's unsurprising that GPT isn't too good here. But people I see that write code more in line with the "copy and paste from stackoverflow" style, get huge utility out of this. (This isn't a dis on that style, lots of work is repetitive and redundant). So I changed how I use GPT (which I do through API. Much cheaper btw). I use it a lot like how I would use SO in the first place. Get outlines, understand how certain lines might work (noisy process here), generate generic chunks of code especially from modules I'm unfamiliar with. A lot of this can just be seen as cutting down time searching. So, the most useful one: using it as a fuzzy search to figure out how to Google. This one is the most common pattern. Since everything on Google is so SEO optimized and Google clearly doesn't give a shit, I can ask GPT a question, get a noisy response that contains useful vernacular or keywords which I can then use to refine a Google search and actually filter out a decent amount of shit. I think people might read this comment and think that you should just build a LLM into Google, but no, what's going on is more complicated and requires the symbiosis. GPT is dumb, doesn't have context, but is good at being a lossy compression system. The whole reason this works is because I'm intelligent and __context aware__, and importantly, critical of relying on GPT's accuracy[0]. Much of this can't be easily conveyed to GPT and isn't just a matter of token length. So that said, the best way to actually improve this system is actually for Google to just get its shit together or some other search engine to replace them. Google, if you're listening, the best way you can make Google search better with LLMs is to: 1) stop enabling SEO bullshit, 2) throw bard into the side and have the LLM talk to you to help you refine a search. Hell, you can use a RL agent for 1 to just look how many times I back out from the links you send me or look at which links I actually use. Going to page 2 is a strong signal that you served shit. [0] accuracy is going to highly depend on frequency of content. While they dedupe data for training, they don't do great semantic deduping (still an unsolved problem. Even in vision). So accuracy still depends on frequency and you can think of well known high frequency knowledge as having many different versions, or that augmentation is built in. You get lower augmentation rates with specific or niche expert knowledge as there's little baked in augmentation and your "test set" is much further from the distribution of training data.
marcyb5st 3y ago
As other said I use copilot or similar for scaffolding/boilerplate code. But you are right, reading almost correct code that you are unfamiliar with is much more demanding than fixing stuff I got wrong to begin with.
thepasswordis 3y ago
Oh man this is the opposite of my experience! Copilot has replaced almost all of the annoying tedious stuff, especially stuff like writing (simple) SQL queries. “Parse this json and put the fields into the database where they belong” is a fantastic use case for copilot writing SQL. (Yes I’m sure there’s an ORM plugin or some middleware I could write, but in an MVP, or a mock-up, that’s too much pre optimization)
- JBorrow 3y ago
  An ORM is not too much pre optimisation…
  - crabmusket 3y ago
    Yeah MVPs and mockups is exactly what ORMs are for, since they get you the first 80% for very little effort. Maybe it depends on the language, but this is definitely the way Rails and Laravel style framework ORMs are designed.
    - mewpmewp2 3y ago
      To me SQL is better for MVPs. It is transparent and clear, easy to debug and can do easily whatever joins, filtering you want to. ORM is more for future, because it abstracts away the database implementation, so you could in theory change the db or change some behaviour more simply. E.g. if you add something like deleted_at column, ORM can have a single place where to configure system to use it, but if you have SQL queries lying around you may need to find all spots to add that to your where clauses everywhere. But otherwise SQL is easier to work with in my view.
      - crabmusket 3y ago
        What ORMs/languages do you usually work with? Dynamic language ORMs usually let you very easily add filters, work with relationships, etc.
        3y ago
Yodel0914 3y ago
When I've tried codepilot and similar tools, I've found them rather unimpressive. I assumed it was because I hadn't put the time in to learn how to make the best use of it, but maybe it's just that it's not very good. On the other hand, I use ChatGPT (via the API) quite often, and it's very handy. For example, I wrote a SQL update that needed to touch millions of rows. I asked ChatGPT to alter the statement batch the updates, and then asked it to log status updates after each batch. As another example, I was getting a 401 accessing a nuget feed from Azure DevOps - I asked ChatGPT what it could be and it not only told me, but gave me the yaml to fix it. In both cases, this is stuff I could have done myself after a bit of research, but it's really nice to not have to.
WithinReason 3y ago
I'm not using Copilot to write code, I use it for autocomplete. For that it's great.
- valcron1000 3y ago
  100% agree. It usually autocompletes better than the language's LSP since it takes a lot of context into consideration. It's a godsend for repetitive tasks or trivial stuff.
- zarzavat 3y ago
  Copilot probably saved my career. I was getting occasional wrist pain due to keyboard overuse, but the autocomplete is so good that my keyboard use is a tiny fraction of what it was previously. Wrist pain gone. Using Copilot is a skill though, you have to live with it and learn its limits and idiosyncrasies to get the most out of it.
cqqxo4zV46cp 3y ago
It sounds to me like you were getting it to do too much at once.
keeganpoppen 3y ago
it's crazy how amazing it is sometimes for certain things, including comments (and including my somewhat pithy style of comment prose), and how incredible and thorough the wastes of time when it autosuggests some function that is plausible, defined, but yet has some weird semantics such that it subtly ruins my life until i audit every line of code i've written (well, "written", i suppose). i finally disabled it, but doing so _did_ make me kind of sad-- it is nice for the easy things to be made 25x easier, but, for programming, not at the expense of making the hard stuff 5x harder (note i didn't say 10x, nor 100x. 5x). it's not that it's that far away from being truly transformative, it's just that the edge cases are really, really rough because for it to be truly useful you have to trust it pretty much completely & implicitly, and i've just gotten snakebitten in the most devious ways a handful of times. an absolute monster of / victim of the pareto principle, except it makes the "90%" stuff 1.5x easier and the "10%" stuff 5x harder (yes, i know i haven't been using my "k%"s rigorously) (and for those keeping score at home, that adds up to making work 10% harder net, which i'd say is about right in my personal experience). highlights: "the ai" and i collaboratively came up with a new programming language involving defining a new tag type in YAML that lets one copy/paste from other (named) fragments of the same document (as in: `!ref /path/to/thing to copy`) (the turing completeness comes from self-referential / self-semi-overlapping references (e.g. "!ref /name/array[0:10]`) where one of the elements thus referred-to is, itself, a "!ref" to said array). lowlights: as already alluded to, using very plausible, semi-deprecated API functions that either don't do what you think they do, or simply don't work the way one would think they do. this problem is magnified by googling for said API functions only to find cached / old versions of API docs from a century ago that further convince you that things are ok. nowadays, every time i get any google result for a doc page i do a little ritual to ensure they are for the most recent version of the library, because it is absolutely insane how many times i've been bitten by this, and how hard.
ec109685 3y ago
If you used copilot in the beginning (and I think still with some plans), it was only GPT 3.5. Likely you’d get much better results with GPT-4.
- theshrike79 3y ago
  The difference in output code quality in 3.5 vs 4 is staggering - just with using regular old ChatGPT.
  - Auracle 3y ago
    I feel like OpenAI is really shooting themselves in the foot by not having some sort of trial of 4. The difference in quality of everything is huge and most people don’t know it.
dehrmann 3y ago
> It was much less effort to just write everything myself because I actually know what I want to write and fixing my own mistakes was easier than fixing the bot’s. Echoing this, it takes longer to read code than to write it, so generally, if you know what you want to write and it's non-trivial, you'll spend more time groking AI-written code for correctness than writing it from scratch.
throwaway2990 3y ago
You cancelled the tool because you didn’t know how to use it?
chrismorgan 3y ago
> I weep for the juniors that will be absolutely crushed by this garbage. This is the real danger of this sort of thing. When your Copilot or whatever are good enough that they replace what is vastly superior for purely economic reasons. I wrote about this trend applied to the unfortunately inevitable doom of the voice acting industry in favour of text-to-speech models a couple of months ago, using my favourite examples of typesetting, book binding and music engraving: https://news.ycombinator.com/item?id=38491203 https://news.ycombinator.com/item?id=38491203. But when it’s development itself that gets hollowed out like this, I’m not sure what the end state is, because it’s the developers who led past instances of supplanting. Some form of societal decline and fall doesn’t feel implausible. (That sentence really warrants expansion into multiple paragraphs, but I’m not going to. It’s a big topic.)
- m3047 3y ago
  Yes, like desktop publishing compared to traditional printing. Its output is comparatively shitty but average people will settle for it on account of its democratization. My context was feature factory web dev versus low / no code rather than autocomplete versus solo author. By democratization, I mean that it enables one-one where previously there was only one-many: instead of the inversion of experience where the same unique app experience is shared by millions, a technology allows the interface to be tailored for an audience of one or dozens: missing toes and fingers, color blindness, particularly difficult and unique operating conditions, etc. Given those unique constraints the mediocrity provides at least some preferable solution. The downside of this is that it sets the floor a lot lower, and people who would never have even tried or contemplated trying typesetting will dabble with desktop publishing to achieve their ends. Somebody on here gifted me with the word "procrustean" and I've taken it and put it in my Minsky fish tank. There are many reasons to eschew the trusted experts model: somebody who has made heads for pins for twenty years is incontrovertably an expert, but who cares? Our uncanny valley appears to be only a local minimum. [PS, nobody understood my point but I thank them for the honest feedback.]
  - chrismorgan 3y ago
    On your P.S.: I contemplated your point being “I can cope with this stuff and know when to correct it or give up on it, but what chance does a beginner have?” which I think is what you actually meant, but ended up deciding to write what I wrote anyway, because it ends up feeling fairly connected to me, though from a slightly different angle. I’m in India at present, and a number of 18–25 year-olds ask me about learning to code (commonly because their college or University is teaching C/C++ and they have no idea). Somehow, they have more access to a computer than ever before, because most of them carry one on their person all day, and more information about this task than ever before, yet their computer, a phone, has been dumbed-down and locked-up in such a way that they can’t really use it to learn to code, because that’s something you do on Computers, and those are just on the campus or your laptop, Chris, or things like that. Easier for the simple tasks, so that the more complex tasks that enough in a previous generation used to work for just don’t get reached, leaving over time perhaps a chasm, and fewer really skilled people in a society that may depend on them more than previously. The lower floor helps some who wouldn’t have got started before, but discourages others by making things too easy so they never find the challenge they seek.
crabmusket 3y ago
> It was much less effort to just write everything myself because I actually know what I want to write This aligns with my observations. I don't use Copilot etc. but the other devs on my small team do. I've observed that I'm generally a faster and more confident type and coder - not knocking their skills, I'm just more experienced, and also spent my teens reading and writing a lot. I've seen that it helps them in cases where they're less certain what they're doing, but also when they know what they're doing and it's quicker about it.
- mewpmewp2 3y ago
  For me I know what I want to write and it seems that Copilot also knows what I want to write, so as an auto complete it just works out for me. Most of the time code I want to write is in my head, I just need to be able to quickly vomit it out and typing speed is the bottleneck. I am also able to intuitively predict that it is going to vomit out exactly what I want. E.g. I know ahead of time what the 10 lines it will give me are.
balaji1 3y ago
so its good my card that was auto-paying for the Copilot subscription expired
arthur_sav 3y ago
For me it's useful for new languages that I'm not familiar with, saves a lot of googling time and looking up the docs.
atoav 3y ago
I never even started. On a vacation I tried to get a LLM to write me an interpolation function for two hours. I had a test data set and checks laid out. Not a single of the resulting algorithms passed all the checks, most didn't even do what I asked for and a good chunk didn't even run. LLMs give you plausible text. That does not mean it is logically coherent or says what it should.
ringofchaos 3y ago
Depends on how you use it. I use a similar vscode assistant bit only for shorter code. I am able to complete code faster than an instructor on video.
_teyd 3y ago
I think there is a sweet spot where if you're junior on the cusp of intermediate it can help you because you know enough to reject the nonsense, but it can point you in the right direction. Similar to if you need to implement a small feature in a language you don't know, but basically know what needs to get done. I've definitely seen juniors just keep refining the garbage until it manages to pass a build and then try to merge it, though, and using it that way just sort of makes you a worse programmer because you don't learn anything and it just makes you more dependent on the bot. Companies without good code reviews are just going to pile this garbage on top of garbage.
orbisvicis 3y ago
I've been using Bing's GPT-4 to learn Fortran for about a week. Well, I'm mainly reading a book which is project-oriented so I'm often researching topics which aren't covered in enough detail or covered later. I think this mixed-mode approach is great for learning. Since Fortran's documentation is sparse and Google's results are garbage the output of GPT4 helps cut through a lot of the cruft. Half the time it teaches me, the rest I'm teaching it and correcting its mistakes. I certainly won't trust it for anything half complicated but I think it does a good job linking to trustworthy supporting sources, which is how I learned how to cast assumed-rank arrays using c_loc/c_f_pointer and sequence-association using (*) or (1). It's great for learning new concepts and I imagine it would be great for pair-coding in which it suggests oversights to your code. However I can't imagine depending on it to generate anything from scratch. What's surprising is how little help the compiler is - about as bad a resource as SEO junk. I'm used to "-Wall -Werror" from C, but so many gfortran warnings are incorrect.
- pklausler 3y ago
  Please be sure to report poor warnings to the gfortran developers -- it's generally a great compiler, and you can help keep it great. A problem with Fortran compiler error and warning messages is that Fortran is largely a legacy language at this point, and most Fortran code hitting the compilers has already had its errors shaken out. New code, and especially new code from new Fortran users, is somewhat more rare -- so those error and warning checks are a part of the compiler that doesn't get as much exercise as one would like.
  - orbisvicis 3y ago
    There's the famous reallocation bug dating from 2016 that made me think reallocation wasn't supported, but yes, I've already filed my first unique gfortran compiler issue at the gcc bugzilla.
culi 3y ago
I've been using it since before the beta and I still do not understand why people have ever used it for multi-line suggestions. I only ever use it as a one-line autocomplete and it has done wonders for my productivity
Sammi 3y ago
I find copilot to be most useful as an orm. It can vomit out bindings between sql and your code so incredibly fast. It's really bad at doing anything novel and complex, so don't use it for that. But doing trivial stuff with tech you are new to is great. If you're new to sql then it can write you a decent table schema with decent indexes and give you the correct insert/update/select queries. It can even do simple joins. But don't venture into some complex nested stuff. Don't.

oldpersonintx 3y ago

[dead]

teaearlgraycold 3y ago

Really? I've been doing web dev as a hobby for 20 years and professionally for 6 or 7 years. It's super helpful for me giving how much boilerplate there is to write and how much of the training set is for websites. Any time I try to get it to write non-trivial SQL or TypeScript types it fails hard. But it's still a nice time saver for writing tests, request handling, React boilerplate, etc.

louthy 3y ago
This is the problem. As programmers we should be focusing effort on reducing boilerplate, so that it’s never needed again. Instead, we’ve created a boilerplate generator.
- CuriouslyC 3y ago
  The challenge there is that reducing boilerplate comes with opinionated magic, which rubs a lot of people the wrong way. So the creators add configs and code level config, and pretty soon you're back in boilerplate hell.
- teaearlgraycold 3y ago
  Nope. The wrong abstraction is too risky vs. having no abstraction. AI can let us move towards an abstraction-free mode of coding. In the future you’ll tell the AI “add a 60 second cache to every source of X data” and it’ll scour your source code for matching code and you’ll review the diffs. C programmers would review LLVM IR diffs. Web devs will review whatever compilation target they’re using. We use high level languages because they improve reading comprehension and save us time when writing. Having a copilot take on a major role allows us to fundamentally rethink programming.
  - norir 3y ago
    I find it unlikely that we would be asking the ai for a very specific caching design. If the optimists are right, the ai should be able to do its own analysis and automatically design an architecture that is superior to what a human would be able to do. A job spent mostly reviewing ai generated diffs sounds like a level of hell beyond even Dante's imagination.
  - deleted 3y ago
    [deleted]
  - louthy 3y ago
    > We use high level languages because they improve reading comprehension and save us time when writing. Which completely contradicts your earlier point. Why not just get copilot write assembly for you? Or, spit out raw machine code? Oh, that’s right, because you need to check that it hasn’t fucked something up. Which, when there’s a ton of boilerplate, is hard. It’s arguable that programming language evolution stopped around the time Java was released (barring a few sprinkles here and there, like async/await, affine/linear types, etc.) We haven’t had a major leap in language power for decades (not like the leap from assembly to procedural languages) - I believe it’s because Java popularised evolution through libraries - and for a long time that was fine, even if it did lead to language evolution stagnating. But now we’ve hit a complexity threshold that demands an abstraction leap, but instead of looking for that abstraction leap we’re getting a computer to generate boilerplate for us, hoping it will dig us out of the complexity hole. We’re still ways off having a computer maintain a complex code base over many years. So humans still have to do that. It’d be much easier if we remove the incidental complexity. “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger W. Dijkstra
- deleted 3y ago
  [deleted]
- Aeolun 3y ago
  I’m starting to lean in the direction that trading some boiletplate for flexibility is a good deal.

Syntaf 3y ago

As with most things in life, moderation is key. I find co-pilot primarily useful as an auto-complete tool to save keystrokes when writing predictable context driven code. Writing an enum class in one window? Co-pilot can use that context to auto complete usage in other windows. Writing a unit test suite? Co-pilot can scaffold your next test case for you with a simple tab keystroke. Especially in the case of dynamic languages, co-pilot nicely compliments your intellisense

thehours 3y ago
My experience is similar. This week I created a few DBT models and Co-Pilot saved a *ton* of keystrokes on the YML portion. It needed some hand-holding in the early parts, but it was so satisfying to tab autocomplete entire blocks of descriptions and tests once it picked up context with my preferences.
choilive 3y ago
Yep, treat copilot like a really good autocomplete system and its wonderful. Saves a lot of time and typing even in languages that aren't known for having a lot of boilerplate. Treat copilot to solve actual problems (y'know, the kind of stuff you are presumably paid to solve), and it falls completely flat.
- craig 3y ago
  I've found it's autocomplete really bad. It often hallucinates import paths and other illegal operations. It's not clear to me why it can't leverage the LSP better.
- Aeolun 3y ago
  It’s also nice for solving small things like ‘euclidian distance’
- deleted 3y ago
  [deleted]

ben_w 3y ago

> I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And it was basically useless when trying to deal with anything non-trivial or anything to do with SQL (even when I frontloaded it with my entire schema). Good to know, that means I'm still economically useful. I'm using ChatGPT rather than Copilot, and I'm surprised by how much it can do, but even so I wouldn't call it "good code" — I use it for JavaScript, because while I can (mostly) read JS code, I've spent the last 14 years doing iOS professionally and therefore don't know what's considered best practice in browser-land. Nevertheless, even though (usually) I get working code, I can also spot it producing bad choices and (what seems like) oddities. > I weep for the juniors that will be absolutely crushed by this garbage. Indeed. You avoid the two usual mistakes I see with current AI, either thinking it's already game over for us or that it's a nothing-burger. For the latter, I normally have to roll out a quote I can't remember well enough to google, that's something along the lines of "your dog is juggling, filing taxes, and baking a cake, and rather than be impressed it can do any of those things, you're complaining it drops some balls, misses some figures, and the cake recipe leaves a lot to be desired".

varispeed 3y ago
> I'm surprised by how much it can do, but even so I wouldn't call it "good code" You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver.
- ben_w 3y ago
  > You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver. Yes, I've even done that to see how well it works (my experience is that half the time it fixes the code, the other half it claims to but doesn't) — but I can do that because I can recognise that it's bad code. I don't expect a junior to notice bad code, and therefore I don't expect them to ask for a fix. (I've also spent 6 months in a recent job just fixing code written by work experience students).

l5870uoo9y 3y ago

This is the problem with using AI for generating SQL statements; it doesn't know the semantics of the your database schema. If you are still open for a solution, I recently deployed a solution[1] that combines AI, db schema and simple way train AI to know your database schema. Essentially, you "like" correct (or manually corrected) generations and a vectorized version is stored and used in future similar generations. An example could be tell which table or foreign key is preferred for a specific query or that is should wrap columns in quotes. From my preliminary tests it works well. I was able to consistently make it use correct tables, foreign keys and quotes on table/column name for case-sensitivity using only a couple of trainings. Will open a public API for that soon too. [1]: https://www.sqlai.ai/ https://www.sqlai.ai/

tilwidnk 3y ago
It's "a public API", not "an public API", because of the consonant rule. I really worry that there are people out there who will anxiously mangle their company's data thinking what is being called AI, which doesn't exist yet, will save the day.
- l5870uoo9y 3y ago
  Use it as a tool not a replacement. However it does do things well even without much additional information like fixing SQL statement[1]. That being said it is consistently improving, GPT-3.5 to GPT-4 was a major upgrade. [1]: https://www.sqlai.ai/snippets/clroq0qn9001xqzqeidtm4jgx https://www.sqlai.ai/snippets/clroq0qn9001xqzqeidtm4jgx
CrimsonRain 3y ago
Just copy paste the db/table schema in a comment at start of your file. Nothing else needed.

godelski 3y ago

I have similar sentiment and looking at how mixed takes are, I think it depends on what you do. I write a lot of research code so I think it's unsurprising that GPT isn't too good here. But people I see that write code more in line with the "copy and paste from stackoverflow" style, get huge utility out of this. (This isn't a dis on that style, lots of work is repetitive and redundant). So I changed how I use GPT (which I do through API. Much cheaper btw). I use it a lot like how I would use SO in the first place. Get outlines, understand how certain lines might work (noisy process here), generate generic chunks of code especially from modules I'm unfamiliar with. A lot of this can just be seen as cutting down time searching. So, the most useful one: using it as a fuzzy search to figure out how to Google. This one is the most common pattern. Since everything on Google is so SEO optimized and Google clearly doesn't give a shit, I can ask GPT a question, get a noisy response that contains useful vernacular or keywords which I can then use to refine a Google search and actually filter out a decent amount of shit. I think people might read this comment and think that you should just build a LLM into Google, but no, what's going on is more complicated and requires the symbiosis. GPT is dumb, doesn't have context, but is good at being a lossy compression system. The whole reason this works is because I'm intelligent and __context aware__, and importantly, critical of relying on GPT's accuracy[0]. Much of this can't be easily conveyed to GPT and isn't just a matter of token length. So that said, the best way to actually improve this system is actually for Google to just get its shit together or some other search engine to replace them. Google, if you're listening, the best way you can make Google search better with LLMs is to: 1) stop enabling SEO bullshit, 2) throw bard into the side and have the LLM talk to you to help you refine a search. Hell, you can use a RL agent for 1 to just look how many times I back out from the links you send me or look at which links I actually use. Going to page 2 is a strong signal that you served shit. [0] accuracy is going to highly depend on frequency of content. While they dedupe data for training, they don't do great semantic deduping (still an unsolved problem. Even in vision). So accuracy still depends on frequency and you can think of well known high frequency knowledge as having many different versions, or that augmentation is built in. You get lower augmentation rates with specific or niche expert knowledge as there's little baked in augmentation and your "test set" is much further from the distribution of training data.

marcyb5st 3y ago

As other said I use copilot or similar for scaffolding/boilerplate code. But you are right, reading almost correct code that you are unfamiliar with is much more demanding than fixing stuff I got wrong to begin with.

thepasswordis 3y ago

Oh man this is the opposite of my experience! Copilot has replaced almost all of the annoying tedious stuff, especially stuff like writing (simple) SQL queries. “Parse this json and put the fields into the database where they belong” is a fantastic use case for copilot writing SQL. (Yes I’m sure there’s an ORM plugin or some middleware I could write, but in an MVP, or a mock-up, that’s too much pre optimization)

JBorrow 3y ago
An ORM is not too much pre optimisation…
- crabmusket 3y ago
  Yeah MVPs and mockups is exactly what ORMs are for, since they get you the first 80% for very little effort. Maybe it depends on the language, but this is definitely the way Rails and Laravel style framework ORMs are designed.
  - mewpmewp2 3y ago
    To me SQL is better for MVPs. It is transparent and clear, easy to debug and can do easily whatever joins, filtering you want to. ORM is more for future, because it abstracts away the database implementation, so you could in theory change the db or change some behaviour more simply. E.g. if you add something like deleted_at column, ORM can have a single place where to configure system to use it, but if you have SQL queries lying around you may need to find all spots to add that to your where clauses everywhere. But otherwise SQL is easier to work with in my view.
    - crabmusket 3y ago
      What ORMs/languages do you usually work with? Dynamic language ORMs usually let you very easily add filters, work with relationships, etc.
      - 3y ago

Yodel0914 3y ago

When I've tried codepilot and similar tools, I've found them rather unimpressive. I assumed it was because I hadn't put the time in to learn how to make the best use of it, but maybe it's just that it's not very good. On the other hand, I use ChatGPT (via the API) quite often, and it's very handy. For example, I wrote a SQL update that needed to touch millions of rows. I asked ChatGPT to alter the statement batch the updates, and then asked it to log status updates after each batch. As another example, I was getting a 401 accessing a nuget feed from Azure DevOps - I asked ChatGPT what it could be and it not only told me, but gave me the yaml to fix it. In both cases, this is stuff I could have done myself after a bit of research, but it's really nice to not have to.

WithinReason 3y ago

I'm not using Copilot to write code, I use it for autocomplete. For that it's great.

valcron1000 3y ago
100% agree. It usually autocompletes better than the language's LSP since it takes a lot of context into consideration. It's a godsend for repetitive tasks or trivial stuff.
zarzavat 3y ago
Copilot probably saved my career. I was getting occasional wrist pain due to keyboard overuse, but the autocomplete is so good that my keyboard use is a tiny fraction of what it was previously. Wrist pain gone. Using Copilot is a skill though, you have to live with it and learn its limits and idiosyncrasies to get the most out of it.

cqqxo4zV46cp 3y ago

It sounds to me like you were getting it to do too much at once.

keeganpoppen 3y ago

it's crazy how amazing it is sometimes for certain things, including comments (and including my somewhat pithy style of comment prose), and how incredible and thorough the wastes of time when it autosuggests some function that is plausible, defined, but yet has some weird semantics such that it subtly ruins my life until i audit every line of code i've written (well, "written", i suppose). i finally disabled it, but doing so _did_ make me kind of sad-- it is nice for the easy things to be made 25x easier, but, for programming, not at the expense of making the hard stuff 5x harder (note i didn't say 10x, nor 100x. 5x). it's not that it's that far away from being truly transformative, it's just that the edge cases are really, really rough because for it to be truly useful you have to trust it pretty much completely & implicitly, and i've just gotten snakebitten in the most devious ways a handful of times. an absolute monster of / victim of the pareto principle, except it makes the "90%" stuff 1.5x easier and the "10%" stuff 5x harder (yes, i know i haven't been using my "k%"s rigorously) (and for those keeping score at home, that adds up to making work 10% harder net, which i'd say is about right in my personal experience). highlights: "the ai" and i collaboratively came up with a new programming language involving defining a new tag type in YAML that lets one copy/paste from other (named) fragments of the same document (as in: `!ref /path/to/thing to copy`) (the turing completeness comes from self-referential / self-semi-overlapping references (e.g. "!ref /name/array[0:10]`) where one of the elements thus referred-to is, itself, a "!ref" to said array). lowlights: as already alluded to, using very plausible, semi-deprecated API functions that either don't do what you think they do, or simply don't work the way one would think they do. this problem is magnified by googling for said API functions only to find cached / old versions of API docs from a century ago that further convince you that things are ok. nowadays, every time i get any google result for a doc page i do a little ritual to ensure they are for the most recent version of the library, because it is absolutely insane how many times i've been bitten by this, and how hard.

ec109685 3y ago

If you used copilot in the beginning (and I think still with some plans), it was only GPT 3.5. Likely you’d get much better results with GPT-4.

theshrike79 3y ago
The difference in output code quality in 3.5 vs 4 is staggering - just with using regular old ChatGPT.
- Auracle 3y ago
  I feel like OpenAI is really shooting themselves in the foot by not having some sort of trial of 4. The difference in quality of everything is huge and most people don’t know it.

dehrmann 3y ago

> It was much less effort to just write everything myself because I actually know what I want to write and fixing my own mistakes was easier than fixing the bot’s. Echoing this, it takes longer to read code than to write it, so generally, if you know what you want to write and it's non-trivial, you'll spend more time groking AI-written code for correctness than writing it from scratch.

throwaway2990 3y ago

You cancelled the tool because you didn’t know how to use it?

chrismorgan 3y ago

> I weep for the juniors that will be absolutely crushed by this garbage. This is the real danger of this sort of thing. When your Copilot or whatever are good enough that they replace what is vastly superior for purely economic reasons. I wrote about this trend applied to the unfortunately inevitable doom of the voice acting industry in favour of text-to-speech models a couple of months ago, using my favourite examples of typesetting, book binding and music engraving: https://news.ycombinator.com/item?id=38491203 https://news.ycombinator.com/item?id=38491203. But when it’s development itself that gets hollowed out like this, I’m not sure what the end state is, because it’s the developers who led past instances of supplanting. Some form of societal decline and fall doesn’t feel implausible. (That sentence really warrants expansion into multiple paragraphs, but I’m not going to. It’s a big topic.)

m3047 3y ago
Yes, like desktop publishing compared to traditional printing. Its output is comparatively shitty but average people will settle for it on account of its democratization. My context was feature factory web dev versus low / no code rather than autocomplete versus solo author. By democratization, I mean that it enables one-one where previously there was only one-many: instead of the inversion of experience where the same unique app experience is shared by millions, a technology allows the interface to be tailored for an audience of one or dozens: missing toes and fingers, color blindness, particularly difficult and unique operating conditions, etc. Given those unique constraints the mediocrity provides at least some preferable solution. The downside of this is that it sets the floor a lot lower, and people who would never have even tried or contemplated trying typesetting will dabble with desktop publishing to achieve their ends. Somebody on here gifted me with the word "procrustean" and I've taken it and put it in my Minsky fish tank. There are many reasons to eschew the trusted experts model: somebody who has made heads for pins for twenty years is incontrovertably an expert, but who cares? Our uncanny valley appears to be only a local minimum. [PS, nobody understood my point but I thank them for the honest feedback.]
- chrismorgan 3y ago
  On your P.S.: I contemplated your point being “I can cope with this stuff and know when to correct it or give up on it, but what chance does a beginner have?” which I think is what you actually meant, but ended up deciding to write what I wrote anyway, because it ends up feeling fairly connected to me, though from a slightly different angle. I’m in India at present, and a number of 18–25 year-olds ask me about learning to code (commonly because their college or University is teaching C/C++ and they have no idea). Somehow, they have more access to a computer than ever before, because most of them carry one on their person all day, and more information about this task than ever before, yet their computer, a phone, has been dumbed-down and locked-up in such a way that they can’t really use it to learn to code, because that’s something you do on Computers, and those are just on the campus or your laptop, Chris, or things like that. Easier for the simple tasks, so that the more complex tasks that enough in a previous generation used to work for just don’t get reached, leaving over time perhaps a chasm, and fewer really skilled people in a society that may depend on them more than previously. The lower floor helps some who wouldn’t have got started before, but discourages others by making things too easy so they never find the challenge they seek.

crabmusket 3y ago

> It was much less effort to just write everything myself because I actually know what I want to write This aligns with my observations. I don't use Copilot etc. but the other devs on my small team do. I've observed that I'm generally a faster and more confident type and coder - not knocking their skills, I'm just more experienced, and also spent my teens reading and writing a lot. I've seen that it helps them in cases where they're less certain what they're doing, but also when they know what they're doing and it's quicker about it.

mewpmewp2 3y ago
For me I know what I want to write and it seems that Copilot also knows what I want to write, so as an auto complete it just works out for me. Most of the time code I want to write is in my head, I just need to be able to quickly vomit it out and typing speed is the bottleneck. I am also able to intuitively predict that it is going to vomit out exactly what I want. E.g. I know ahead of time what the 10 lines it will give me are.

balaji1 3y ago

so its good my card that was auto-paying for the Copilot subscription expired

arthur_sav 3y ago

For me it's useful for new languages that I'm not familiar with, saves a lot of googling time and looking up the docs.

atoav 3y ago

I never even started. On a vacation I tried to get a LLM to write me an interpolation function for two hours. I had a test data set and checks laid out. Not a single of the resulting algorithms passed all the checks, most didn't even do what I asked for and a good chunk didn't even run. LLMs give you plausible text. That does not mean it is logically coherent or says what it should.

ringofchaos 3y ago

Depends on how you use it. I use a similar vscode assistant bit only for shorter code. I am able to complete code faster than an instructor on video.

_teyd 3y ago

I think there is a sweet spot where if you're junior on the cusp of intermediate it can help you because you know enough to reject the nonsense, but it can point you in the right direction. Similar to if you need to implement a small feature in a language you don't know, but basically know what needs to get done. I've definitely seen juniors just keep refining the garbage until it manages to pass a build and then try to merge it, though, and using it that way just sort of makes you a worse programmer because you don't learn anything and it just makes you more dependent on the bot. Companies without good code reviews are just going to pile this garbage on top of garbage.

orbisvicis 3y ago

I've been using Bing's GPT-4 to learn Fortran for about a week. Well, I'm mainly reading a book which is project-oriented so I'm often researching topics which aren't covered in enough detail or covered later. I think this mixed-mode approach is great for learning. Since Fortran's documentation is sparse and Google's results are garbage the output of GPT4 helps cut through a lot of the cruft. Half the time it teaches me, the rest I'm teaching it and correcting its mistakes. I certainly won't trust it for anything half complicated but I think it does a good job linking to trustworthy supporting sources, which is how I learned how to cast assumed-rank arrays using c_loc/c_f_pointer and sequence-association using (*) or (1). It's great for learning new concepts and I imagine it would be great for pair-coding in which it suggests oversights to your code. However I can't imagine depending on it to generate anything from scratch. What's surprising is how little help the compiler is - about as bad a resource as SEO junk. I'm used to "-Wall -Werror" from C, but so many gfortran warnings are incorrect.

pklausler 3y ago
Please be sure to report poor warnings to the gfortran developers -- it's generally a great compiler, and you can help keep it great. A problem with Fortran compiler error and warning messages is that Fortran is largely a legacy language at this point, and most Fortran code hitting the compilers has already had its errors shaken out. New code, and especially new code from new Fortran users, is somewhat more rare -- so those error and warning checks are a part of the compiler that doesn't get as much exercise as one would like.
- orbisvicis 3y ago
  There's the famous reallocation bug dating from 2016 that made me think reallocation wasn't supported, but yes, I've already filed my first unique gfortran compiler issue at the gcc bugzilla.

culi 3y ago

I've been using it since before the beta and I still do not understand why people have ever used it for multi-line suggestions. I only ever use it as a one-line autocomplete and it has done wonders for my productivity

Sammi 3y ago

I find copilot to be most useful as an orm. It can vomit out bindings between sql and your code so incredibly fast. It's really bad at doing anything novel and complex, so don't use it for that. But doing trivial stuff with tech you are new to is great. If you're new to sql then it can write you a decent table schema with decent indexes and give you the correct insert/update/select queries. It can even do simple joins. But don't venture into some complex nested stuff. Don't.