4 ms·
In my anecdotal experience it's been sort of terrible from the start. In my first interaction with it, it suggested that I use a library that had been deprecate
by devjab 4y ago
In my anecdotal experience it's been sort of terrible from the start. In my first interaction with it, it suggested that I use a library that had been deprecated for a few years, which is when I found out it's data cut-off point was in 2021.
We've been building an API with: Asp.Versioning, Microsoft.AspNetCore.OData.Query,, Microsoft.AspNetCore.OData.Deltas and Microsoft.EntityFrameworkCore, and it's been very bad at it. I think it's sort of understandable, because there isn't a lot of documentation or examples for these libraries, and, some of them have changed a lot since 2021, but it can't even write an ActionResult function correctly without a lot of help. At one point we asked it to do something resulting in some very terrible code. When I pointed out what was wrong, it apologized and then proceded to give me the exact same piece of code.
We use it quite a lot, along with co-pilot to test the waters, and so far it's rather unimpressive. From my completely anecdotal experience, it hasn't gotten worse, but neither are useful for things that haven't been solved a million times before. I think the major advantages we're going to see from it is in terms of automated documentation and possibly having it write tests.
That being said, I don't think it's that much worse than google programming. C# documentation is really hard to find. Some of the Odata documentation is a github repository with very few comments and only in-memory example code, but it was easier to find through the use of chatGPT than it was on Google. I do think it needs to automatically include the source and date for what it bases it answers on to help you navigate the answer. What I mean by this is that IActionResult wasn't replaced yet by ActionResult in 2021, so if it simply told you that it's answer is old, then you'd probably be more inclined to look things up in the official documentation. I know I would.
- jack_pp 4y agoI asked it to create a struct representing a table in Go, without telling it the types and it inferred those just from the names, including having lat/long as floats. Then I asked it to write a fiber app using that struct with a single endpoint for adding new entries. Did so perfectly. Asked it to add single entry and multi entry retrieval and done so from context, no bugs
- bradhe 4y agoI mean we can go back and forth on this all day long. What’s the conclusion you’re driving towards?
- jack_pp 4y agoWhy do I have to drive to a conclusion? I was just sharing my experience
- throwbadubadu 4y agoNot surprising, or? It can do all these things for which plentiful examples exist out there quite perfectly, but everything else that requires true more complex understanding of things, transfer of knowledge, or where only little documentation and less examples exist... not really.
- jack_pp 4y agoI wouldn't have thought it could do even this, I am truly amazed that it was able to infer type from name and then keep in context everything it did. I decided to use postgres instead of mysql, asked it to rewrite it with that in mind and it did so perfectly. To me that is amazing.
- 0x6c6f6c 4y agoIt more likely that most examples of lat/long in structs were floats, so that's just what it used.
- malux85 4y agoYeah I have been trying to use it to design neural nets, but it always gets the Tensor shapes wrong and it uses the incorrect layers and parameters. I tried being brief thinking that it has a short “working memory” but that leads to definition ambiguity. I tried being exhaustive with explanations about things like the data range, input shapes and reasons for parameters, and with enough explanation it will eventually come to the right conclusions - but it’s far more effort to exhaustively explain what I need then to just write the code - and even worse, during the explanation phases it gives me “working” code (I.E. code that executes) but is functionally wrong! So if I wasn’t so experienced I would likely have accepted one of these incorrect implementations and then been not sure why it doesn’t work… When I say “wrong” I don’t really mean things like “this hyperparameter is non-optimal” - I mean things like “This network topology doesn’t make sense, and the whole thing is wired together wrong” I think overall it’s a great tool though - one of my favourite hobbies now is to sit down before bed and spend an hour chatting with it on random subjects, asking it deep Domain questions on ancillary interests of mine, but I am cautious about what it produces knowing it gets code so wrong I look forward to future versions that are improved!
- TowerTall 4y ago> We use it quite a lot, along with co-pilot to test the waters Can you shred some lights on how co-pilot is doing comparable?
- llamaLord 4y agoInterestingly, I've found copilot has been getting seriously dumber I feel. Seemingly simple things that I could Google in 5 mins (which I feel is really really what copilot is good for) it seems to be struggling more and more with. This weekend I asked it to write "an event listener that listens for when a user cmd + clicks and binds the event to the onCmdClick function" in a Vue app. I tried around 3 different variation's of the prompt, it just COULD NOT figure out that it needed to check if the cmd key was down. All it would do is just bind "click" events to the function I mentioned.
- devjab 4y agoI think it's hard to compare the two because of how you approach them. We dismiss the co-pilot suggestions a lot, but it's not like we get the same amount of suggestions from chatGPT because you more actively have to seek it out. If that makes sense. I think co-pilot is eventually going to compete with a lot of the auto-generation tools that exist today. Like, you can auto-generate REST controllers in C# if you're not doing the whole Odata with generic <T> controllers, and Microsoft has made a tool for that, but in the future, it'll likely be co-pilot that handles that sort of thing.