7 ms·
> I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And
by ben_w 3y ago
> I cancelled my subscription after 2 months because I was spending way too much mental effort going over all of the code vomit fixing all of the mistakes. And it was basically useless when trying to deal with anything non-trivial or anything to do with SQL (even when I frontloaded it with my entire schema).
Good to know, that means I'm still economically useful.
I'm using ChatGPT rather than Copilot, and I'm surprised by how much it can do, but even so I wouldn't call it "good code" — I use it for JavaScript, because while I can (mostly) read JS code, I've spent the last 14 years doing iOS professionally and therefore don't know what's considered best practice in browser-land. Nevertheless, even though (usually) I get working code, I can also spot it producing bad choices and (what seems like) oddities.
> I weep for the juniors that will be absolutely crushed by this garbage.
Indeed.
You avoid the two usual mistakes I see with current AI, either thinking it's already game over for us or that it's a nothing-burger.
For the latter, I normally have to roll out a quote I can't remember well enough to google, that's something along the lines of "your dog is juggling, filing taxes, and baking a cake, and rather than be impressed it can do any of those things, you're complaining it drops some balls, misses some figures, and the cake recipe leaves a lot to be desired".
- varispeed 3y ago> I'm surprised by how much it can do, but even so I wouldn't call it "good code" You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver.
- ben_w 3y ago> You can tell it the code is bad and how, a lot of the times it will correct it. For some bs code that you have to write it is a great time saver. Yes, I've even done that to see how well it works (my experience is that half the time it fixes the code, the other half it claims to but doesn't) — but I can do that because I can recognise that it's bad code. I don't expect a junior to notice bad code, and therefore I don't expect them to ask for a fix. (I've also spent 6 months in a recent job just fixing code written by work experience students).
- varispeed 3y ago> I don't expect a junior to notice bad code That is true. I actually seen instances of juniors struggling with code that doesn't work and frankly doesn't make sense, but they claim they wrote it :)
- aleph_minus_one 3y ago> You can tell it the code is bad and how, a lot of the times it will correct it. For the code questions that I ask, it is sometimes quite non-trivial to check whether the code is correct or not. It never happened in my tests that it could correct incorret code that it generated. Typically, the bot then generated code that is wrong for sometimes a different and sometimes even a similar reason. > For some bs code that you have to write it is a great time saver. Again I disagree: the common case where you have to write BS code is when the abstraction is wrong. Implementing a proper abstraction that strongly reduces the BS code to write is the way to go.
- mewpmewp2 3y agoWriting that abstraction can also feel BS.
- web3-is-a-scam 3y agoThe entire industry is BS all the way down
- _teyd 3y agoIf you get to a capacitor, you've gone too far - you may be in a power supply.
- philwelch 3y ago> For the latter, I normally have to roll out a quote I can't remember well enough to google, that's something along the lines of "your dog is juggling, filing taxes, and baking a cake, and rather than be impressed it can do any of those things, you're complaining it drops some balls, misses some figures, and the cake recipe leaves a lot to be desired". Something can be very impressive without actually being useful, but that still doesn’t make it useful. There’s no market for a working dog that does a bad job of baking cakes and filing taxes, while dogs that can retrieve game birds or tackle fleeing suspects are in high demand.
- godelski 3y agoI'm certainly impressed by a man that can dig a tunnel with a spoon. But you're definitely right that it's not that useful. But I disagree that half assed work is not useful. It's just lower usefulness. My laundry app isn't even half assed. The programmers couldn't even sort the room list (literal random order) or cache your most recent room. It's still better than the BS system they had before where I had to load a prepaid card and that machine was locked in a building that isn't open on weekends or after 6pm. I'm still immensely frustrated, but I don't want to go back to the old system. Though I'll mention that my fear is that because so many people see LLMs as having far more utility than they offer, we'll get more shit like the above instead of higher quality stuff. Most issues are solved for me to be comfortable in my life, so I definitely value quality a lot more. Plus, reduces a lot of mental stress as I'm not thinking "how can the person that made this be so dumb? How do you learn how to program and not know what sort is?"
- andy99 3y agoThis is the crux of most of modern AI. Nobody debates it's cool. Since 2014 or so there has been no shortage of amazing demos of computers doing stuff many thought wasn't possible or required human level intelligence. But there's not automatically a bridge from those demos to commercial relevance, no matter how amazing.
- deleted 3y ago[deleted]
- godelski 3y ago> You avoid the two usual mistakes I see with current AI, either thinking it's already game over for us or that it's a nothing-burger. This is always really surprising to me that it appears to be these two camps. Though what frustrates me is that if you suggest something in the middle people usually assume you're in the opposite camp than they are. It reminds me a lot of politics and I'm not sure why we're so resistant to nuance when our whole job is typically composed of nuance. Though I'll point out, I think it is natural to complain about your juggling dog dropping balls or making mistakes on your taxes. That doesn't mean you aren't impressed. I think this response is increasingly common considering these dogs are sold as if they are super-human at these tasks. That's quite disappointing and our satisfaction is generally relative to expectations, not actual utility. If you think something is shit and it turns out to be just okay, you're happy and feel like you got a bargain. If you're expecting something to be great but it turns out to be just okay, you're upset and feel cheated. But these are different than saying the juggling dog is a useless pile of crap and will never be useful. I just want to make that clear, so we avoid my first paragraph.
- hinkley 3y agoHistory says that the things in the middle, usually ones with the clearest computational model beneath them, get a fancy name and we stop calling it AI and we just call it algorithms. I could see someone making a case for this being that 'middle' group but there's a sour note to this process that I don't know from one week to the next whether I find it sneaky or delightful. Someone else can make that argument, because I'm so sick of silver bullets and gold rushes that I just. don't. care. Sturgeon's Law applies (90% of everything is crap) and I'll listen just enough to see if anyone is proposing which bits are the 10% to keep a finger on the scale if I think it'll matter. But let everyone else bleed over this, because in another ten years they'll be laughing about how silly they were to think this was going to solve all of our problems or end our profession. Software is eating most things. If something eats software, your employability will be the least of your existential crises.
- coldtea 3y ago>History says that the things in the middle, usually ones with the clearest computational model beneath them, get a fancy name and we stop calling it AI and we just call it algorithms. I've heard this before, but do we? Which were those this from the past that we "stopped called AI and we just call algorithms"? I see three categories: (1) very complex algorithms that we never did call "AI". (2) stuff we did call AI, and we still do - things like expert systems, or IBM's Watson, or game AI. We knew, and still know, that those weren't AGI. (3) some stuff promoted as AI (like the "AI assistant" Clippy), but which, marketing materials aside, nobody really considered AI or called them that. But I don't remember this demoting/relabelling of stuff from AI to "just algorithms". It might have happened with some stuff, but I doubt it was a "classic" development as its portrayed.
- agumonkey 3y agoThe value I found in my short trial of gpt3 was a bidirectional path finder. Don't have me read 20 pages of docs just to integrate into a browser or a framework.. cutting the legwork essentially so I can keep my motivation and inspiration going.
- blibble 3y ago> and therefore don't know what's considered best practice in browser-land. wanting this is probably the worst possible use case for LLM code vomit
- mvdtnz 3y agoI feel like if I ever try something less trivial than generating a wedding speech in the style of HP Lovecraft some AI evangelist tells me I've chosen the wrong use case for an LLM.
- chipotle_coyote 3y agoThat is because you have already found the ideal use case for one.
- ben_w 3y agoFor a sense of scale, the last time my JS knowledge was close to familiar with the best practices of the day, was 1-1.5 years prior to the release of jQuery. I way trying to say how ChatGPT is good in relative terms, not absolute.
- kcrwfrd_ 3y agoAs a senior frontend/javascript guy, I’m afraid that relying on ChatGPT/copilot for _current best practices_ is probably where it works the worst. Oftentimes it will produce code that’s outdated. Or, it will output code that seems great, unless you have an advanced understanding of the browser APIs and behaviors or you thoroughly test it and realize it doesn’t work as you hoped. But it’s pretty good at getting a jumpstart on things. Refining down to best practices is where the engineer comes in, which is what makes it so dicey in the hands of a jr dev.
- sdm 3y ago> I’m afraid that relying on ChatGPT/copilot for _current best practices_ is probably where it works the worst This matches my experience. When ChatGPT started going viral, I started getting a lot of PRs from juniors who where trying it out. Pretty much every single one was using depreciated API calls or best practices from 5-10 years ago. I'd ask why they chose to use an API that is scheduled to be removed in the next release of whatever library or system we are using. ChatGPT does have it's place. But you need to understand the tools you're using. It can't be great for a first spike or just getting something working. But then you have to go and look at what it's doing and make sure you understand it.
- Ferret7446 3y agoSo it's basically StackOverflow except you get answers instantly and without having to deal with "closed as duplicate" nonsense.
- mwarkentin 3y agoYes. Although it’s also hollowed out the group of people using StackOverflow (and perhaps stack overflow has restricted open access to its data for further scraping), so future iterations of LLMs will have less up to date training data to use.
- ben_w 3y agoStackOverflow's making their own competing LLM for all this stuff. IMO, one of the biggest problems with the way people use LLMs right now, is that they're being treated as a single oracle: to know Java, it must be trained on examples of Java. It would be much better if their natural language comprehension abilities were kept separated from their knowledge (and there are development efforts in this direction), so in this example it would be trained to be able to be able to read a Java tutorial rather than by actually reading a Java tutorial, so when the overall system is asked to write something in Java, the language model within the system decides to do this by opening https://learnxinyminutes.com https://learnxinyminutes.com and combining the user query with the webpage. I think this will help make the models more compact, which is a benefit all by itself, but it would also mean that knowledge can be updated much more easily. Someone would have to actually do this in order to see if those benefits are worth the extra cost of having to load a potentially huge a tutorial into the context window, and likewise the extent to which a more compact training set makes the language comprehension worse.
- firecall 3y agoRight now, for me, it’s fancy Auto Complete at best. I get a Free subscription to it by using my kids EDU email accounts. Which is handy :) But I absolutely would not pay for it. I recall the last time I tried using the chat feature to do something, the code it produced wasn’t very useful and it referenced chapters from a book for further information. It was clearly just regurgitating code from a book on the subject, and that just feels wrong to me. At least give credit to the Authors and reference the book so I can go read the suggested chapters LOL
- hinkley 3y agoIf there are patterns that are good, idiomatic, and mostly repeatable, we should be putting those into the standard library, not an AI tool. What we have right now is a system to collect information about the sorts of problems developers want existing code to solve for them. We should embrace it.
- antod 3y ago>For the latter, I normally have to roll out a quote I can't remember well enough to google, that's something along the lines of "your dog is juggling, filing taxes, and baking a cake, and rather than be impressed it can do any of those things, you're complaining it drops some balls, misses some figures, and the cake recipe leaves a lot to be desired". Not the quote, but there was a Farside cartoon along those lines where the dog was being berated for not doing a very good job mowing the lawn: https://i.pinimg.com/originals/22/22/79/222279ceaa98f293e76e1565459f800a.jpg https://i.pinimg.com/originals/22/22/79/222279ceaa98f293e76e...
- seagullriffic 3y agoOh no, a really sad Far Side cartoon! Which is very closely related to a shaggy dog joke you can spin out for ages, "$5 talking dog for sale", which ends with the setup / punchline, "why so cheap?" / "because he's a goddamn liar!"
- noogle 3y agoThe criticism from the second camp stems from the fact that the WHOLE job is to not drop anything. A fence with a hole is useless even if it's 99% intact. A lot of human jobs, especially white collar, are about providing reassurance about the correctness of the results. A system that cannot provide that may be worse than useless since it creates noise, false sense of security and information load.
- geraldhh 3y agoadding nuance to your fence analogy, most fences are decorative and/or suggestive of a border but can be overcome somewhat easily and are hence not useless because they have a hole somewhere.
- strix_varius 3y agoThe spirit of the analogy holds, as there are plenty of clear alternatives that map verbatim: - drinking glass that is 99% hole-free - car that doesn't explode 99% of the time - bag of candy where 99% of the pieces are not poisonous In all of these cases, it's more optimal to start from scratch and build something that you know is 100% reliable than to start with whatever already exists and try to fix it after-the-fact. Personally, I use AI to assist development, especially in unfamiliar stacks, but in the form of a discussion rather than code-vomit. It's primarily synthesizing documentation into more-specific whole answers and providing options and suggestions.
- ben_w 3y agoI recognise the point, but also > - drinking glass that is 99% hole-free This describes my travel flask when the screw cap is on and the slot in the cap is open; most of my drinking glasses have (and need) a much bigger hole-to-surface ratio to get the fluid in and for me to drink from. More relevantly for the output of an AI: in cases where testing is easy, a system which has a 99% chance of a producing a saleable drinking glass and just discards the other 1% to recycling isn't unreasonable… provided you can be sufficiently confident about the test. For AI, the quality of the automated tests of the output is a very solid "it depends on what you're doing".
- Mawr 3y ago> You avoid the two usual mistakes I see with current AI, either thinking it's already game over for us or that it's a nothing-burger. A lot of the latter is caused by the former. It is a nothing burger compared to the shocking amount of hysteria on HN about AI putting programmers out of a job. You'd expect a programmer to know what his job is, but alas, apparently even programmers think of themselves as glorified typewriters.
- jrochkind1 3y ago> therefore don't know what's considered best practice in browser-land. I myself also don't know what's considered best practice in Javascript generally (browser or server-side), even though I also have to write it occasionally -- but I wouldn't feel safe trusting that ChatGPT suggestions were likely to be model current best practices either. On what are you basing your thinking that ChatGPT is more likely than not to be suggesting best practices? (Real question, I'm curious!)
- ben_w 3y ago> On what are you basing your thinking that ChatGPT is more likely than not to be suggesting best practices? (Real question, I'm curious!) I phrased that part badly, you're not the only one to get that (unintended) meaning from my words; where I wrote: > I've spent the last 14 years doing iOS professionally and therefore don't know what's considered best practice in browser-land That was meant to convey my weakness rather than its strength. I can tell when ChatGPT makes a wild error, but that's all because I don't know JS best practices.