5 ms·

A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited

• bigstrat2003 2y ago
  I cannot, in fact, attest that they are useful and smart. LLMs remain a fun toy for me, not something that actually produces useful results.
  • pdntspa 2y ago
    I have been deploying useful code from LLMs right and left over the last several months. They are a significant force accelerator for programmers if you know how to prompt them well.
    • fiddlerwoaroof 2y ago
      We’ll see if this is a good idea when we start having millions of lines of LLM-written legacy code. My experience maintaining such code so far has been very bad: accidentally quadratic algorithms; subtly wrong code that looks right; and un-idiomatic use of programming language features.
      • deisteve 2y ago
        ah i see so you're saying that LLM-written code is already showing signs of being a maintenance nightmare, and that's a reason to be skeptical about its adoption. But isn't that just a classic case of 'we've always done it this way' thinking? legacy code is a problem regardless of who wrote it. Humans have been writing suboptimal, hard-to-maintain code for decades. At least with LLMs, we have the opportunity to design and implement better coding standards and review processes from the start. let's be real, most of the code written by humans is not exactly a paragon of elegance and maintainability either. I've seen my fair share of 'accidentally quadratic algorithms' and 'subtly wrong code that looks right' written by humans. At least with LLMs, we can identify and address these issues more systematically. As for 'un-idiomatic use of programming language features', isn't that just a matter of training the LLM on a more diverse set of coding styles and idioms? It's not like humans have a monopoly on good coding practices. So, instead of throwing up our hands, why not try to address these issues head-on and see if we can create a better future for software development?
        • fiddlerwoaroof 2y ago
          Maybe it will work out, but I think we’ll regret this experiment because it’s the wrong sort of “force accelerator”: writing tons of code that should be abstracted rather than just dumped out literally has always caused the worst messes I’ve seen.
          • medvezhenok 2y ago
            Yes, same way that the image model outputs have already permeated the blogosphere and pushed out some artists, the other models will all bury us under a pile of auto-generated code. We will yearn for the pre-GPT years at some point, like we yearn for the internet of the late 90s/early 2000s. Not for a while though. We're going through the early phases of GPT today, so it hasn't been taken over by the traditional power players yet.
        • Eggpants 2y ago
          When the tool is statistical word vomit based, it will never move beyond cool bar trick levels.
        • oblio 2y ago
          LLMs will allow us to write code faster and create applications and systems faster. Which is how we ended up here, which I guess is tolerable, where a webpage with a bit of styling and a table uses up 200MB of RAM.
      • pdntspa 2y ago
        Honestly the code it's been giving me has been fairly cromulent. I don't believe in premature optimization and it is perfect for getting features out quick and then I mold it to what it needs to be.
    • deisteve 2y ago
      same...but have you considered the broader implications of relying on LLMs to generate code? It's not just about being a 'force accelerator' for individual programmers, but also about the potential impact on the industry as a whole. If LLMs can generate high-quality code with minimal human input, what does that mean for the wages and job security of programmers? Will companies start to rely more heavily on AI-generated code, and less on human developers? It's not hard to imagine a future where LLMs are used to drive down programming costs, and human developers are relegated to maintenance and debugging work. I'm not saying that's necessarily a bad thing, but it's definitely something that needs to be considered. As someone who's enthusiastic about the potential of code gen this O1 reasoning capability is going to make big changes. do you think you'll be willing to take a pay cut when your employer realizes they can get similar results from a machine in a few seconds?
      • deleted 2y ago
        [deleted]
      • airstrike 2y ago
        As a society we're not solving for programmer salaries but for general welfare which is basically code for "cheaper goods and services".
      • pdntspa 2y ago
        My boss is holding a figurative gun to my head to use this stuff. His performance targets necessitate the use of it. It is what it is.
        • oblio 2y ago
          Yeah, but this, in itself, is triggered by a hype wave. These come and go. So we can't really judge the long term impact from inside the wave.
      • fragmede 2y ago
        Your job won't be taken by AI, it will be taken by someone wielding AI.
    • attilakun 2y ago
      In a way it's not surprising that people are getting vastly different results out of LLMs. People have different skill levels when it comes to using even Google. An LLM has a vastly bigger input space.
    • criddell 2y ago
      What's a sample prompt that you've used? Every time I've tried to use one for programming, they invent APIs that don't exist (but sound like they might) or fail to produce something that does what it says it does.
      • GaggiX 2y ago
        Have you tried Claude 3.5 Sonnet?
      • disgruntledphd2 2y ago
        Use Python or JS. The models definitely don't seem to perform as well on less hyper prevalent languages.
        • randomdata 2y ago
          Even then it is hit and miss. If you are doing something that is also copy/paste-able out of a StackOverflow comment, you're apt to be fine, but as soon as you are doing anything slightly less common... Good luck.
          • disgruntledphd2 2y ago
            Yeah, fair. It's good for short snippets and ways of approaching the problem but not great at execution. It's like infinitely tailored blog posts, for me at least.
            • randomdata 2y ago
              True. It can be good at giving you pointers towards approaching the problem, even if the result is flawed, for slightly less common problems. But as you slide even father towards esotericism, there is no hope. It won't even get you in the right direction. Unfortunately – as that is where it would be most useful.
      • brianshaler 2y ago
        No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools. I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
    • unshavedyak 2y ago
      I think that's just the same as using an autocomplete efficiently, though. I tend to like them for Search, but not for anything i have to "prompt correctly" because i feel like i can type fast enough that i'm not too worried about auto-completing. With that said i'm not one of those "It's just a parrot!" people. It is, definitely just a parrot atm.. however i'm not convinced we're not parrots as well. Notably i'm not convinced that that complexity won't be sufficient to walk talk and act like intelligence. I'm not convinced that intelligence is different than complexity. I'm not an expert though, so this is just some dudes stupid opinion. I suspect if LLMs can prove to have duck-intelligence (ie duck typing but for intelligence) then it'll only be achieved in volumes much larger than we imagine. We'll continue to refine and reduce how much volume is necessary, but nevertheless i expect complexity to be the real barrier.
  • rfw300 2y ago
    It’s definitely the case that there are some programming workflows where LLMs aren’t useful. But I can say with certainty that there are many where they have become incredibly useful recently. The difference between even GPT-4 last year and C3.5/GPT-4o this year is profound. I recently wrote a complex web frontend for a tool I’ve been building with Cursor/Claude and I wrote maybe 10% of the code; the rest with broad instructions. Had I done it all myself (or even with GitHub Copilot only) it would have taken 5 times longer. You can say this isn’t the most complex task on the planet, but it’s real work, and it matters a lot! So for increasingly many, regardless of your personal experience, these things have gone far beyond “useful toy”.
    • uoaei 2y ago
      The sooner those paths are closed for low-effort high-pay jobs, the better, IMO. All this money for no work is going to our heads. It's time to learn some real math and science, the era of regurgitating UI templates is over.
      • rfw300 2y ago
        I don’t want to be in the business of LLM defender, but it’s just hard to imagine this aging well when you step back and look at the pace of advancement here. In the realm of “real math and science”, O1 has improved from 0% to 50% on AIME today. A year ago, LLMs could only write little functions, not much better than searching StackOverflow. Today, they can write thousands of lines of code that work together with minimal supervision. I’m sure this tech continues to have many limitations, but every piece of trajectory evidence we have points in the same direction. I just think you should be prepared for the ratio of “real” work vs. LLM-capable work to become increasingly small.
        • uoaei 2y ago
          I can probably climb a tree faster than I can build a rocket. But only one will get me all the way to the moon. Don't confuse local optima for global ones.
      • oblio 2y ago
  • deisteve 2y ago
    'Not useful' is a pretty low bar to clear, especially when you consider the state of the art just 5 years ago. LLMs may not be solving world hunger, but they're already being used in production for coding If you're not seeing value in them, maybe it's because you're not looking at the right problems. Or maybe you're just not using them correctly. Either way, dismissing an entire field of research because it doesn't fit your narrow use case is pretty short-sighted. FWIW, I've been using LLMs to generate production code and it's saved me weeks if not months. YMMV, I guess
  • bongodongobob 2y ago
    At this point, you're either saying "I don't understand how to prompt them" or "I'm a Luddite". They are useful, here to stay, and only getting better.
  • baq 2y ago
    Familiarize yourself with a tool which does half the prompting for you, e.g. cursor is pretty good at prompting claude 3.5 and it really does make code edits 10x faster (I'm not even talking about the fancy stuff about generating apps in 5 mins - just plain old edits.)
• awesomeMilou 2y ago
  Is it? They talk about 10k attempts to reach gold medal status in the mathematics olympiad, but zero shot performance doesn't even place it in the upper 50th percentile. Maybe I'm confused but 10k attempts on the same problem set would make anyone an expert in that topic? It's also weird that zero shot performance is so bad, but over a lot of attempts it seems to get correct answers? Or is it learning from previous attempts? No info given.
  • joshribakoff 2y ago
    The correct metaphor is that 10,000 attempts would allow anyone to cherry pick a successful attempt. You’re conflating cherry picking with online learning. This is like if an entire school of students randomized their answers on a multiple choice test, and then you point to someone who scored 100% and claim it is proof of the school’s expertise.
    • awesomeMilou 2y ago
      Yeah but how is it possible that it has such a high margin of error? 10k attempts is insane! Were talking about an error margin of 50%! How can you deliver "expert reasoning" with such an error margin?
  • zone411 2y ago
    That's not what "zero shot" means.
  • rfw300 2y ago
    It’s undeniably less impressive than a human on the same task, but who cares at the end of the day? It can do 10,000 attempts in the time a person can do 1. Obviously improving that ratio will help for any number of reasons, but if you have a computer that can do a task in 5 minutes that will take a human 3 hours, it doesn’t necessarily matter very much how you got there.
    • jsheard 2y ago
      How long does it take the operator to sift through those 10,000 attempts to find the successful one, when it's not a contrived benchmark where the desired answer is already known ahead of time? LLMs generally don't know when they've failed, they just barrel forwards and leave the user to filter out the junk responses.
• apsec112 2y ago
  Even without AI, it's gotten ~10,000 times easier to write software than in the 1950s (eg. imagine trying to write PyTorch code by hand in IBM 650 assembly), but the demand for software engineering has only increased, because demand increases even faster than supply does. Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
  • aantix 2y ago
    The number of tech job postings has tanked - which loosely correlates with the rise of AI. https://x.com/catalinmpit/status/1831768926746734984 https://x.com/catalinmpit/status/1831768926746734984
    • apsec112 2y ago
      GPT-4 came out in March 2023, after most of this drop was already finished.
    • disgruntledphd2 2y ago
      And also with a large increase in interest rates.
    • macinjosh 2y ago
      The tanking is more closely aligned with new tax rules that went to effect that make it much harder to claim dev time as an expense.
    • Meekro 2y ago
      I'm skeptical because "we fired half our programmers and our new AI does their jobs as well as they did" is a story that would tear through the Silicon Valley rumor mill. To my knowledge, this has not happened (yet).
    • guluarte 2y ago
      this drop is more related to the FED increasing the interest rates
    • 2y ago
• afavour 2y ago
  > People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. Can you explain what this statement means? It sounds like you're saying LLMs are now smart enough to be able to jump through arbitrary hoops but are not able to do so when taken outside of that comfort zone. If my reading is correct then it sounds like skepticism is still warranted? I'm not trying to be an asshole here, it's just that my #1 problem with anything AI is being able to separate fact from hype.
  • rfw300 2y ago
    I think what I’m saying is a bit more nuanced than that. LLMs currently struggle with very “wide”, long-run reasoning tasks (e.g., the evolution over time of a million-line codebase). That isn’t because they are secretly stupid and their capabilities are all hype, it’s just that this technology currently has a different balance of strengths and weaknesses than human intelligence, which tends to more smoothly extrapolate to longer-horizon tasks. We are seeing steady improvement on long-run tasks (SWE-Bench being one example) and much more improvement on shorter, more well-defined tasks. The latter capabilities aren’t “hype” or just for show, there really is productive work like that to be done in the world! It’s just not everything, yet.
• crystal_revenge 2y ago
  I have written a ton of evaluations and run countless benchmarks and I'm not even close to convinced that we're at > the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark” so much as we're over-fitting these bench marks (and in many cases fishing for a particular way of measuring the results that looks more impressive). While it's great that the LLM community has so many benchmarks and cares about attempting to measure performance, these benchmarks are becoming an increasingly poor signal. > This is a nerve-wracking time to be a knowledge worker for sure. It might because I'm in this space, but I personally feel like this is the best time to working in tech. LLMs still are awful at things requiring true expertise while increasingly replacing the need for mediocre programmers and dilettantes. I'm increasingly seeing the quality of the technical people I'm working with going up. After years of being stuck in rooms with leetcode grinding TC chasers, it's very refreshing.
• skepticATX 2y ago
  > People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark This seems like a bold statement considering we have so few benchmarks, and so many of them are poorly put together.
• grbsh 2y ago
  I like your phrasing - "any task limited in scope enough to be a 'benchmark'". Exactly! This is the real gap with LLMs, and will continue to be an issue with o1 -- sure, if you can write down all of the relevant context information you need to perform some computation, LLMs should be able to do it. In other words, LLMs are calculators! I'm not especially nerve-wracked about being a knowledge worker, because my day-to-day doesn't consist of being handed a detailed specification of exactly what is required, and then me 'computing' it. Although this does sound a lot like what a product manager does!
• rvz 2y ago
  > And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowledge worker for sure. If you have to keep checking the result of an LLM, you do not trust it enough to give you the correct answer. Thus, having to 'prompt' hundreds of times for the answer you believe is correct over something that claims to be smart - which is why it can confidently convince others that its answer is correct (even when it can be totally erroneous). I bet if Google DeepMind announced the exact same product, you would equally be as skeptical with its cherry-picked results.
• deleted 2y ago
  [deleted]
• deleted 2y ago
  [deleted]
• latexr 2y ago
  > And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! I have spent significant time with GPT-4o, and I disagree. LLMs are as useful as a random forum dweller who recognises your question as something they read somewhere at some point but are too lazy to check so they just say the first thing which comes to mind. Here’s a recent example I shared before: I asked GPT-4o which Monty Python members have been knighted (not a trick question, I wanted to know). It answered Michael Palin and Terry Gilliam, and that they had been knighted for X, Y, and Z (I don’t recall the exact reasons). Then I verified the answer on the BBC, Wikipedia, and a few others, and determined only Michael Palin has been knighted, and those weren’t even the reasons. Just for kicks, I then said I didn’t think Michael Palin had been knighted. It promptly apologised, told me I was right, and that only Terry Gilliam had been knighted. Worse than useless. Coding-wise, it’s been hit or miss with way more misses. It can be half-right if you ask it uninteresting boilerplate crap everyone has done hundreds of times, but for anything even remotely interesting it falls flatter than a pancake under a steam roller.
  • gizmo 2y ago
    I asked GPT-4o and I got the correct answer in one shot: > Only one Monty Python member, Michael Palin, has been knighted. He was honored in 2019 for his contributions to travel, culture, and geography. His extensive work as a travel documentarian, including notable series on the BBC, earned him recognition beyond his comedic career with Monty Python (NERDBOT) (Wikipedia). > Other members, such as John Cleese, declined honors, including a CBE (Commander of the British Empire) in 1996 and a peerage later on (8days). Maybe you just asked the question wrong. My prompt was "which monty python actors have been knighted. look it up and give the reasons why. be brief".
    • deleted 2y ago
      [deleted]
    • latexr 2y ago
      Yes yes, there’s always some “you're holding it wrong” apologist.¹ Look, it’s not a complicated question to ask unambiguously. If you understand even a tiny bit of how these models work, you know you can make the exact same question twice in a row and get wildly different answers. The point is that you never know what you can trust or not. Unless you’re intimately familiar with Monty Python history, you only know you got the correct answer in one shot because I already told you what the right answer is. Oh, and by the way, I just asked GPT-4o the same question, with your phrasing, copied verbatim and it said two Pythons were knighted: Michael Palin (with the correct reasons this time) and John Cleese. ¹ And I’ve had enough discussions on HN where someone insists on the correct way to prompt, then they do it and get wrong answers. Which they don’t realise until they shared it and disproven their own argument.