9 ms·
I have a friend that always says "innovation happens at the speed of trust". Ever since GPT3, that quote comes to mind over and over. Verification has a high c
by dirkc 1y ago
I have a friend that always says "innovation happens at the speed of trust". Ever since GPT3, that quote comes to mind over and over.
Verification has a high cost and trust is the main way to lower that cost. I don't see how one can build trust in LLMs. While they are extremely articulate in both code and natural language, they will also happily go down fractal rabbit holes and show behavior I would consider malicious in a person.
- acedTrex 1y agoAuthor here: I quite like that quote. A very succinct way of saying what took me a few paragraphs. This new world of having to verify every single thing at all points is quite exhausting and frankly pretty slow.
- Herring 1y agoSo get another LLM to do it. Judging is considerably easier [For LLMs] than writing something from scratch, so LLM judges will always have that edge in accuracy. Equivalently, I also like getting them to write tons of tests to build trust in correct behavior.
- acedTrex 1y ago> Judging is considerably easier than writing something from scratch I don't agree with this at all. Writing new code is trivially easy, to do a full in depth review takes significantly more brain power. You have to fully ascertain and insert yourself into someone elses thought process. Thats way more work than utilizing your own thought process.
- Herring 1y agoSorry, I should have been more specific. I meant LLMs are more reliable and accurate at judging than at generating from scratch. They basically achieve over 80% agreement with human evaluators [1]. This level of agreement is similar to the consensus rate between two human evaluators, making LLM-as-a-judge a scalable and reliable proxy for human judgment. [1] https://arxiv.org/abs/2306.05685 https://arxiv.org/abs/2306.05685 (2023)
- habinero 1y ago80% is a pretty abysmal success rate and means it's very unreliable. It sounds nice but it means at least 1 in 5 are bad. That's worse odds than rolling 1 on a d6. You'll be tripping over mistakes constantly.
- andrekandre 1y ago> 80% is a pretty abysmal success rate and means it's very unreliable. EXACTLY. imagine if your car would at best start 80% of the time...
- bluedel 1y agoAt least if the car doesn't start, you can immediately tell.
- yencabulator 1y agoImagine Google Maps driving directions took you in the correct direction 80% of the time.
- PeterStuer 1y agoFrom the study: " [80% is] the same level of agreement between humans." So if 80% is abysmal, you are getting the same abysmal level from human code reviewers.
- habinero 1y agoNot the same thing. With human code reviewers, you can talk to them and resolve conflicts. You can even ask them to review code differently in future. The whole point of a tool is to reduce workload. If you have to frequently correct it, then it's not worthwhile to use.
- malfist 1y agoLLMs will not have the context behind the lines of code in the CR. Sure there's no bug with how the logic is defined in the CR or even in the context of the project, ti maybe won't throw an exception. But the LLM won't know that the query is iterating over an unindexed field in the DB with the table in prod having 10s of millions of rows. The LLM won't know that even though the code says the button should be red and the comments say the button should be red, the corporate style guide says red should be a very specific hex code that it isn't.
- inetknght 1y ago> So get another LLM to do it. Oh goodness that's like trusting one kid to tell you whether or not his friend lied. In matters where trust matters, it's a recipe for disaster.
- Herring 1y ago*shrug this kid is growing up fast Give it another year and HN comments will be very different. Writing tests already works now. It's usually easier to read tests than to read convoluted logic.
- catlifeonmars 1y agoIt’s also easy to misread tests FWIW.
- dingnuts 1y agothey've been saying that for three years and the performance improvement has been asymptotic (logarithmic) for a decade, if you've been following the state of the art that long.
- inetknght 1y ago> shrug this kid is growing up fast Mmmhmm. And you think this "growing up" doesn't have biases to lie in circumstances where it matters? Consider politics. Politics matter. It's inconceivable that a magic algorithm would lie to us about various political concerns, right? Right...? A magic algorithm lying to us about anything would be extremely valuable to liars. Do you think it's possible that liars are guiding the direction of these magic algorithms?
- habinero 1y agoSure, and there was a lot of hype about the blockchain a decade ago and how it would take over everything. YC funded a ton of blockchain startups. I notice a distinct lack of blockchain hegemony.
- malfist 1y agoLLMs inspecting LLM code is like the police investigating themselves for wrong doing.
- skim1420 1y ago0.9 * 0.9 == 0.81
- kbelder 1y ago0.1 * 0.1 == 0.01
- tayo42 1y agoWe do this is in professional environments already with documentation for designs upfront and code reviews though
- JackFr 1y agohttps://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref... The classic on the subject.
- EGreg 1y ago“Freedom of speech” in politics
- whiplash451 1y ago> "innovation happens at the speed of trust" You'll have to elaborate on that. How much trust was there in electricity, flight and radioactivity when we discovered them? In science, you build trust as you go.
- agent281 1y agoHave you heard of the War of the Currents? > As the use of AC spread rapidly with other companies deploying their own systems, the Edison Electric Light Company claimed in early 1888 that high voltages used in an alternating current system were hazardous, and that the design was inferior to, and infringed on the patents behind, their direct current system. > In the spring of 1888, a media furor arose over electrical fatalities caused by pole-mounted high-voltage AC lines, attributed to the greed and callousness of the arc lighting companies that operated them. https://en.wikipedia.org/wiki/War_of_the_currents https://en.wikipedia.org/wiki/War_of_the_currents
- bori5 1y agoTesla is barely mentioned in that article which is somewhat surprising
- throw4847285 1y agoNot surprising at all. He was a minor player in the Current Wars compared to his primary benefactor, George Westinghouse. His image was rehabilitated first by Serbian-Americans and then by webcomic artists and Redditors, who turned him into a secular saint. Most of what people think they know about Tesla is not actually true if you examine the historical record. But software engineering as a discipline demands business villains and craftsman heroes, and so Edison and Tesla were warped to fit those roles even though in real life there is only evidence of cordial interactions.
- dirkc 1y agoI use it to mean that the more people trust each other, the quicker things get done. Maybe the statement can be rephrased as "progress happens at the speed of trust" to avoid the specific scientific connotation.
- lubujackson 1y agoWe never can have total trust in LLM output, but we can certainly sanitize it and limit it's destructive range. Just like we sanitize user input and defend with pentests and hide secrets in dot files, we will eventually resolve to "best practices" and some "SOC-AI compliance" standard down the road. It's just too useful to ignore, and trust is always built, brick by brick. Let's not forget humans are far from reliable anyway. Just like with driving cars, I imagine producing less buggy code (along predefined roads) will soon outpace humans. Then it is just blocking and tackling to improve complexity.
- bluefirebrand 1y ago> We never can have total trust in LLM output, but we can certainly sanitize it and limit it's destructive range Can we really do this reliably? LLMs are non-deterministic, right, so how do we validate the output in a deterministic way? We can validate things like shape of data being returned, but how do we validate correctness without an independent human in the loop to verify?
- lovich 1y agoThe same way we did it with humans in the loop? I check AI output for hallucinations and issues as I don’t fully trust it to work, but we also do PRs with humans to have another set of eyes check because humans also make mistakes. For the soft sciences and arts I’m not sure how to validate anything from AI but for software and hard sciences I don’t see why test suites wouldn’t continue serving their same purpose
- aDyslecticCrow 1y agoFamously, "it's easier to write code than to read it". That goes for humans. So why did we automate the easy part and move the effort over to the hard part? If we need a human in the loop to check every row of code for the deep logic errors... then we could just get the human to write it no?
- 1y ago