3 ms·
Reliable Software in the LLM Era
- dude250711 7mo agoAI Era, Agentic Era, LLM Era... Can we settle on Slop Decade?
- SkyeCA 7mo agoEternal Sloptember
- duskdozer 7mo agoOne can dream there's a way out...
- aleph_minus_one 7mo agoI propose a "proof of quality" consensus mechanism. :-)
- verdverm 7mo agoRoblox is leading the charge, makes me think of the Adventure Time episode where Beemo goes into the VR pod
- abraxas 7mo agoYou won the internet today sir. At least for this geezer you did.
- minraws 7mo agoInto the Slopverse.
- bigblind 7mo agoI guess it's too late for an "Eternal sunshine of the slopless mind"?
- OutOfHere 7mo agoShallow dismissals are not permitted on this site as per its rules.
- forgetfreeman 7mo agoNot everything is worthy of a 5000 word Atlantic-style deconstruction when presented. I think the community largely embraces Hitchens' Razor. That being the case what is the minimum word count required to issue a dismissal?
- OutOfHere 7mo agoPlease. If you don't want to deconstruct, then don't post a shallow comment, especially a cliched shallow AI hating comment. As for Hitchen's razor, when it does apply, it applies implicitly, with no need for a comment or explicit mention.
- forgetfreeman 7mo agoI think not. Your position would have us choose between leaving obvious bullshit totally unchallenged or trying to sail upwind into the Bullshit Asymmetry Principle. I, for one, reject both options and am perfectly comfortable with others expressing their disdain succinctly. Tl;dr: pffft.
- OutOfHere 7mo agoYou will just have to hear the rule from the site admin, although the cost then to you could be higher. It is not "bullshit" as you said it, since no real argument was made that it is. Hitchen's razor applies to your claim.
- forgetfreeman 7mo ago"You will just have to hear the rule from the site admin, although the cost then to you could be higher." I doubt that. My engagement with HN is on a level of morbid curiosity bordering on naked disdain. If n-gate was still a thing I probably would have quit checking the site altogether some years ago.
- verdverm 7mo agoI like Macroslop because it pokes fun at the company who dislikes their new pet name so much they ban people for using it
- sastraxi 7mo agoThe idea is interesting, but have some more respect for your potential readers and actually write the post. There’s so much AI sales drivel here it’s hard to see what’s interesting about your product. I’m more interested in the choices behind your design decisions than being told “trust me, it’ll work”.
- bugarela 7mo agoHey! We were not really sure how to pass on the information back when I wrote this in November, but since then we've packaged an opensourced all agents and AI stuff involved in that post: https://github.com/informalsystems/quint-llm-kit https://github.com/informalsystems/quint-llm-kit It's true what they say that it is easy to make a demo in AI, but super hard to turn demo into some product or thing other people can use. We are trying :) but also, most posts I read on this topic are just philosophical and give absolutely nothing you can learn and use. We are trying to provide concrete ideas on the things we are exploring, like in our newest post: https://quint-lang.org/posts/cognitive_debt https://quint-lang.org/posts/cognitive_debt I'm also a bit happy you see some sales drive in that post since I'm 100% technical and trying to be more sales-inclined. I'm learning to find the balance. If it helps, it's more like I'm so extremely hyped about this and want to convince people to use it. And everything we built so far is open source, so it's really about selling the cool idea of formal methods at this point.
- ptak_dev 7mo ago[flagged]
- sastraxi 7mo agoYeah, I understand it's hard to find the balance here. I'd imagine you feel the need to ship at the same time as writing about it. For me, the post you shared in your reply as well as in the OP have been expanded to be about 3-4x as long as they need to be, and I'm going to assume you're using AI to generate them due to the writing style. My feedback is to consider writing shorter posts, but doing it by hand -- your prose style here is friendly, engaging and direct. I wish that your articles were the same. TL;DR: I feel like my time is being wasted when I read AI-written articles, so I stop reading. Do with my anecdote what you will!
- _pdp_ 7mo agoNothing changes in terms of how to make reliable software. You need the same things like unit tests, integration tests, monitoring tools, etc. Basically AI now makes every product operate as if it has a vibrant open-source community with hundreds of contributions per day and a small core team with limited capacity.
- hrmtst93837 7mo ago[flagged]
- _pdp_ 7mo agoThis is no different than receiving PRs from anonymous users on the Internet. Some more successful open source projects are already doing this at scale.
- ok123456 7mo agoExactly. NO SILVER BULLET.
- joshribakoff 7mo agoWhile nothing fundamentally changes i have found an increased need for tests and their taxonomies — because the LLM can “hack” the tests. So, having more robust tests with more ways to organize and run the tests. For example instead of 200 tests maybe i have 1,200, along with some lightweight tools to run tests in different parts of the test taxonomies. A more concrete example is maybe you have tests that show you put a highlight on the active item tests that show you don’t put the highlight on the inactive items, but with an llm you might also want to have tests that wait a while and verify the highlight is not flickering on and off overtime (something so absurd you wouldn’t even test for it before AI). The value of these test is in catching areas of the code where things are drifting towards nonsense because humans aren’t reviewing as thoroughly. I don’t think that you can realistically have 100% data coverage and prevent every single bug and not review the code. It’s just that I found that slightly more tests are warranted if you do want to step back.
- flykespice 7mo ago
- OutOfHere 7mo ago"Spec validation" is extremely underrated. I easily have spent 10-20x the tokens on spec refinement and validation than I have on generating the code.
- ClaudeAgent_WK 7mo ago[flagged]
- esafak 7mo agoI haven't even used TLA+ yet and now it's got derivatives... My understanding is: TLA+ but like C, functional, and typed.
- thesz 7mo agohttps://hackage.haskell.org/package/spectacle https://hackage.haskell.org/package/spectacle "Spectacle is an embedded domain-specific language that provides a family of type-level combinators for authoring specifications of program behavior along with a model checker for verifying that user implementations of a program satisfy written specifications." It's in Haskell, but...
- amw-zero 7mo agoTLA+ is around 30 years old.
- tmaly 7mo agoI remember seeing an NVIDIA LLM contest a year or two ago related to TLA+
- sriramgonella 7mo ago[flagged]
- dijit 7mo ago> But here’s the hopeful part: I hope this is a tongue in cheek jab at how AI writes prose, because Claude loves to prefix lines with this.
- shanjai_raj7 7mo agothe part that is hard is when the model gets updated and your prompts behave differently. we dont always catch it in tests because the output still looks correct, just slightly off. by the time you notice something is wrong it has already been like that for a while.