10 ms·

Using LLMs at Oxide

• deleted 11mo ago
  [deleted]
• bryancoxwell 11mo ago
  Find it interesting that the section about LLM’s tells when using it for writing is absolutely littered with emdashes
  • matt_daemon 11mo ago
    I believe Bryan is a well known em dash addict
    • bryancoxwell 11mo ago
      And I mean no disrespect to him for it, it’s just kind of funny
    • rl3 11mo ago
      >I believe Bryan is a well known em dash addict I was hoping he'd make the leaderboard, but perhaps the addiction took proper hold in more recent years: https://www.gally.net/miscellaneous/hn-em-dash-user-leaderboard-v2.html https://www.gally.net/miscellaneous/hn-em-dash-user-leaderbo... https://news.ycombinator.com/user?id=bcantrill https://news.ycombinator.com/user?id=bcantrill No doubt his em dashes are legit, of course.
  • minimaxir 11mo ago
    You can stop LLMs from using em-dashes by just telling it to "never use em-dashes". This same type of prompt engineering works to mitigate almost every sign of AI-generated writing, which is one reason why AI writing heuristics/detectors can never be fully reliable.
    • dcre 11mo ago
      This does not work on Bryan, however.
    • jgalt212 11mo ago
      I guess, but if even in you set aside any obvious tells, pretty much all expository writing out of an LLM still reads like pablum without any real conviction or tons of hedges against observed opinions. "lack of conviction" would be a useful LLM metric.
      • minimaxir 11mo ago
        I ran a test for a potential blog post where I take every indicator of AI writing and tell the LLM "don't do any of these" and resulted in high school AP English quality writing. Which could be considered a lack of conviction level of writing.
  • bccdee 11mo ago
    To be fair, LLMs usually use em-dashes correctly, whereas I think this document misuses them more often than not. For example: > This can be extraordinarily powerful for summarizing documents — or of answering more specific questions of a large document like a datasheet or specification. That dash shouldn't be there. That's not a parenthetical clause, that's an element in a list separated by "or." You can just remove the dash and the sentence becomes more correct.
    • NobodyNada 11mo ago
      LLMs also generally don't put spaces around em dashes — but a lot of human writers do.
      • kimixa 11mo ago
        I think you're thinking of british-style "en-dashes" – which is often used for something that could have been separated by brackets but do have a space either side – rather than "em" dashes. They can also be used in a similar place as a colon – that is to separate two parts of a single sentence. British users regularly use that sort of construct with "-" hyphens, simply because they're pretty much the same and a whole lot easier to type on a keyboard.
    • the_af 11mo ago
      I don't know whether that use of the em-dash is grammatically correct, but I've seen enough native English writers use it like that. One example is Philip K Dick.
      • bccdee 11mo ago
        Perhaps you have—or perhaps you've seen this construction instead, where (despite also using "or") the phrase on the other side of the dash is properly parenthetical and has its own subject.
  • _zagj 11mo ago
    There was a comment recently by HN's most enthusiastic LLM cheerleader, Simon Willison, that I stopped reading almost immediately (before seeing who posted it), because it exuded the slop stench of an LLM: https://news.ycombinator.com/item?id=46011877 https://news.ycombinator.com/item?id=46011877 However, I was surprised to see that when someone (not me) accused him of using an LLM to write his comment, he flatly denied it: https://news.ycombinator.com/item?id=46011964 https://news.ycombinator.com/item?id=46011964 Which I guess means (assuming he isn't lying) if you spend too much time interacting with LLMs, you eventually resemble one.
    • Jweb_Guru 11mo ago
      > if you spend too much time interacting with LLMs, you eventually resemble one Pretty much. I think people who care about reducing their children's exposure to screen time should probably take care to do the same for themselves wrt LLMs.
    • Philpax 11mo ago
      I don't know what to tell you: that really does not read like it was written by a LLM. You were perhaps set off by the very first sentence, which sounds like it was responding to a prompt?
    • jph00 11mo ago
      It reads exactly like all his writing over many years afaict. Which is to say - it reads well. Just because someone is clear, thoughtful, and thorough, does not make them an AI. AI writing is actually quite different to this.
• thatxliner 11mo ago
  The empathy section is quite interesting
• monkaiju 11mo ago
  Hmmm, I'm a bit confused of their conclusions (encouraging use) given some of the really damning caveats they point out. A tool they themselves determine to need such careful oversight probably just shouldn't be used near prod at all.
  • gghffguhvc 11mo ago
    For the same quality and quantity output, if the cost of using LLMs + the cost of careful oversight is less than the cost of not using LLMs then the rational choice is to use them. Naturally this doesn’t factor in things like human obsolescence, motivation and self-worth.
    • zihotki 11mo ago
      And it doesn't factor seniority/experience. What's good for a senior developer is not necessarily same for a beginner
    • ahepp 11mo ago
      It seems like this would be a really interesting field to research. Does AI assisted coding result in fewer bugs, or more bugs, vs an unassisted human? I've been thinking about this as I do AoC with Copilot enabled. It's been nice for those "hmm how do I do that in $LANGUAGE again?" moments, but it's also wrote some nice looking snippets that don't do quite what I want it to. And many cases of "hmmm... that would work, but it would read the entire file twice for no reason". My guess, however, is that it's a net gain for quality and productivity. Humans make bugs too and there need to be processes in place to discover and remediate those regardless.
      • sunshowers 11mo ago
        I'm not sure about research, but I've used LLMs for a few things here at Oxide with (what I hope is) appropriate judgment. I'm currently trying out using Opus 4.5 to take care of a gnarly code reorganization that would take a human most of a week to do -- I spent a day writing a spec (by hand, with some editing advice from Claude Code), having it reviewed as a document for humans by humans, and feeding it into Opus 4.5 on some test cases. It seems to work well. The spec is, of course, in the form of an RFD, which I hope to make public soon. I like to think of the spec is basically an extremely advanced sed script described in ~1000 English words.
• thundergolfer 11mo ago
  A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line: > it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they should approach LLM use. Bryan has 30+ years of challenging software (and now hardware) engineering experience. He memorably said that he's worked on and completed a "hard program" (an OS), which he defines as a program you doubt you can actually get working. The way Bryan approaches an LLM is super different to how a 2025 junior engineer does so. That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM.
  • pests 11mo ago
    > That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM. Years ago I had to spend many months building nothing but Models (as in MVC) for a huge data import / ingest the company I worked on was rewriting. It was just messy enough that it couldn't be automated. I almost lost my mind from the dull monotony and started even having attendance issues. I know today that could have been done with an LLM in minutes. Almost crazy how much time I put into that project compared to if I did it today.
    • aatd86 11mo ago
      The issue is that it might look good but an LLM often inserts weird mistakes. Or ellipses. Or overindex on the training data. If someone is not careful it is easy to completely wreck the codebase by piling on seemingly innocuous commits. So far I have developed a good sense for when I need to push the llm to avoid sloppy code. It is all in the details. But a junior engineer would never find/anticipate those issues. I am a bit concerned. Because the kind of software I am making, a llm would never prompt on its own. A junior cannot make it, it requires research and programming experience that they do not have. But I know that if I were a junior today, I would probably try to use llms as much as possible and would probably know less programming over time. So it seems to me that we are likely to have worse software over time. Perhaps a boon for senior engineers but how do we train junior devs in that environment? Force them to build slowly, without llms? Is it aligned with business incentives? Do we create APIs expecting the code to be generated by LLMs or written by hand? Because the impact of verbosity is not necessarily the same. LLMs don't get tired as fast as humans.
• deleted 11mo ago
  [deleted]
• an_ko 11mo ago
  I would have expected at least some consideration of public perception, given the extremely negative opinions many people hold about LLMs being trained on stolen data. Whether it's an ethical issue or a brand hazard depends on your opinions about that, but it's definitely at least one of those currently.
  • john01dav 11mo ago
    He speaks of trust and LLMs breaking that trust. Is this not what you mean, but by another name? > First, to those who can recognize an LLM’s reveals (an expanding demographic!), it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. But there are deeper problems: LLM-generated writing undermines the authenticity of not just one’s writing but of the thinking behind it as well. If the prose is automatically generated, might the ideas be too? The reader can’t be sure — and increasingly, the hallmarks of LLM generation cause readers to turn off (or worse). > Specifically, we must be careful to not use LLMs in such a way as to undermine the trust that we have in one another > our writing is an important vessel for building trust — and that trust can be quickly eroded if we are not speaking with our own voice
  • tolerance 11mo ago
    I made the mistake of first reading this as a document intended for all in spite of it being public. This is a technical document that is useful in illustrating how the guy who gave a talk once that I didn’t understand but was captivated by and is well-respected in his field intends to guide his company’s use of the technology so that other companies and individual programmers may learn from it too. I don’t think the objective was to take any outright ethical stance, but to provide guidance about something ostensibly used at an employee’s discretion.
• john01dav 11mo ago
  > it is presumed that of the reader and the writer, it is the writer that has undertaken the greater intellectual exertion. (That is, it is more work to write than to read!) This applies to natural language, but, interestingly, the opposite is true of code (in my experience and that of other people that I've discussed it with).
  • worble 11mo ago
    See: Kernighan's Law > Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it? https://www.laws-of-software.com/laws/kernighan/ https://www.laws-of-software.com/laws/kernighan/
    • DrewADesign 11mo ago
      I think people misunderstand this quote. Cleverness in this context is referring to complexity, and generally stems from falling in love with some complex mechanism you dream up to solve a problem rather than challenging yourself to create something simpler and easier to maintain. Bolting together bits of LLM-created code is is far more likely to be “clever” rather than good.
    • SilverSlash 11mo ago
      What an amazing quote!
  • tikhonj 11mo ago
    That's because embarrassingly bad writing is useless, while embarrassingly bad code can still make the computer do (roughly) the right thing and lets you tick off a Jira ticket. So we end up having way more room for awful code than for awful prose. Reading good code can be a better way to learn about something than reading prose. Writing code like that takes some real skill and insight, just like writing clear explanations.
    • zeroonetwothree 11mo ago
• jhhh 11mo ago
  I've had the same thought about 'written' text with an LLM. If you didn't spend time writing it don't expect me to read it. I'm glad he seems to be taking a hard stance on that saying they won't use LLMs to write non-code artifacts. This principle extends to writing code as well to some degree. You shouldn't expect other people to peer review 'your' code which was simply generated because, again, you spent no time making it. You have to be the first reviewer. Whether these cultural norms are held firmly remains to be seen (I don't work there), but I think they represent thoughtful application of emerging technologies.
• john01dav 11mo ago
  > Wherever LLM-generated code is used, it becomes the responsibility of the engineer. As part of this process of taking responsibility, self-review becomes essential: LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it. Moreover, once in the loop of peer review, generation should more or less be removed: if code review comments are addressed by wholesale re-generation, iterative review becomes impossible. My general procedure for using an LLM to write code, which is in the spirit of what is advocated here, is: 1) First, feed in the existing relevant code into an LLM. This is usually just a few source files in a larger project 2) Describe what I want to do, either giving an architecture or letting the LLM generate one. I tell it to not write code at this point. 3) Let it speak about the plan, and make sure that I like it. I will converse to address any deficiencies that I see, and I almost always do. 4) I then tell it to generate the code 5) I skim & test the code to see if it's generally correct, and have it make corrections as needed 6) Closely read the entire generated artifact at this point, and make manual corrections (occasionally automatic corrections like "replace all C style casts with the appropriate C++ style casts" then a review of the diff) The hardest part for me is #6, where I feel a strong emotional bias towards not doing it, since I am not yet aware of any errors compelling such action. This allows me to operate at a higher level of abstraction (architecture) and remove the drudgery of turning an architectural idea into written, precise, code. But, when doing so, you are abandoning those details to a non-deterministic system. This is different from, for example, using a compiler or higher level VM language. With these other tools, you can understand how they work and rapidly have a good idea of what you're going to get, and you have robust assurances. Understanding LLMs helps, but thus not to the same degree.
  • ryandrake 11mo ago
    I've found that your step 6 takes the vast majority of the time I spend programming with LLMs. Like 10X+ the combined total of time steps 1-5 take. And that's if the code the LLM produced actually works. If it doesn't work (which happens quite often), then even more handholding and corrections are needed. It's really a grind. I'm still not sure whether I am net saving time using these tools. I always wonder about the people who say LLMs save them so much time: Do you just accept the edits they make without reviewing each and every line?
• 000ooo000 11mo ago
  >Wherever LLM-generated code is used, it becomes the responsibility of the engineer. As part of this process of taking responsibility, self-review becomes essential: LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it By this own article's standards, now there are 2 authors who don't understand what they've produced.
  • MobiusHorizons 11mo ago
    This is exactly what the advice is trying to mitigate. At least as I see it, the responsible engineer (meaning author, not some quality of the engineer) needs to understand the intent of the code they will produce. Then if using an llm, they must take full owners of that code by carefully reviewing it or molding it until it reflects their intent. If at the end of this the “responsible” engineer does not understand the code the advice has not been followed.
• rgoulter 11mo ago
  > LLM-generated writing undermines the authenticity of not just one’s writing but of the thinking behind it as well. I think this points out a key point.. but I'm not sure the right way to articulate it. A human-written comment may be worth something, but an LLM-generated is cheap/worthless. The nicest phrase capturing the thought I saw was: "I'd rather read the prompt". It's probably just as good to let an LLM generate it again, as it is to publish something written by an LLM.
  • averynicepen 11mo ago
    I'll give it a shot. Text, images, art, and music are all methods of expressing our internal ideas to other human beings. Our thoughts are the source, and these methods are how they are expressed. Our true goal in any form of communication is to understand the internal ideas of others. An LLM expresses itself in all the same ways, but the source doesn't come from an individual - it comes from a giant dataset. This could be considered an expression of the aggregate thoughts of humanity, which is fine in some contexts (like retrieval of ideas and information highly represented in the data/world), but not when presented in a context of expressing the thoughts of an individual. LLMs express the statistical summation of everyone's thoughts. It presents the mean, when what we're really interested in are the data points a couple standard deviations away from the mean. That's where all the interesting, unique, and thought provoking ideas are. Diversity is a core of the human experience. --- An interesting paradox is the use of LLMs for translation into a non-native language. LLMs are actively being used to better express an individual's ideas using words better than they can with their limited language proficiency, but for those of us on the receiving end, we interpret the expression to mirror the source and have immediate suspicions on the legitimacy of the individual's thoughts. Which is a little unfortunate for those who just want to express themselves better.
  • crabmusket 11mo ago
    I think more people should read Naur's "programming as theory building". A comment is an attempt to more fully document the theory the programmer has. Not all theory can be expressed in code. Both code and comment are lossy artefacts that are "projections" of the theory into text. LLMs currently, I believe, cannot have a theory of the program. But they can definitely perform a useful simulacrum of such. I have not yet seen an LLM generated comment that is truly valuable. Of course, lots of human generated comments are not valuable either. But the ceiling for human comments is much, much higher.
• mcqueenjordan 11mo ago
  As usual with Oxide's RFDs, I found myself vigorously head-nodding while reading. Somewhat rarely, I found a part that I found myself disagreeing with: > Unlike prose, however (which really should be handed in a polished form to an LLM to maximize the LLM’s efficacy), LLMs can be quite effective writing code de novo. Don't the same arguments against using LLMs to write one's prose also apply to code? Was this structure of the code and ideas within the engineers'? Or was it from the LLM? And so on. Before I'm misunderstood as a LLM minimalist, I want to say that I think they're incredibly good at solving for the blank page syndrome -- just getting a starting point on the page is useful. But I think that the code you actually want to ship is so far from what LLMs write, that I think of it more as a crutch for blank page syndrome than "they're good at writing code de novo". I'm open to being wrong and want to hear any discussion on the matter. My worry is that this is another one of the "illusion of progress" traps, similar to the one that currently fools people with the prose side of things.
  • dcre 11mo ago
    In my experience, LLMs have been quite capable of producing code I am satisfied with (though of course it depends on the context — I have much lower standards for one-off tools than long-lived apps). They are able to follow conventions already present in a codebase and produce something passable. Whereas with writing prose, I am almost never happy with the feel of what an LLM produces (worth noting that Sonnet and Opus 4.5’s prose may be moving up from disgusting to tolerable). I think of it as prose being higher-dimensional — for a given goal, often the way to express it in code is pretty obvious, and many developers would do essentially the same thing. Not so for prose.
  • lukasb 11mo ago
    One difference is that clichéd prose is bad and clichéd code is generally good.
    • joshka 11mo ago
      Depends on what your prose is for. If it's for documentation, then prose which matches the expected tone and form of other similar docs would be clichéd in this perspective. I think this is a really good use of LLMs - making docs consistent across a large library / codebase.
• fallat 11mo ago
  The problem with this text is it's a written anecdote. Could all be fake.
• bgwalter 11mo ago
  Cantrill jumps on every bandwagon. When he assisted in cancelling a Node developer (not a native English speaker) over pronouns he was following the Zeitgeist, now "Broadly speaking, LLM use is encouraged at Oxide." He is a long way from Sun.
  • crabmusket 11mo ago
    For those interested, here's a take from Bryan after that incident https://bcantrill.dtrace.org/2013/11/30/the-power-of-a-pronoun/ https://bcantrill.dtrace.org/2013/11/30/the-power-of-a-prono...
    • KronisLV 11mo ago
      The change: https://github.com/joyent/libuv/pull/1015/files https://github.com/joyent/libuv/pull/1015/files > Sorry, not interested in trivial changes like that. - bnoordhuis As a not native English speaker, I think the change itself is okay (women will also occasionally use computers), but saying you're not interested in merging it is kinda cringe, for a lack of a better term - do you not realize that people will take issue with this and you're turning a trivial change into a messy discussion? Stop being a nerd and merge the damn changeset, it won't break anything either, read the room. Admittedly, I also view the people arguing in the thread to be similarly cringe, purely on the basis that if someone is uninterested/opposed to stuff like this, you are exceedingly unlikely to be able to make them care. Feels the same as how allowlist/denylist reads more cleanly, as well as main for a branch name uses a very common word as well - as long as updating your CI config isn't too much work. To show a bit of empathy the other way as well, maybe people get tired of too many changes like that (e.g. if most of the stuff you review is just people poking the docs by rewording stuff to be able to say that they contributed to project X). Or maybe people love to take principled stances and to argue idk > ...it’s not the use of the gendered pronoun that’s at issue (that’s just sloppy), but rather the insistence that pronouns should in fact be gendered. Yeah, odd thing to get so fixated on when the they/them version is more accurate in this circumstance. While I don't cause drama when I see gendered ones (again, most people here have English as a second language), I wouldn't argue with someone a bunch if they wanted to correct the docs or whatever.
• torben-friis 11mo ago
  The guide is generally very well thought, but I see an issue in this part: It sets the rule that things must be actually read when there’s a social expectation (code interviews for example) but otherwise… remarks that use of LLMs to assist comprehension has little downside. I find two problems with this: - there is incoherence there. If LLMs are flawless in reading and summarization, there is no difference with reading the original. And if they aren’t flawless, then that flaw also extends to non social stuff. - in practice, I haven’t found LLMs so good as reading assistants. I’ve send them to check a linked doc and they’ve just read the index and inferred the context, for example. Just yesterday I asked for a comparison of three technical books on a similar topic, and it wrongly guessed the third one rather than follow the three links. There is a significant risk in placing a translation layer between content and reader.
  • gpm 11mo ago
    > Just yesterday I asked for a comparison of three technical books on a similar topic, and it wrongly guessed the third one rather than follow the three links. I would consider this a failure in their tool use capabilities, not their reading ones. To use them to read things (without relying on their much less reliable tool use) take the thing and put it in the context window yourself. They still aren't perfect of course, but they are reasonably good. Three whole books likely exceeds their context window size of course, I'd take this as a sign that they aren't up to a task of that magnitude yet.
    • torben-friis 11mo ago
      >Three whole books likely exceeds their context window size of course This was not “read all three books”, this was “check these three links with the (known) book synopsis/reviews there” and it made up the third one. >I would consider this a failure in their tool use capabilities, not their reading ones. Id give it to you if I got an error message, but the text being enhanced with wrong-but-plausible data is clearly a failure of reliability.
  • fastball 11mo ago
    > It sets the rule that things must be actually read when there’s a social expectation (code interviews for example) but otherwise… remarks that use of LLMs to assist comprehension has little downside. I think you got this backwards, because I don't think the RFD said that at all. The point was about a social expectation for writing, not for reading.
• tonkinai 11mo ago
  Based on paragraph length, I would assume that "LLMs as writers" is the most extensive use case.
• forrestthewoods 11mo ago
  > When debugging a vexing problem one has little to lose by using an LLM — but perhaps also little to gain. This probably doesn't give them enough credit. If you can feed an LLM a list of crash dumps it can do a remarkable job producing both analyses and fixes. And I don't mean just for super obvious crashes. I was most impressed with a deadlock where numerous engineers and tried and failed to understand exactly how to fix it.
  • nrhrjrjrjtntbt 11mo ago
    LLMs are good where there is a lot of detail but the answer to be found is simple. This is sort of the opposite of vibe coding, but LLMs are OK at that too.
    • forrestthewoods 11mo ago
      > LLMs are good where there is a lot of detail but the answer to be found is simple. Oooo I like that. Will try and remember that one. Amusingly, my experience is that the longer an issue takes me to debug the simpler and dumber the fix is. It's tragic really.
  • throwdbaaway 11mo ago
    After the latest production issue, I have a feeling that opus-4.5 and gpt-5.1-codex-max are perhaps better than me at debugging. Indeed my role was relegated to combing through the logs, finding the abnormal / suspicious ones, and feeding those to the models.
• bdangubic 11mo ago
  > assurance that the model will not use the document to train future iterations of itself. believing this in 2025 is really fascinating. this is like believing Meta won’t use info they (i)legally collected about you to serve you ads
• AlexCoventry 11mo ago
  I wonder if they would be willing to publish the "LLMs at Oxide" advice, linked in the OP [1], but currently publicly inaccessible. [1] https://github.com/oxidecomputer/meta/tree/master/engineering/llms https://github.com/oxidecomputer/meta/tree/master/engineerin...
  • sudomateo 11mo ago
    Disclaimer: Oxide employee here. To be honest there's really no secret sauce in there. It's primarily how to get started with agents, when to abandon your context and start anew, and advice on models, monitoring cost, and prompting. This is not to diminish the value of the information as it's good information written by great colleagues. I just wanted to note that most of the information can be obtained from the official AI provider documentation and blog posts from AI boosters like Thorsten Ball.
    • AlexCoventry 11mo ago
      Thanks.
      • sudomateo 11mo ago
        You're welcome. My colleague published the text for it: https://gist.github.com/david-crespo/5c5eaf36a2d20be8a3013ba3c7c265d9 https://gist.github.com/david-crespo/5c5eaf36a2d20be8a3013ba...
        • AlexCoventry 11mo ago
          Cool, thanks again to both of you. :-)
• StarterPro 11mo ago
  Nobody has yet to explain how an LLM can be better than a well paid human expert.
  • nrhrjrjrjtntbt 11mo ago
    The not needing to pay it well.
  • WhyOhWhyQ 11mo ago
    A well paid human expert can find lots of uses of LLMs. I'm still not convinced that humans will ever be totally replaced, and what work will look like is human experts using LLMs as another tool in the toolbox, just like how an engineer would have used a slide rule or mechanical calculator back in the day. The kind of work they're good at doesn't cover the full range of necessary engineering tasks, but they do open up new avenues. For instance, yesterday I was able to get the basic gist of three solutions for a pretty complex task in about an hour. The result of that was me seeing that two of them were unlikely to work for what I'm doing, so that now I can invest actual effort in the third solution.
  • felipeerias 11mo ago
    Tools can make individuals and teams more effective. This is just as true for LLM-based tools as it was for traditional ones. The question is not whether one (1) LLM can replace one (1) expert. Rather, it is how much farther an expert can get through better tooling. In my experience, it can be pretty far indeed.
  • deleted 11mo ago
    [deleted]
• 0x0000000 11mo ago
  > Ironically, LLMs are especially good at evaluating documents to assess the degree that an LLM assisted their creation Is there any evidence for this?
  • fearnot 11mo ago
    no
  • sethops1 11mo ago
    If anything my experience has been the opposite of this. LLM detection is guesswork for an LLM.
• koolala 11mo ago
  I disagree with LLM's as Editors. The about of — in the post is crazy.
• keeda 11mo ago
  Here's the only simple, universal law that should apply: THOU SHALT OWN THE CODE THAT THOU DOST RENDER. All other values should flow from that, regardless of whether the code itself is written by you or AI or by your dog. If you look at the values in the article, they make sense even without LLMs in the picture. The source of workslop is not AI, it's a lack of ownership. This is especially true for Open Source projects, which are seeing a wave of AI slop PR's precisely because the onus of ownership is largely on the maintainers and not the upstart "contributors." Note also that this does not imply a universal set of values. Different organizations may well have different values for what ownership of code means -- E.g. in the "move fast, break things" era of FaceBook, workslop may have been perfectly fine for Zuck! (I'd bet it may even have hastened the era of "Move fast with stable infrastructure.") But those values must be consistently applied regardless of how the code came to be.
• cobertos 11mo ago
  > LLMs are especially good at evaluating documents to assess the degree that an LLM assisted their creation!) That's a bold claim. Do they have data to back this up? I'd only have confidence to say this after testing this against multiple LLM outputs, but does this really work for, e.g. the em dash leaderboard of HN or people who tell an LLM to not do these 10 LLM-y writing cliches? I would need to see their reasoning on why they think this to believe.
  • elAhmo 11mo ago
    I would be surprised they have any data about this. There are so many ways LLMs can be involved, from writing everything, to making text more concise or just "simple proofreading". Detecting all this with certainty is not trivial and probably not possible with the current tools we have.
  • yard2010 11mo ago
    I thought about it - a quick way to verify whether something was created with LLM is to feed an LLM half of the text and then let it complete token by token. Every completion, check not just for the next token but the next n-probable tokens. If one of them is the one you have in the text, pick it and continue. This way, I think, you can identify how much the model is "correct" by predicting the text it hasn't yet seen. I didn't test it and I'm far from an expert, maybe someone can challenge it?
    • jampekka 11mo ago
      That seems somewhat similar to perplexity based detection, although you can just get the probabilities of each token instead of picking n-best, and you don't have to generate. It kinda works, but is not very reliable and is quite sensitive to which model the text was generated with. This page has nice explanations: https://www.pangram.com/blog/why-perplexity-and-burstiness-fail-to-detect-ai https://www.pangram.com/blog/why-perplexity-and-burstiness-f...
    • akoboldfrying 11mo ago
      I expect that, for values of n for which this test consistently reports "LLM-generated" on LLM-generated inputs, it will also consistently report "LLM-generated" on human-generated inputs. But I haven't done the test either so I could be wrong.
• navbvvdasvv 11mo ago
  [dead]
• csb6 11mo ago
  Strange to see no mention of potential copyright violations found in LLM-generated code (e.g. LLMs reproducing code from Github verbatim without respecting the license). I would think that would be a pretty important consideration for any software development company, especially one that produces so much free software.
  • dboreham 11mo ago
    Is there current generation LLMs do this? I suppose I mean "do this any more than human developers do".
    • theresistor 11mo ago
      A very recent example: https://github.com/ocaml/ocaml/pull/14369 https://github.com/ocaml/ocaml/pull/14369
      • phyzome 11mo ago
        ...what a remarkable thread.
        • menaerus 11mo ago
          Right? If this is really true, that some random folk without compiler engineering experience, implemented a completely new feature in ocaml compiler by prompting the LLM to produce the code for him, then I think it really is remarkable.
          • ccortes 11mo ago
            Oh wow, is that what you got from this? It seems more like a non experienced guy asked the LLM to implement something and the LLM just output what and experienced guy did before, and it even gave him the credit
            • rcxdude 11mo ago
• fearnot 11mo ago
  I fully disagree with 1) the stance, 2) the conclusions.
• Simplita 11mo ago
  Oxide’s approach is interesting because it treats LLMs as a tool inside a much stricter engineering boundary. Makes me wonder how many teams would avoid chaos if they adopted the same discipline.
• hexo 11mo ago
  "LLMs are amazingly good at writing code" that one was good. I cant stop laughing.
  • Madmallard 11mo ago
    I wrote an entire multiplayer game in XNA that I've tried repeatedly to get LLMs to translate to javascript it's just utterly hopeless how bad they are at doing it even if I break it down into parts once you get into the stuff that actually matters i.e. the physics, event handling, and game logic induced by events, it just completely falls apart 100% of the time
    • azemetre 11mo ago
      I felt this the other day. I wouldn't even consider my example exotic, p2p systems using electron? It just couldn't figure out how to work with YJS correctly. These things aren't hard if you're familiar with the documentation and have made them before, but what there is is an extreme dearth of information about it compared to web dev tutorials.
  • leecommamichael 11mo ago
    I agree with your sentiment, but I do find it amazing that the underlying techniques of inference can emit code that is as apparently coherent as it is. (This does not imply actual coherence.)
• philippta 11mo ago
  > LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it. To extend that: If the LLM is the author and the responsible engineer is the genuine first reviewer, do you need a second engineer at all? Typically in my experience one review is enough.
  • bananapub 11mo ago
    yes, obviously? anyone who is doing serious enough engineering that they have the rule of "one human writes, one human reviews" wants two humans to actually put careful thought in to a thing, and only one of them is deeply incentivised to just commit the code. your suggestion means less review and worse incentives.
    • Yeask 11mo ago
      anyone who is doing serious enough engineering is not using LLMS.
  • ares623 11mo ago
    Yeesss this is what I’ve been (semi-sarcastically) thinking about. Historically it’s one author and one reviewer before code gets shipped. Why introduce a second reviewer and reduce the rumoured velocity gained by LLMs? After all, “it doesn’t matter what wrote the code” right. I say let her rip. Or as the kids say, code goes brrr.
    • sevensor 11mo ago
      I disagree. Code review has a social purpose as well as a technical one. It reinforces a shared understanding of the code and requires one person to assure another that the code is ready for review. It develops consensus about design decisions and agreement about what the code is for. With only one person, this is impossible. “Code goes brrr” is a neutral property. It can just as easily take you to the wrong destination as the right one.
  • K0nserv 11mo ago
    More eyes are better, but more importantly code review is also about knowledge dissemination. If only the original author and the LLM saw the code you have a bus factor of 1. If another person reviews the bus factor is closer to 2.
• Madmallard 11mo ago
  "LLMs can be quite effective writing code de novo." Maybe for simple braindead tasks you can do yourself anyway. Try doing it on something actually hard or complex and they get it wrong 100/100 if they don't have adequate training data, and 90/100 if they do.
• Iridescent_ 11mo ago
  > Oxide employees bear responsibility for the artifacts we create, whatever automation we might employ to create them. Yes, allow the use of LLMs, encourage your employees to use them to move faster by rewarding "performance" regardless of risks, but make sure to place responsibility of failure upon them so that when it happens, the company culture should not be blamed.
• tizzy 11mo ago
  The idea that LLMs are amazing at comprehension but we are expected to read original documents seems contradictory to me? I’m also wary of using them as editors and losing the writers voice as that feels heavily prompt dependent and whether or not the writer does a final pass without any LLM. Asking someone else to re-write is losing your voice if you don’t have an opinion on how the re-write turns out
• atmosx 11mo ago
  Nothing new here. Antirez for once has taken a similar stance on his YouTube video channel which has material on the topic. But it's worthwhile having a document like this publicly available by a company that the tech crowd seems to respect. <offtopic> The "RFD" here stands for "Reason/Request for Decision" or something else? (Request for Decision doesn't have a nice _ring_ on it tbh). I'm aware of RFCs ofc and the respective status changes (draft, review, accepted, rejected) or ADR (Architectural Decision Record) but have not come across the RFD acronym. Google gave several different answers. </offtopic> </offtopic>
  • __jonas 11mo ago
    It stands for ‘Request for Discussion’: https://rfd.shared.oxide.computer/rfd/0001 https://rfd.shared.oxide.computer/rfd/0001
    • atmosx 11mo ago
      Thanks.
• oldpersonintx2 11mo ago
  [dead]
• peheje 11mo ago
  I know I'm walking into a den of wolves here and will probably get buried in downvotes, but I have to disagree with the idea that using LLMs for writing breaks some social contract. If you hand me a financial report, I expect you used Excel or a calculator. I don't feel cheated that you didn't do long division by hand to prove your understanding. Writing is no different. The value isn't in how much you sweated while producing it. The value is in how clear the final output is. Human communication is lossy. I think X, I write X' (because I'm imperfect), you understand Y. This is where so many misunderstandings and workplace conflicts come from. People overestimate how clear they are. LLMs help reduce that gap. They remove ambiguity, clean up grammar, and strip away the accidental noise that gets in the way of the actual point. Ultimately, outside of fiction and poetry, writing is data transmission. I don't need to know that the writer struggled with the text. I need to understand the point clearly, quickly, and without friction. Using a tool that delivers that is the highest form of respect for the reader.
  • growse 11mo ago
    > The value is in how clear the final output is. Clarity is useless if it's inaccurate. Excel is deterministic. ChatGPT isn't.
    • kstrauser 11mo ago
      While I understand the point you’re making, the idea that Excel is deterministic is not commonly shared among Excel experts. It’s all fun and games until it guesses that your 10th separator value, “SEP-10”, is a date.
  • grufkork 11mo ago
    I think the main problem is people using the tool badly and not producing concise material. If what they produced was really lean and correct it'd be great, but you grow a bit tired when you have to expend time reviewing and parsing long, winding and straight wrong PRs and messages from _people_ who have not put in the time.
  • mft_ 11mo ago
    I’m with you, and further, I’d apply this (with some caveats) to images created by generative AI too. I’ve come across a lot of people recently online expressing anger and revulsion at any images or artwork that have been created by genAI. For relatively mundane purposes, like marketing materials, or diagrams, or the sort of images that would anyway be sourced from a low-cost image library, I don’t think there’s an inherent value to the “art”, and don’t see any problem with such things being created via genAI. Possible consequences: 1) Yes, this will likely lead to loss/shifts in employment, but wasn’t progress ever like this? People have historically reacted strongly against many such shifts when advancing technology threatens some sector, but somehow we always figure it out and move on. 2) For genuine art, I suspect this will in time lead to a greater value being placed in demonstrably human-created originals. Related, there’s probably of money to be made by whoever can create a trusted system somehow capturing proof of human work, in a way that can’t be cheated or faked.
• cvcderringer 11mo ago
  I had trouble getting past the Early Modern English tinge of the language used in this. It’s fun, but it distracts from the comprehension in attempt to just sound epic. It’s fine if you’re writing literature, but it comes off sounding uppity in a practical doc for devs. Writing is not just about conveying something in a mood you wish to set. Study how Richard Feynman and Warren Buffett communicated to their audiences; part of their success is that they speak to their people in the language all can easily understand.
  • batney 11mo ago
    Here it is, rewritten in accessible English: Using Large Language Models (LLMs) at Oxide This document explains how we should think about using LLMs (like ChatGPT or similar tools) at Oxide. What are LLMs? LLMs are very advanced computer programs that can understand and generate text. They've become a big deal in the last five years and can change how we work. But, like any powerful tool, they have good and bad sides. They are very flexible, so it’s hard to give strict rules about how to use them. Still, because they are changing so fast, we need to think carefully about when and how we use them at Oxide. What is Important When Using LLMs We believe using LLMs should follow our core values: Responsibility: We are responsible for the work we produce. Even if we use an LLM to help, a human must make the final decisions. The person using the LLM is responsible for what comes out. Rigor (Care and Precision): LLMs can help us think better or find mistakes, but if we use them carelessly, they can cause confusion. We should use them to improve our work, not to cut corners. Empathy: Remember, real people read and write what we produce. We should be kind and respectful in our language, whether we are writing ourselves or letting an LLM help. Teamwork: We work as a team. Using LLMs should not break trust among team members. If we tell others we used an LLM, it might seem like we’re avoiding responsibility, which can hurt trust. Urgency (Doing Things Quickly): LLMs can help us work faster, but we shouldn’t rush so much that we forget responsibility, care, and teamwork. Speed is good, but not at the cost of quality and trust. How We Use LLMs LLMs can be used in many ways. Here are some common uses: 1. As Readers LLMs are great at quickly understanding documents, summaries, or answering questions about texts. Important: When sharing documents with an LLM, make sure your data is private. Also, remember that uploading files might allow the LLM to learn from your data unless you turn that off. Note: Use LLMs to help understand documents, but don’t skip reading them yourself. LLMs are tools, not replacements for reading carefully. 2. As Editors LLMs can give helpful feedback on writing, especially after you’ve written a draft. They can suggest improvements in structure and wording. Caution: Sometimes, LLMs may flatter your work too much or change your style if used too early. Use them after you’ve done some work yourself. 3. As Writers LLMs can write text, but their writing can be basic or obvious. Sometimes, they produce text that shows it was made by a machine. Why be careful? If readers see that the writing is from an LLM, they might think the author didn’t put in enough effort or don’t truly understand the ideas. Our rule: Usually don’t let LLMs write your final drafts. Use them to help, but own your words and ideas. 4. As Code Reviewers LLMs can review code and find problems, but they can also miss issues or give bad advice. Use them as a helper, not a replacement for human review. 5. As Debuggers LLMs can sometimes help find solutions to tricky problems. They might give helpful hints. But don’t rely on them too much—use them as a second opinion. 6. As Programmers LLMs are very good at writing code, especially simple or experimental code. They can be useful for quick tasks like writing tests or prototypes. Important: When an LLM writes code, the person responsible must review it carefully. Responsibility for the code stays with the human. Teamwork: If you use an LLM to generate code, make sure you understand and review it yourself first. How to Use LLMs Properly There are detailed guidelines and tips in the internal document called "LLMs at Oxide." In general: Using LLMs is encouraged, but always remember your responsibilities—to your product, your customers, and your team.
• deleted 11mo ago
  [deleted]
• petetnt 11mo ago
  Funny how the article states that "LLMs can be excellent editors" and then the post repeats all the mistakes that no editor would make: 1. Because reading posts like this 2. Is actually frustrating as hell 3. When everything gets dragged around and filled with useless anecdotes and 3 adjective mumbojumbos and endless emdashes — because somehow it's better than actually just writing something up. Which just means that people in tech or in general have no understanding what an editor does.
  • xondono 11mo ago
    Funnily enough, the text is so distinctively Cantrillian that I have no doubts this is 100% an “organic intelligence” product.
• CerryuDu 11mo ago
  > LLMs are superlative at reading comprehension, able to process and meaningfully comprehend documents effectively instantly. I couldn't disagree more. (In fact I'm shocked that Bryan Cantrill uses words like "comprehension" and "meaningfully" in relation to LLMs.) Summaries provided by ChatGPT, conclusions drawn by it, contain exaggerations and half-truths that are NOT there in the actual original sources, if you bother enough to ask ChatGPT for those, and to read them yourself. If your question is only slightly suggestive, ChatGPT's tuning is all too happy to tilt the summary in your favor; it tells you what you seem to want to hear, based on the phrasing of your prompt. ChatGPT presents, using confident and authoritative language, total falsehoods and deceptive half-truths, after parsing human-written originals, be the latter natural language text, or source code. I now only trust ChatGPT to recommend sources to me, and I read those -- especially the relevant-looking parts -- myself. ChatGPT has been tuned by its masters to be a lying sack of shit. I've recently asked ChatGPT a factual question: I asked it about the identity of a public figure (an artist) whom I had seen in a video on youtube. ChatGPT answered with "Person X", and even explained why Person X's contribution was so great to the piece of art in question. I knew the answer was wrong, so I retorted only with: "Source?". Then ChatGPT apologized, and did the exact same thing, just with "Person Y"; again explaining why Person Y was so influental in making that piece of art so great. I knew the answer was wrong still, so I again said: "Source?". And at third attempt, ChatGPT finally said "Person Z", with a verifiable reference to a human-written document that identified the artist. FUCK ChatGPT.
• OptionOfT 11mo ago
  > Wherever LLM-generated code is used, it becomes the responsibility of the engineer. As part of this process of taking responsibility, self-review becomes essential: LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it. I think the review by the prompt writer should be at a higher level than another person who reviews the code. If I know how to do something, it is easier for me to avoid mistakes while doing it. When I'm reviewing it it requires different pathways in my brain. Since there is code out there I'm drawn to that path, and I might not not always spot the problem points. Or code might be written in a way that I don't recognize, but still exhibits the same mistake. In the past, as a reviewer I used to be able to count on my colleagues' professionalism to be a moat. The size of the moat is inverse to the amount of LLM generated code in a PR / project. At a certain moment you can no longer guarantee that you stand behind everything. Combine that with the push to do more faster, with less, meaning we're increasing the amount of tech debt we're taking on.
• leecommamichael 11mo ago
  What is the downside of using them to prototype? to generate throwaway code? What do we lose if we default to that behavior?
  • j2kun 11mo ago
    Read the article, which discusses this already, and maybe respond to that.
    • leecommamichael 11mo ago
      I faithfully gave it another read and the article does not explore my question.
  • gpm 11mo ago
    Time wasted on failed prototypes? Understanding that could have been generated by the act of prototyping? Doesn't mean you shouldn't ever do so, but there are tradeoffs that become obvious as soon as you start attempting it.
• davexunit 11mo ago
  > Large language models (LLMs) are an indisputable breakthrough of the last five years Actually a lot people dispute this and I'm sure the author knows that!