3 ms·

> How do you know this was a "slop" if you did not try it? Because I have eyes and can look at the code for 2 seconds. It's not very difficult to check for the

> How do you know this was a "slop" if you did not try it? Because I have eyes and can look at the code for 2 seconds. It's not very difficult to check for the hallmarks of careless slop code. If you can't tell in a few seconds then you can continue testing it out just like any actual project.

minimaxir 9mo ago
And what are those hallmarks in this case? There is no heuristic that is applicable for every programming language. Unfortunately, ratatui requires a lot of verbose code that may be indistinguishable from LLM generated code: https://ratatui.rs/examples/apps/demo/ https://ratatui.rs/examples/apps/demo/
- acedTrex 9mo ago
  Easiest one is ridiculous comments, always a dead ringer for an LLM. ESPECIALLY when its from a plan and comments '// STEP 2: ...' Like here in this posts repo https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L232 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... This a dead ringer for LLM slop that someone didnt even care enough to go through and clean up. Edit: Just to keep it going, you ever seen a human write dependencies like this for a small toy tui? https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/Cargo.toml https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... Edit2: Sure why not, lets reimplement randomly a custom jsonpath when serde_json_path exists https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/app.rs#L253 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...
  - minimaxir 9mo ago
    That's not a smoking gun. I've definitely seen pre-2023 open source code mentioning steps in their comments. Even though that file also has a lot of tautological comments which are more indicative of LLM coding, it's not a smoking gun either: the frequency of comments is an editorial decision which has pros and cons. It's the equivalent of calling something an AI generated images just because the fingers are weird, and requires a judgment more concrete than "I have eyes." > you ever seen a human write dependencies like this for a small toy tui? Yes? That's just TOML syntax. I'm not sure which dependency in that list is excessive, especially for something that has to handle HTTP requests. If you mean adding a comment header for each section, then that's a better argument, but see argument above.
    - acedTrex 9mo ago
      I mean if you want bury your head in the sand and try to play semantics over if its ENOUGH proof or not go right ahead. But it's more than LLM enough for anyone who has experience with them to conclude the LLM drove the majority of the output. Hence, slop
      - minimaxir 9mo ago
        To be clear, I agree that there was likely AI assistance with the code (as it will be a safe assumption going forward in SWE given the progress agentic LLMs have achieved in the past few months), but a) that doesn't intrinsically mean it's slop and b) the "all AI is slop" bit is intellectually lazy at best especially without objective evidence and doesn't lead to constructive HN discussion.
        acedTrex 9mo ago
        > a) that doesn't intrinsically mean it's slop If it can quickly and easily be identified as LLM code then yes, it is intrinsically slop and of no value. The person who submitted it did not even value it enough to look at/clean it up. Why would anyone else care to look at it. If it is LLM generated but then HAS BEEN cleaned up then you cant immediately see the LLM and it passes the check anyways.
    - mahogany 9mo ago
      The more you see and review LLM-generated code, the more you can detect its fingerprints. Obviously you're not going to prove this is LLM-generated. I wouldn't bet $1M that it is. This could be 100% human made. But read the same link from above: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L232 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52.... LLMs leave temporal comments like "// Now do X", or "// Do X using the new Y", as responses to prompts like "Can you do X with Y instead?". or below: "// Auto-refresh every 5 seconds (only in Normal mode)". I would guess this comment was during a response to a prompt like: "can you only auto-refresh in Normal mode?" Sometimes there are tautological comments and sometimes not: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L77 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... ``` // Get log file path let log_path = get_log_path(); ``` This is another signal to me that there is less human influence over the project. No, none of these are a smoking gun. Also none of this means it was completely vibe coded. To me personally, the worrying part is that these patterns signal that perhaps human eyes were never on that section of the code, or at least the code was not considered carefully. For a toy app, who cares? For something that ingests your AWS creds, it's more of a red flag. Edit: changed the language a bit to sound less sardonic. My comment is more about LLM signals than a judgment on LLM usage.
  - justamaze 9mo ago
    [flagged]
  - viraptor 9mo ago
    > Easiest one is ridiculous comments, always a dead ringer for an LLM. > ESPECIALLY when its from a plan and comments '// STEP 2: ...' There are people who actually program that way. The most extreme I know was Bogdan Iancu from OpenSIPS who I've seen create functions, write step-by-step comments for what they will do, then fill out the implementation. It's just a signal, not a certain thing.
- joshka 9mo ago
  https://ratatui.rs/examples/apps/demo/ https://ratatui.rs/examples/apps/demo/ is pretty much the oldest untouched remnant of code from tui-rs days (pre-ratatui fork in Feb 2023). Ratatui itself has a lot of much nicer AI generated code in it since then ;) We've also done a bunch of things to help drive down some of the boilerplate (not all of it mind you - as it's a library, not a framework like other TUI libs)

minimaxir 9mo ago

And what are those hallmarks in this case? There is no heuristic that is applicable for every programming language. Unfortunately, ratatui requires a lot of verbose code that may be indistinguishable from LLM generated code: https://ratatui.rs/examples/apps/demo/ https://ratatui.rs/examples/apps/demo/

acedTrex 9mo ago
Easiest one is ridiculous comments, always a dead ringer for an LLM. ESPECIALLY when its from a plan and comments '// STEP 2: ...' Like here in this posts repo https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L232 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... This a dead ringer for LLM slop that someone didnt even care enough to go through and clean up. Edit: Just to keep it going, you ever seen a human write dependencies like this for a small toy tui? https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/Cargo.toml https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... Edit2: Sure why not, lets reimplement randomly a custom jsonpath when serde_json_path exists https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/app.rs#L253 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...
- minimaxir 9mo ago
  That's not a smoking gun. I've definitely seen pre-2023 open source code mentioning steps in their comments. Even though that file also has a lot of tautological comments which are more indicative of LLM coding, it's not a smoking gun either: the frequency of comments is an editorial decision which has pros and cons. It's the equivalent of calling something an AI generated images just because the fingers are weird, and requires a judgment more concrete than "I have eyes." > you ever seen a human write dependencies like this for a small toy tui? Yes? That's just TOML syntax. I'm not sure which dependency in that list is excessive, especially for something that has to handle HTTP requests. If you mean adding a comment header for each section, then that's a better argument, but see argument above.
  - acedTrex 9mo ago
    I mean if you want bury your head in the sand and try to play semantics over if its ENOUGH proof or not go right ahead. But it's more than LLM enough for anyone who has experience with them to conclude the LLM drove the majority of the output. Hence, slop
    - minimaxir 9mo ago
      To be clear, I agree that there was likely AI assistance with the code (as it will be a safe assumption going forward in SWE given the progress agentic LLMs have achieved in the past few months), but a) that doesn't intrinsically mean it's slop and b) the "all AI is slop" bit is intellectually lazy at best especially without objective evidence and doesn't lead to constructive HN discussion.
      - acedTrex 9mo ago
        > a) that doesn't intrinsically mean it's slop If it can quickly and easily be identified as LLM code then yes, it is intrinsically slop and of no value. The person who submitted it did not even value it enough to look at/clean it up. Why would anyone else care to look at it. If it is LLM generated but then HAS BEEN cleaned up then you cant immediately see the LLM and it passes the check anyways.
  - mahogany 9mo ago
    The more you see and review LLM-generated code, the more you can detect its fingerprints. Obviously you're not going to prove this is LLM-generated. I wouldn't bet $1M that it is. This could be 100% human made. But read the same link from above: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L232 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52.... LLMs leave temporal comments like "// Now do X", or "// Do X using the new Y", as responses to prompts like "Can you do X with Y instead?". or below: "// Auto-refresh every 5 seconds (only in Normal mode)". I would guess this comment was during a response to a prompt like: "can you only auto-refresh in Normal mode?" Sometimes there are tautological comments and sometimes not: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52356cf61268529742c5ea5a/src/main.rs#L77 https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52... ``` // Get log file path let log_path = get_log_path(); ``` This is another signal to me that there is less human influence over the project. No, none of these are a smoking gun. Also none of this means it was completely vibe coded. To me personally, the worrying part is that these patterns signal that perhaps human eyes were never on that section of the code, or at least the code was not considered carefully. For a toy app, who cares? For something that ingests your AWS creds, it's more of a red flag. Edit: changed the language a bit to sound less sardonic. My comment is more about LLM signals than a judgment on LLM usage.
- justamaze 9mo ago
  [flagged]
- viraptor 9mo ago
  > Easiest one is ridiculous comments, always a dead ringer for an LLM. > ESPECIALLY when its from a plan and comments '// STEP 2: ...' There are people who actually program that way. The most extreme I know was Bogdan Iancu from OpenSIPS who I've seen create functions, write step-by-step comments for what they will do, then fill out the implementation. It's just a signal, not a certain thing.
joshka 9mo ago
https://ratatui.rs/examples/apps/demo/ https://ratatui.rs/examples/apps/demo/ is pretty much the oldest untouched remnant of code from tui-rs days (pre-ratatui fork in Feb 2023). Ratatui itself has a lot of much nicer AI generated code in it since then ;) We've also done a bunch of things to help drive down some of the boilerplate (not all of it mind you - as it's a library, not a framework like other TUI libs)