4 ms·

This seems insane and totally unenforceable. Even when I wrote very single line of code myself, I use AI to ask it about questions regarding the programming la

This seems insane and totally unenforceable. Even when I wrote very single line of code myself, I use AI to ask it about questions regarding the programming language or the library that I use. Banning that is just handicapping yourself. I do like the sentiment. You absolutely do not want people to commit code they don't understand themselves, but the solution isn't to outright ban AI. The solution is to have trusted, knowledgeable developers who are aware of the limits of AI and use it appropriately.

mattsan 3y ago
It should just be banning the use of AI in code reviews
- constantcrying 3y ago
  I don't see what you would even do with an LLM in a code review.
  - mannykannot 3y ago
    The positive view: generate scenarios in which the code will fail. The negative view: write plausible-seeming explanations justifying the code as correct.
    - mattsan 3y ago
      yep this is what I meant. hallucinating, justifying or missing bad stuff. additionally, similar to how large PRs are more likely to just be skimmed and replied with a "LGTM!", an LLM missing some bad stuff but still producing a seemingly thorough review would increase the chance of the bad stuff making its way in. allowing LLMs to write code would be fine if its truly verified by a human, but let another LLM hallucinate and cloud a persons judgement and you've got a problem
  - kragen 3y ago
    i thought i'd find out, so i picked a quasirandom piece of my own code (http://canonical.org/~kragen/sw/dev3/nonarytbl.py http://canonical.org/~kragen/sw/dev3/nonarytbl.py) and gave it to gpt-4 to review, with the prompt: "Please review the above code. How does it work? Is it well designed? Is it efficient? What are its good points and its bad points? How should it be improved? Is it readable and maintainable?" i feel like gpt-4's code review (included below) was mostly correct and useful. however, the efficiency concerns in particular are unfounded, and the python approach to handling errors like those cited is to just let the exception propagate, suboptimal though that is for usability. also, i'm not sure i agree with its high opinion of the modularity, usability, and readability simply pasting gpt-4's partly incorrect analysis into a textarea when someone asks you for a code review would be malpractice, but it could be a useful source of ideas for possible improvements to suggest and possible problems to investigate. also, i've found it to be especially good at coming up with plausible ways that someone could misunderstand my code, which is to say, ways that my code is unclear and therefore could have its clarity improved overall i'd say that gpt-4 is a lot better at reviewing code than at writing code. i've gotten significant improvements out of its code by asking it to review its own code looking for improvements, then asking it what the code would look like with those improvements. also, having it write test suites, then showing it the results of running the test suites — ⁂ — The provided Python script is designed to generate and print arithmetic tables for balanced nonary (base 9) numbers using a unique set of Unicode glyphs. The script covers operations such as multiplication, maximum, addition, and subtraction. Here's a breakdown of how the code works and an evaluation of its design: ### How It Works: 1. `gen_table` Function: - This function generates a table for a given binary operation (`op`) like multiplication or addition. - It takes an optional `name` parameter for labeling the operation in the table header. - The `values` parameter specifies the range of values to be used in generating the table (default is from -4 to 4). - It constructs each row by applying the operation between all pairs of values. 2. `nonary` Function: - Converts an integer into its balanced nonary representation using specified glyphs. - Handles negative numbers and numbers greater than 8 by recursive calls. 3. `layout` Function: - Formats the table into aligned columns for better readability. 4. `paste` Function: - Combines multiple tables horizontally with spaces between them. 5. Main Execution Block (`if __name__ == '__main__'`): - Generates specific tables for different operations and prints them formatted properly. ### Evaluation: Good Points: - Modularity: Functions like `gen_table`, `nonary`, and `layout` are modular, making specific tasks well encapsulated. - Reusability: The functions can be reused or extended easily; e.g., adding new operations or changing glyph sets. - Readability: Despite some complexity in logic, especially in recursion and formatting, variable names and function names are descriptive which aids understanding. Areas for Improvement: 1. Efficiency Concerns: - Recursion in `nonary`: For large numbers, this could lead to deep recursion levels. - Repeated calculation of widths in `layout`: This could be optimized if performance becomes an issue with larger data sets. 2. Error Handling: - There’s no error handling if invalid inputs are passed to functions like non-existent glyphs or unsupported operations. 3. Documentation & Comments: - While there's a detailed module-level docstring explaining high-level concepts, inline comments explaining complex parts (like recursion in `nonary`) would improve maintainability. - More detailed docstrings on what each function expects as input and what it outputs would help other developers understand their interfaces quickly without diving into code details. 4. Testing & Validation: - Adding unit tests would ensure that changes do not break existing functionality and help validate edge cases like very large/small numbers or unusual sequences of operations. 5. Scalability & Generalization: – Currently tailored specifically to balanced nonary system; generalizing this to handle any base might make it more useful as a tool for exploring other numeral systems. 6. User Interface Improvements: – Providing command-line arguments or interactive prompts could allow users to specify ranges or select specific tables without modifying source code directly. ### Conclusion: The script is well-designed concerning modularity and separation of concerns but lacks robustness due to missing error handling and testing components. Enhancements towards documentation, user interaction, scalability/generalization could significantly increase its utility while maintaining ease-of-use for educational purposes or numerical system explorations.
Nullabillity 3y ago
If you don't trust it to write… you probably shouldn't trust it to tell you how to write either.
- constantcrying 3y ago
  Why? It is clearly better at one thing than the other.
  - Nullabillity 3y ago
    Is it, or are you just having a harder time spotting the errors?
    - constantcrying 3y ago
      It seems far superior to Google.
    - kragen 3y ago
      llms like gpt-4 are enormously better than google at finding the relevant functions in an api and telling you how to chain them together to accomplish a given task. sometimes there are errors, but in most cases those are easy to spot, especially if you read the documentation for those functions. for some purposes, testing is enough (and it will happily write you a test suite too) it's kind of annoying to ferret out the bugs it has carefully concealed in the code it wrote for you, though
      - consp 3y ago
        > especially if you read the documentation for those functions Shouldn't that be enough in the first place? RTFM is a real thing.
        kragen 3y ago
        the trouble is that you don't know which functions to read the manual for. last night, for example, i was trying to find out how to get objdump to disassemble an ms-dos .com file with the correct load address of 0x100 rather than 0. google was unhelpful, but gpt-4 told me that the option i was looking for was called `--adjust-vma`. probably if i'd read the objdump manual i would have been able to find it (though the objdump documentation isn't very clear about what `--adjust-vma` will do on a raw binary file), and technically that's only about 15 pages, and spending the requisite half hour reading it would probably produce returns the next time i use objdump. so it's a reasonable idea but it's pretty common for gpt-4 to be able to instantly find the thing you need out of 150 pages or 1500 pages or 15000 pages. or the five things you need, and how to combine them. if you want to know what libraries can probably solve your problem, asking gpt-4 is a much better idea than reading the manuals of all possible libraries
TheFreim 3y ago
Using LLMs as a glorified documentation search is the best use case I've found. Sometimes I've forgotten something, or never knew it in the first place, so being able to use human language to describe a function or library I'm looking for is very useful. Once I've done that I move to the real documentation as fast as possible.
- madeofpalk 3y ago
  LLMs have mostly replaced Stack Overflow for me. I find I can get to a working solution quicker using ChatGPT compared to Google + Stack Overflow.
- jeltz 3y ago
  The funny thing is that if a function does not exist they often just make up one. But that happened on Stack Overflow too. Most of the answers in areas I knew had incorrect answers on SO. Only in areas where I am a beginner there is usually correct answers.

mattsan 3y ago

It should just be banning the use of AI in code reviews

constantcrying 3y ago
I don't see what you would even do with an LLM in a code review.
- mannykannot 3y ago
  The positive view: generate scenarios in which the code will fail. The negative view: write plausible-seeming explanations justifying the code as correct.
  - mattsan 3y ago
    yep this is what I meant. hallucinating, justifying or missing bad stuff. additionally, similar to how large PRs are more likely to just be skimmed and replied with a "LGTM!", an LLM missing some bad stuff but still producing a seemingly thorough review would increase the chance of the bad stuff making its way in. allowing LLMs to write code would be fine if its truly verified by a human, but let another LLM hallucinate and cloud a persons judgement and you've got a problem
- kragen 3y ago
  i thought i'd find out, so i picked a quasirandom piece of my own code (http://canonical.org/~kragen/sw/dev3/nonarytbl.py http://canonical.org/~kragen/sw/dev3/nonarytbl.py) and gave it to gpt-4 to review, with the prompt: "Please review the above code. How does it work? Is it well designed? Is it efficient? What are its good points and its bad points? How should it be improved? Is it readable and maintainable?" i feel like gpt-4's code review (included below) was mostly correct and useful. however, the efficiency concerns in particular are unfounded, and the python approach to handling errors like those cited is to just let the exception propagate, suboptimal though that is for usability. also, i'm not sure i agree with its high opinion of the modularity, usability, and readability simply pasting gpt-4's partly incorrect analysis into a textarea when someone asks you for a code review would be malpractice, but it could be a useful source of ideas for possible improvements to suggest and possible problems to investigate. also, i've found it to be especially good at coming up with plausible ways that someone could misunderstand my code, which is to say, ways that my code is unclear and therefore could have its clarity improved overall i'd say that gpt-4 is a lot better at reviewing code than at writing code. i've gotten significant improvements out of its code by asking it to review its own code looking for improvements, then asking it what the code would look like with those improvements. also, having it write test suites, then showing it the results of running the test suites — ⁂ — The provided Python script is designed to generate and print arithmetic tables for balanced nonary (base 9) numbers using a unique set of Unicode glyphs. The script covers operations such as multiplication, maximum, addition, and subtraction. Here's a breakdown of how the code works and an evaluation of its design: ### How It Works: 1. `gen_table` Function: - This function generates a table for a given binary operation (`op`) like multiplication or addition. - It takes an optional `name` parameter for labeling the operation in the table header. - The `values` parameter specifies the range of values to be used in generating the table (default is from -4 to 4). - It constructs each row by applying the operation between all pairs of values. 2. `nonary` Function: - Converts an integer into its balanced nonary representation using specified glyphs. - Handles negative numbers and numbers greater than 8 by recursive calls. 3. `layout` Function: - Formats the table into aligned columns for better readability. 4. `paste` Function: - Combines multiple tables horizontally with spaces between them. 5. Main Execution Block (`if __name__ == '__main__'`): - Generates specific tables for different operations and prints them formatted properly. ### Evaluation: Good Points: - Modularity: Functions like `gen_table`, `nonary`, and `layout` are modular, making specific tasks well encapsulated. - Reusability: The functions can be reused or extended easily; e.g., adding new operations or changing glyph sets. - Readability: Despite some complexity in logic, especially in recursion and formatting, variable names and function names are descriptive which aids understanding. Areas for Improvement: 1. Efficiency Concerns: - Recursion in `nonary`: For large numbers, this could lead to deep recursion levels. - Repeated calculation of widths in `layout`: This could be optimized if performance becomes an issue with larger data sets. 2. Error Handling: - There’s no error handling if invalid inputs are passed to functions like non-existent glyphs or unsupported operations. 3. Documentation & Comments: - While there's a detailed module-level docstring explaining high-level concepts, inline comments explaining complex parts (like recursion in `nonary`) would improve maintainability. - More detailed docstrings on what each function expects as input and what it outputs would help other developers understand their interfaces quickly without diving into code details. 4. Testing & Validation: - Adding unit tests would ensure that changes do not break existing functionality and help validate edge cases like very large/small numbers or unusual sequences of operations. 5. Scalability & Generalization: – Currently tailored specifically to balanced nonary system; generalizing this to handle any base might make it more useful as a tool for exploring other numeral systems. 6. User Interface Improvements: – Providing command-line arguments or interactive prompts could allow users to specify ranges or select specific tables without modifying source code directly. ### Conclusion: The script is well-designed concerning modularity and separation of concerns but lacks robustness due to missing error handling and testing components. Enhancements towards documentation, user interaction, scalability/generalization could significantly increase its utility while maintaining ease-of-use for educational purposes or numerical system explorations.

Nullabillity 3y ago

If you don't trust it to write… you probably shouldn't trust it to tell you how to write either.

constantcrying 3y ago
Why? It is clearly better at one thing than the other.
- Nullabillity 3y ago
  Is it, or are you just having a harder time spotting the errors?
  - constantcrying 3y ago
    It seems far superior to Google.
  - kragen 3y ago
    llms like gpt-4 are enormously better than google at finding the relevant functions in an api and telling you how to chain them together to accomplish a given task. sometimes there are errors, but in most cases those are easy to spot, especially if you read the documentation for those functions. for some purposes, testing is enough (and it will happily write you a test suite too) it's kind of annoying to ferret out the bugs it has carefully concealed in the code it wrote for you, though
    - consp 3y ago
      > especially if you read the documentation for those functions Shouldn't that be enough in the first place? RTFM is a real thing.
      - kragen 3y ago
        the trouble is that you don't know which functions to read the manual for. last night, for example, i was trying to find out how to get objdump to disassemble an ms-dos .com file with the correct load address of 0x100 rather than 0. google was unhelpful, but gpt-4 told me that the option i was looking for was called `--adjust-vma`. probably if i'd read the objdump manual i would have been able to find it (though the objdump documentation isn't very clear about what `--adjust-vma` will do on a raw binary file), and technically that's only about 15 pages, and spending the requisite half hour reading it would probably produce returns the next time i use objdump. so it's a reasonable idea but it's pretty common for gpt-4 to be able to instantly find the thing you need out of 150 pages or 1500 pages or 15000 pages. or the five things you need, and how to combine them. if you want to know what libraries can probably solve your problem, asking gpt-4 is a much better idea than reading the manuals of all possible libraries

TheFreim 3y ago

Using LLMs as a glorified documentation search is the best use case I've found. Sometimes I've forgotten something, or never knew it in the first place, so being able to use human language to describe a function or library I'm looking for is very useful. Once I've done that I move to the real documentation as fast as possible.

madeofpalk 3y ago
LLMs have mostly replaced Stack Overflow for me. I find I can get to a working solution quicker using ChatGPT compared to Google + Stack Overflow.
jeltz 3y ago
The funny thing is that if a function does not exist they often just make up one. But that happened on Stack Overflow too. Most of the answers in areas I knew had incorrect answers on SO. Only in areas where I am a beginner there is usually correct answers.