3 ms·

> Didn't you just do that to an entire class of engineers Not really. He said "if you claim LLM's are next thing since sliced butter I am doubting your abiliti

by Ygg2 2y ago

> Didn't you just do that to an entire class of engineers Not really. He said "if you claim LLM's are next thing since sliced butter I am doubting your abilities". Which is fair. It's not really a class as much as a group. I've never been wowed over by LLMs. At best they are boilerplate enhancers. At worst they write plausibly looking bullshit that compiles but breaks everything. Give it something truly novel and/or fringe and it will fold like a deck of cards. Even latest research called LLM's benefits into question: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 That said. They are fine at generating commit messages and docs than me.

soulofmischief 2y ago
> Not really. He said "if you claim LLM's are next thing since sliced butter I am doubting your abilities". Which is fair. No, OP said: > Claiming LLMs are a massive boost for coding productivity is becoming a red flag that the claimant has a tenuous grasp on the skills necessary Quotation marks are usually reserved for direct quotes, not paraphrases or straw mans. > I've never been wowed over by LLMs. Cool. I have, and many others have. I'm unsure why your experience justifies invalidating the experiences of others or supporting prejudice against people who have made good use of them. > Give it something truly novel and/or fringe and it will fold like a deck of cards. Few thoughts are truly novel, most are derivative or synergistic. Cutting edge LLMs, when paired with a capable human, are absolutely capable of productive work. I have long, highly technical and cross-cutting discussions with GPT 4o which I simply could not have with any human that I know. Humans like that exist, but I don't know them and so I'm making due with a very good approximation. Your and OP's lack of imagination at the capabilities of LLMs are more telling than you realize to those intimate with them, which is what makes this all quite ironic given that it started from OP making claims about how people who say LLMs massively boost productivity are giving tells that they're not skilled enough.
- Ygg2 2y ago
  > Quotation marks are usually reserved for direct quotes, Not on HN. Customary is to use > paragraph quotes like you did. However I will keep that in mind. > Cool. I have, and many others have. I'm unsure why your experience justifies invalidating the experiences of others If we're both grading a single student (LLM) in same field (programming), and you find it great and I find it disappointing, it means one of us is scoring it wrong. I gave papers that demonstrate its failings, where is your counter-proof? > Your and OP's lack of imagination at the capabilities of LLMs It's not lack of imagination. It's terribleness of results. It can't consistently write good doc comments. I does not understand the code nor it's purpose, but roughly guesses the shape. Which is fine for writing something that's not as formal as code. It can't read and understand specifications, and even generate something as simple as useful API for it. The novel part doesn't have to be that novel just something out of its learned corpus. Like Yaml parser in Rust. Maybe Zig or something beyond it's gobbled data repo. > Few thoughts are truly novel, most are derivative or synergistic. Sure but you still need A mind to derive/synergize the noise of everyday environment into something novel. It can't even do that but remix data into plausibly looking forms. A stochastic parrot. Great for DnD campaign. Shit for code.
  - soulofmischief 2y ago
    > Not on HN. Customary is to use > paragraph quotes like you did. However I will keep that in mind. Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. > If we're both grading a single student (LLM) in same field (programming), and you find it great and I find it disappointing, it means one of us is scoring it wrong. No, it means we have different criteria and general capability for evaluating the LLM. There are plenty of standard criteria which LLMs are pitted against, and we have seen continued improvement since their inception. > It can't consistently write good doc comments. I does not understand the code nor it's purpose, but roughly guesses the shape. Writing good documentation is certainly a challenging task. Experience has led me to understand where current LLMs typically do and don't succeed with writing tests and documentation. Generally, the more organized and straightforward the code, the better. The smaller each module is, the higher the likelihood of a good first pass. And then you can fix deficiencies in a second, manual pass. If done right, it's generally faster than not making use of LLMs for typical workflows. Accuracy also goes down for more niche subject material. All tools have limitations, and understanding them is crucial to using them effectively. > It can't read and understand specifications, and even generate something as simple as useful API for it. Actually, I do this all the time and it works great. Keep practicing! In general, the stochastic parrot argument is oft-repeated but fails to recognize the general capabilities of machine learning. We're not talking about basic Markov chains, here. There are literally academic benchmarks against which transformers have blown away all initial expectations, and they continue to incrementally improve. Getting caught up criticizing the crudeness of a new, revolutionary tool is definitely my idea of unimaginative.
    - Ygg2 2y ago
      > Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. > Writing good documentation is certainly a challenging task. Doctests isn't same as writing documentation. Doctest are the simplest form of documentation. Given function named so and so write API doc + example. It could not even write example that passed syntax check. > Actually, I do this all the time and it works great. Keep practicing! Then you haven't given it interesting/complex enough problems. Also this isn't about practice. It's about its capabilities. > In general, the stochastic parrot argument is oft-repeated but fails to recognize the general capabilities of machine learning. I gave it write YAML parser given Yaml org spec, and it wrote following struct: enum Yaml { Scalar(String), List(Vec<Box<Yaml>>), Map(HashMap<String, Box<Yaml>>), } This is the stochastic parrot in action. Why? Because it tried to pass of JSON like structure as YAML. Whatever LLM's are they aren't intelligent. Or they have attention spans of a fruit fly and can't figure out basic differences.

by Ygg2 2y ago

soulofmischief 2y ago

> Not really. He said "if you claim LLM's are next thing since sliced butter I am doubting your abilities". Which is fair. No, OP said: > Claiming LLMs are a massive boost for coding productivity is becoming a red flag that the claimant has a tenuous grasp on the skills necessary Quotation marks are usually reserved for direct quotes, not paraphrases or straw mans. > I've never been wowed over by LLMs. Cool. I have, and many others have. I'm unsure why your experience justifies invalidating the experiences of others or supporting prejudice against people who have made good use of them. > Give it something truly novel and/or fringe and it will fold like a deck of cards. Few thoughts are truly novel, most are derivative or synergistic. Cutting edge LLMs, when paired with a capable human, are absolutely capable of productive work. I have long, highly technical and cross-cutting discussions with GPT 4o which I simply could not have with any human that I know. Humans like that exist, but I don't know them and so I'm making due with a very good approximation. Your and OP's lack of imagination at the capabilities of LLMs are more telling than you realize to those intimate with them, which is what makes this all quite ironic given that it started from OP making claims about how people who say LLMs massively boost productivity are giving tells that they're not skilled enough.

Ygg2 2y ago
> Quotation marks are usually reserved for direct quotes, Not on HN. Customary is to use > paragraph quotes like you did. However I will keep that in mind. > Cool. I have, and many others have. I'm unsure why your experience justifies invalidating the experiences of others If we're both grading a single student (LLM) in same field (programming), and you find it great and I find it disappointing, it means one of us is scoring it wrong. I gave papers that demonstrate its failings, where is your counter-proof? > Your and OP's lack of imagination at the capabilities of LLMs It's not lack of imagination. It's terribleness of results. It can't consistently write good doc comments. I does not understand the code nor it's purpose, but roughly guesses the shape. Which is fine for writing something that's not as formal as code. It can't read and understand specifications, and even generate something as simple as useful API for it. The novel part doesn't have to be that novel just something out of its learned corpus. Like Yaml parser in Rust. Maybe Zig or something beyond it's gobbled data repo. > Few thoughts are truly novel, most are derivative or synergistic. Sure but you still need A mind to derive/synergize the noise of everyday environment into something novel. It can't even do that but remix data into plausibly looking forms. A stochastic parrot. Great for DnD campaign. Shit for code.
- soulofmischief 2y ago
  > Not on HN. Customary is to use > paragraph quotes like you did. However I will keep that in mind. Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. > If we're both grading a single student (LLM) in same field (programming), and you find it great and I find it disappointing, it means one of us is scoring it wrong. No, it means we have different criteria and general capability for evaluating the LLM. There are plenty of standard criteria which LLMs are pitted against, and we have seen continued improvement since their inception. > It can't consistently write good doc comments. I does not understand the code nor it's purpose, but roughly guesses the shape. Writing good documentation is certainly a challenging task. Experience has led me to understand where current LLMs typically do and don't succeed with writing tests and documentation. Generally, the more organized and straightforward the code, the better. The smaller each module is, the higher the likelihood of a good first pass. And then you can fix deficiencies in a second, manual pass. If done right, it's generally faster than not making use of LLMs for typical workflows. Accuracy also goes down for more niche subject material. All tools have limitations, and understanding them is crucial to using them effectively. > It can't read and understand specifications, and even generate something as simple as useful API for it. Actually, I do this all the time and it works great. Keep practicing! In general, the stochastic parrot argument is oft-repeated but fails to recognize the general capabilities of machine learning. We're not talking about basic Markov chains, here. There are literally academic benchmarks against which transformers have blown away all initial expectations, and they continue to incrementally improve. Getting caught up criticizing the crudeness of a new, revolutionary tool is definitely my idea of unimaginative.
  - Ygg2 2y ago
    > Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. > Writing good documentation is certainly a challenging task. Doctests isn't same as writing documentation. Doctest are the simplest form of documentation. Given function named so and so write API doc + example. It could not even write example that passed syntax check. > Actually, I do this all the time and it works great. Keep practicing! Then you haven't given it interesting/complex enough problems. Also this isn't about practice. It's about its capabilities. > In general, the stochastic parrot argument is oft-repeated but fails to recognize the general capabilities of machine learning. I gave it write YAML parser given Yaml org spec, and it wrote following struct: enum Yaml { Scalar(String), List(Vec<Box<Yaml>>), Map(HashMap<String, Box<Yaml>>), } This is the stochastic parrot in action. Why? Because it tried to pass of JSON like structure as YAML. Whatever LLM's are they aren't intelligent. Or they have attention spans of a fruit fly and can't figure out basic differences.