6 ms·

There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact

There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts. I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism. 1. https://www.nature.com/articles/ncomms14652 https://www.nature.com/articles/ncomms14652

testtesttesst 4y ago
>people misunderstand and anthropomorphize LLMs I agree but there is a certain "god of the gaps" process that's happening here. To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at. When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play. I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning". Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard". Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes. [1]: https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re... [2]: https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduction https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...
kirse 4y ago
I wonder how this code generation “assistant” will impact the coding skills of its users in long term. My brain is already negatively impacted having to juggle 3 separate package managers, know the syntax and inputs for 17+ different dotfiles in a repo and then actively context switch between at least 3-4 languages on a daily basis. I counted it up recently and I've built products in at least 20 languages over the past two+ decades. At some point you really just can't effectively compartmentalize it all and context switch perfectly. I try by using different fonts+colors per lang etc but it still is hard. AI has been a net win for syntax recall just by giving it gentle nudges towards intent.
- jokethrowaway 4y ago
  On the contrary, I'd argue all the switching probably kept you sharp
  - throwaway675309 4y ago
    I sincerely doubt it, humans are historically poor at multitasking, being able to focus on solving problems at an architectural and algorithmic level requires concentration - not having to worry about some bull crap DevOps cloud formation nonsense.
- geraneum 4y ago
  I didn’t mean just any negative impact like getting tired from context switching or feeling exhausted because the workload is high. What I meant was: Will you lose it if you don’t use it? Maybe you won’t, I am not certain. Or maybe you will and you don’t see it now because the “net win for syntax recall just by giving it gentle nudges towards intent” relies on your hard earned ability to understand the code which is gained partly through exercise. You’ve already done your exercise. I wouldn’t form an opinion just yet. For now we can ask the questions. Edit: wording of the first sentence for clarity
  - FeepingCreature 4y ago
    But losing it may free up resources to gain other things. Like how "losing" assembly skills allowed people to dedicate more capability to higher-level approaches.
sbst32 4y ago
Interesting research but I don't really see any reflections on "GPS navigation could impact your brain function negatively" there
Kostchei 4y ago
I mean "generate BS authoritatively" seems fairly similar to "fake it till you make it".. Some people writing code now generate BS authoritatively, some don't.
chubot 4y ago
I just tried ChatGPT today after weeks of hype on Twitter, and I was surprised to NOT be impressed? First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but then the details are frequently wrong. I asked which languages have the F# |> operator, and it told me that OCaml, Elixir, Haskell and Scala do. 2 of them are correct, and 2 appeared wrong, so I googled it. After Googling, then I asked it specifically if Haskell does, and it says no. I ask it if Scala does, and it apologizes for its earlier mistake. It gave me a good although slightly shallow answer about how PyPy works, but then fell down hard for a prolog type checker, and a few other programming queries. It doesn't understand the question. --- It is generally very mealy-mouthed and vague. It often doesn't answer the question directly, but just tells you what it happens to know in the ballpark. I like another framing I saw on Twitter -- it's "mansplaining as a service". If you want someone to confidently explain things you didn't really ask about, and be frequently wrong, that's kinda what it does!! --- After programming, I asked it about Black Sabbath, UFC, travel and biking, health, and got similarly vague/slightly incorrect answers. It confabulates A LOT. The best I can say is that Google is filled with spam now, so chatting with ChatGPT was a relatively fun and focused experience, after I took into account the fact that it is not "honest" and just trying to "please me" with closely related words. I honestly don't get the hype, and I feel like there must have been some coordinated marketing aimed at me ... If this chat is representative of what it can do for programming, then I think this article saying that "programmer salaries will be affected by LLM" is very weird. It is assuming something that simply has NOT happened! --- I mean even Geoff Hinton (who I respect) said several years ago that "we should stop training radiologists", and that plainly did not happen. Likewise with Elon and the fleet of self-driving taxis. I'd honestly be surprised if someone wants to argue that a tech company staffed with LLMs + a few smart people is NOT like those predictions. Of course the blog post doesn't give a time frame, so it can't be falsified. But it appears to be nonsense. (FWIW it doesn't matter to me personally, as I left the industry several years ago. Also FWIW I did work a little bit on deep learning, when the hype was in its upswing.) I agree that a lot of it is anthropomorphism -- projecting some kind of intelligence, knowledge, or intention on what is really a big, expensive bag of data organized to respond to inputs. Maybe there are other models that are better, but I was really, honestly not impressed with ChatGPT. Maybe people were hyping it too much for the last several weeks, and inflated my expectations.
pdntspa 4y ago
> for we are human and vulnerable to anthropomorphism. Stop anthropomorphizing computers, they hate that!
- geraneum 4y ago
  Cupcakes and clouds feel them same. Stop excluding them!
whateveracct 4y ago
Most programmers wish to be stupid. They willingly embrace philosophies predicated on their inability to be conscious and learn. Goddamn do I hate this industry. I work hard to sow dissent in my blue-chip, VC startup though. Guerrilla tactics. And I get to do it raking in a quarter mil a year in my underwear.
- HideousKojima 4y ago
  It's a real struggle, the company I work for was just bought out by a megacorp (in the top 1,000 in the world by employee count, and most posters here would recognize their name) and the sheer lack of motivation to learn of many of the devs both at the acquired company and the acquiring company is staggering. Really basic stuff like "instead of package management let's pass DLLs around over email/Slack/wherever and then complain about all the time we have to spend debugging because of version mismatches." It's like they intentionally make things harder on themselves through willful ignorance.
fexecve 4y ago
> There is research [1] that suggests using GPS navigation could impact your brain function negatively. Wrong, that research only shows that when you use a GPS, you don't learn routes as quickly. That is only equal to "impacts brain function negatively" in a narrow sense, which you should specify rather than use such a broad and scary-sounding phrase!
- geraneum 4y ago
  > … could impact your brain function negatively. Brain function being negatively impacted is not scary on its own and not necessarily permanent. I didn’t mean to scare you. It’s likely you are confusing the above phrase with “your brain will be altered forever” which is not what it means.
  - fexecve 4y ago
    Who said I was scared? I'm just pointing out that the phrase you used it very vague and some interpretations can be scary. It pays to be specific, especially in tech and medicine, you know? Winkyface.
kqr 4y ago
> As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. To be fair, this could be said of a surprising number of the developers I've interviewed too, and I checked -- they were hired by someone.
johnfn 4y ago
> There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. I wonder how code completion and syntax highlighting have altered the brains of programmers negatively as well.
- somethingreen 4y ago
  Have you never seen a project where exactly one use of a term is mistyped across whole project? Guess what tool is responsible for that. Also there's distinct project structure difference between users of tree style project navigation vs find-anywhere or editor tabs vs modal windows/buffers. I mean, the influence is not negative, at least not obviously, but the deformation is there.

testtesttesst 4y ago

>people misunderstand and anthropomorphize LLMs I agree but there is a certain "god of the gaps" process that's happening here. To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at. When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play. I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning". Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard". Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes. [1]: https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re... [2]: https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduction https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...

kirse 4y ago

I wonder how this code generation “assistant” will impact the coding skills of its users in long term. My brain is already negatively impacted having to juggle 3 separate package managers, know the syntax and inputs for 17+ different dotfiles in a repo and then actively context switch between at least 3-4 languages on a daily basis. I counted it up recently and I've built products in at least 20 languages over the past two+ decades. At some point you really just can't effectively compartmentalize it all and context switch perfectly. I try by using different fonts+colors per lang etc but it still is hard. AI has been a net win for syntax recall just by giving it gentle nudges towards intent.

jokethrowaway 4y ago
On the contrary, I'd argue all the switching probably kept you sharp
- throwaway675309 4y ago
  I sincerely doubt it, humans are historically poor at multitasking, being able to focus on solving problems at an architectural and algorithmic level requires concentration - not having to worry about some bull crap DevOps cloud formation nonsense.
geraneum 4y ago
I didn’t mean just any negative impact like getting tired from context switching or feeling exhausted because the workload is high. What I meant was: Will you lose it if you don’t use it? Maybe you won’t, I am not certain. Or maybe you will and you don’t see it now because the “net win for syntax recall just by giving it gentle nudges towards intent” relies on your hard earned ability to understand the code which is gained partly through exercise. You’ve already done your exercise. I wouldn’t form an opinion just yet. For now we can ask the questions. Edit: wording of the first sentence for clarity
- FeepingCreature 4y ago
  But losing it may free up resources to gain other things. Like how "losing" assembly skills allowed people to dedicate more capability to higher-level approaches.

sbst32 4y ago

Interesting research but I don't really see any reflections on "GPS navigation could impact your brain function negatively" there

Kostchei 4y ago

I mean "generate BS authoritatively" seems fairly similar to "fake it till you make it".. Some people writing code now generate BS authoritatively, some don't.

chubot 4y ago

I just tried ChatGPT today after weeks of hype on Twitter, and I was surprised to NOT be impressed? First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but then the details are frequently wrong. I asked which languages have the F# |> operator, and it told me that OCaml, Elixir, Haskell and Scala do. 2 of them are correct, and 2 appeared wrong, so I googled it. After Googling, then I asked it specifically if Haskell does, and it says no. I ask it if Scala does, and it apologizes for its earlier mistake. It gave me a good although slightly shallow answer about how PyPy works, but then fell down hard for a prolog type checker, and a few other programming queries. It doesn't understand the question. --- It is generally very mealy-mouthed and vague. It often doesn't answer the question directly, but just tells you what it happens to know in the ballpark. I like another framing I saw on Twitter -- it's "mansplaining as a service". If you want someone to confidently explain things you didn't really ask about, and be frequently wrong, that's kinda what it does!! --- After programming, I asked it about Black Sabbath, UFC, travel and biking, health, and got similarly vague/slightly incorrect answers. It confabulates A LOT. The best I can say is that Google is filled with spam now, so chatting with ChatGPT was a relatively fun and focused experience, after I took into account the fact that it is not "honest" and just trying to "please me" with closely related words. I honestly don't get the hype, and I feel like there must have been some coordinated marketing aimed at me ... If this chat is representative of what it can do for programming, then I think this article saying that "programmer salaries will be affected by LLM" is very weird. It is assuming something that simply has NOT happened! --- I mean even Geoff Hinton (who I respect) said several years ago that "we should stop training radiologists", and that plainly did not happen. Likewise with Elon and the fleet of self-driving taxis. I'd honestly be surprised if someone wants to argue that a tech company staffed with LLMs + a few smart people is NOT like those predictions. Of course the blog post doesn't give a time frame, so it can't be falsified. But it appears to be nonsense. (FWIW it doesn't matter to me personally, as I left the industry several years ago. Also FWIW I did work a little bit on deep learning, when the hype was in its upswing.) I agree that a lot of it is anthropomorphism -- projecting some kind of intelligence, knowledge, or intention on what is really a big, expensive bag of data organized to respond to inputs. Maybe there are other models that are better, but I was really, honestly not impressed with ChatGPT. Maybe people were hyping it too much for the last several weeks, and inflated my expectations.

pdntspa 4y ago

> for we are human and vulnerable to anthropomorphism. Stop anthropomorphizing computers, they hate that!

geraneum 4y ago
Cupcakes and clouds feel them same. Stop excluding them!

whateveracct 4y ago

Most programmers wish to be stupid. They willingly embrace philosophies predicated on their inability to be conscious and learn. Goddamn do I hate this industry. I work hard to sow dissent in my blue-chip, VC startup though. Guerrilla tactics. And I get to do it raking in a quarter mil a year in my underwear.

HideousKojima 4y ago
It's a real struggle, the company I work for was just bought out by a megacorp (in the top 1,000 in the world by employee count, and most posters here would recognize their name) and the sheer lack of motivation to learn of many of the devs both at the acquired company and the acquiring company is staggering. Really basic stuff like "instead of package management let's pass DLLs around over email/Slack/wherever and then complain about all the time we have to spend debugging because of version mismatches." It's like they intentionally make things harder on themselves through willful ignorance.

fexecve 4y ago

> There is research [1] that suggests using GPS navigation could impact your brain function negatively. Wrong, that research only shows that when you use a GPS, you don't learn routes as quickly. That is only equal to "impacts brain function negatively" in a narrow sense, which you should specify rather than use such a broad and scary-sounding phrase!

geraneum 4y ago
> … could impact your brain function negatively. Brain function being negatively impacted is not scary on its own and not necessarily permanent. I didn’t mean to scare you. It’s likely you are confusing the above phrase with “your brain will be altered forever” which is not what it means.
- fexecve 4y ago
  Who said I was scared? I'm just pointing out that the phrase you used it very vague and some interpretations can be scary. It pays to be specific, especially in tech and medicine, you know? Winkyface.

kqr 4y ago

> As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. To be fair, this could be said of a surprising number of the developers I've interviewed too, and I checked -- they were hired by someone.

johnfn 4y ago

> There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. I wonder how code completion and syntax highlighting have altered the brains of programmers negatively as well.

somethingreen 4y ago
Have you never seen a project where exactly one use of a term is mistyped across whole project? Guess what tool is responsible for that. Also there's distinct project structure difference between users of tree style project navigation vs find-anywhere or editor tabs vs modal windows/buffers. I mean, the influence is not negative, at least not obviously, but the deformation is there.