15 ms·

Things software developers should learn about learning

507 points by jarmitage 3y ago

selimthegrim 3y ago
500 error
acosmism 3y ago
hug of death: https://web.archive.org/web/20231227005023/https://cacm.acm.org/magazines/2024/1/278891-10-things-software-developers-should-learn-about-learning/fulltext https://web.archive.org/web/20231227005023/https://cacm.acm....
- emmanueloga_ 3y ago
  It seems they have clouflare support, but must be misconfigured somehow? Also rails? 1.4 secs seems like a long response time, even for a rails app. I guess even the pros get websites wrong sometimes :-p Cache-Control: private, max-age=0, must-revalidate Cf-Cache-Status: DYNAMIC Cf-Ray: 83be9f774919fa8a-SJC Content-Encoding: gzip Content-Type: text/html; charset=utf-8 Date: Wed, 27 Dec 2023 03:54:35 GMT Server: cloudflare Status: 200 OK Vary: Accept-Encoding X-Powered-By: Phusion Passenger X-Runtime: 1.42775
readthenotes1 3y ago
Iirc, System 1/2 thinking has been largely de-based (i.e., it could be a right model, but the evidence for that is tenuous) https://www.psychologytoday.com/us/blog/a-hovercraft-full-of-eels/202103/the-false-dilemma-system-1-vs-system-2 https://www.psychologytoday.com/us/blog/a-hovercraft-full-of... Makes me wonder if item (0) should be "Suspect every one" (with a nod to Maria Gambrelli, of course)
- jarmitage 3y ago
  Article footnote b. states: "b. Parts of Kahneman's book were undermined by psychology's "replication crisis," which affected some of its findings, but not the idea of system 1 and 2." Would have been a better footnote if additional references were provided for the latest in that area of discourse.
darkclouds 3y ago
[dead]
mewpmewp2 3y ago
The article looks very accurate to me. Having read all of that, I find myself agreeing with most of it, which I think usually would not be the case for me with an article like this.
- thebenedict 3y ago
  I was also pleasantly surprised by the quality of this article, especially the nuance around the value of a growth mindset. It's consistent with my experience of having had to work pretty hard to learn programming.
  - mewpmewp2 3y ago
    It seems like a clickbait title combined with nuanced points of high quality and relevancy.
  - mobiuscog 3y ago
    > the value of a growth mindset This was buzzword bingo 2023. Hopefully it dies in 2024.
- deleted 3y ago
  [deleted]
bedobi 3y ago
allow me to shamelessly plug something I wrote on the topic https://gist.github.com/androidfred/75629dfda63180b6f0a0eaa4655ec5f0 https://gist.github.com/androidfred/75629dfda63180b6f0a0eaa4... no data, no research :P just anecdote and opinon
- eszed 3y ago
  I really like what you've written. It jibes with my own experience and opinions. If I were to expand it a bit further, I'd point out that there are teaching methodologies that seem to be derived from these paradigms. In one, an apprentice copies the master as well as they are able, sometimes with no explicit instruction at all, until they've filled in the details for themselves; in the other the student memorizes facts and methods, often by rote, until they build up the big picture for themselves. I've learned within (and from) both approaches, and find each - when rigidly followed - to be highly frustrating! In my own pedagogy I try to hold both in mind, and calibrate students' learning paths accordingly. When they're mired in detail, I re-orient them towards the end goal; when they're not sure what to do, or how to do it, I guide them through the next step. It's a lot more effort, because you have to pay attention to them, and not only the subject, which most teachers would prefer not (or don't know how) to do. (On a side note, I'll say that - in the field(s?) where I'm an expert - nearly all of the pleasure comes from refining the last 2% of the details. It's never going to be perfect, but it can always be incrementally better. It's not "productive", certainly in a commercial sense, but it's immensely satisfying.)
alphazard 3y ago
In section 7: > Problem Solving is Not a Generic Skill There is going to be some difference between solving problems in a specific domain and solving problems generally (which is what TFA argues for). And since we really care about the specific domain of software engineering, it makes sense to pry open that difference when possible. However problem solving in the general case is very close to fluid intelligence and IQ. Some interpretations claim that intelligence in humans is just problem solving, and that problem solving is most of what is captured by g [0]. All problem solving will be positively correlated with all other problem solving, and you would never expect to see someone good at one, but not the other. In section 9 they cite the research on programming ability and its (expected) relation to general intelligence. I'm not sure how much of a distinction there is to draw here. Psychometrics has historically been filled with attempts to factor out additional clusters from things like g e.g. multiple intelligences. Those findings often fail to replicate. Section 7 seems more like an attempt to draw a distinction without a difference. While section 9 seems like a standard summary of the research (like most things, a mixture of innate intelligence and cumulative experience). [0] https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
- Kinrany 3y ago
  Their point is that you can't learn general problem solving. IQ is not a skill.
  - ozim 3y ago
    I would say you can learn general problem solving. There are strategies that help with that an IQ helps with finding patterns. That said it still would be junior level problem solving in unknown domain. Expert will always run circles around newcomers. I most likely could switch fields but I don’t want to spend 5 years getting experience. Even if I have decent generic solving skills it might take me less time but still- I would have to be really interested in the topic.
    - Jensson 3y ago
      > Expert will always run circles around newcomers. That isn't true, there are many problems less than mediocre experts fails to solve that a smart junior can manage to solve. Experts are great at common problems but not great at rarer ones, non-standard problems depend more on natural ability than experience. Of course smart experts can also solve those problems, but you didn't say a smart expert, you just said experts in general. The ability to generalize based on experience requires intelligence. That is why you have so many expert programmers who can't solve problems well, because they lack the ability to generalize their experience well enough to apply it to new problems.
quickthrower2 3y ago
> Long-term memory is where information is permanently stored and is functionally limitless; in that sense, it functions somewhat like a computer's disk storage. Doesn't jibe with my experience.
- jpoesen 3y ago
  Having trouble retrieving the knowledge does not mean the knowledge is no longer there.
revskill 3y ago
To me, intelligence in general is ability of solving things in a generic and abstract way.
- deleted 3y ago
  [deleted]
wisemang 3y ago
I appreciate the succinct detail they go into regarding how the concept of “learning styles” has been debunked. I’ve always considered myself more of a visual learner, and seeing posts here and elsewhere about this supposedly not being a thing has never sat well with me (though I never delved into the actual research). Turns out I do in fact agree with this explanation; that it’s more about what’s being taught that should dictate the how: > While learners have preferred styles, effective instruction matches the content, not learning styles. A science class should use graphs to present data rather than verbal descriptions, regardless of visual or auditory learning styles, just like cooking classes should use hands-on practices rather than reading, whether learners prefer a kinesthetic style or not.
- oatmeal1 3y ago
  Courses teaching music theory through visual depictions, take note!
  - NetOpWibby 3y ago
    This is why I failed music class in high-school. I was the only non-band student so the work was accelerated.
    - jacobolus 3y ago
      This is why trying to rank/judge/grade everyone by a uniform standard is almost universally terrible. Students should be encouraged to try their best in every subject, allowed to make the mistakes they are naturally going to make at whatever level they are currently, and helped to improve over time. Punishing people for being less prepared than peers who did more practice or for making ordinary and expected mistakes actively gets in the way of their learning, as well as making them feel terrible. It's pretty bad for the students who are more prepared as well, as many of them internalize the idea that they are inherently good at some things and inherently bad at others, which is sometimes temporarily gratifying but often stops them from pushing themselves to try anything new or hard.
tbwriting 3y ago
This is a fantastic piece. If you find yourself thinking “okay, so the brain is different — now what? What should I actually do to learn better?” I wrote about this last year. Pardon the clickbaity title, HackerNoon changed it up on me. https://hackernoon.com/the-four-rs-how-to-become-a-good-programmer https://hackernoon.com/the-four-rs-how-to-become-a-good-prog...
- james-revisoai 3y ago
  I'd be interested in discussing this with you - we think alike, although, having background in psychology too, I was at first sceptical of your use of terms like "Rearranging". But I like your summary and how you orient it to the programming domain but also reference examples of good learning app techniques. I am working on an app that makes many decisions in this area, and is truly trying to attempt the psychology of learning ethically to university studying (including parts of computer science you can learn without actually "doing" the coding etc). I have a few ideas and new learning user flows I would love to get your feedback on as well as have a wider discussion and nerd out a bit on psychology studies I think you'll find revealing, would you be up to talk? As a quick peak: My own thesis on student learning broadly begins with three e's: "Effectiveness, Enjoyment(or motivation), and Environment", as I believe the first two are necessary, and the third one an augmentation, as properties of effective learning systems for people. This is based on combining my psychology knowledge (mostly the first and last two E's) with my experience volunteering and being a TA and hearing from students who through 90% of the semester struggle with the middle e - Enjoyment - more than others, or to the point where it prevents applying the others(not motivated to use effective techniques, not motivated to go to the library when procrastinating, etc) correctly. I am super interested in combining this with the teachers view and how they work too (e.g. curriculum design, personalised tutoring)
rTX5CMRXIfFG 3y ago
> a fact does not exist in a binary state of either definitively known or unknown; it can exist in intermediate states. We can forget things we previously knew, and knowledge can be unreliable, especially when recently learned. I’m sorry, what? If I only vaguely understood physics and believed now that the earth is flat, that would neither count as knowledge in some intermediate state. Knowing is binary—you either know or you don’t, no matter how strongly believe what you think.
- Cerium 3y ago
  An example: What did you eat for dinner? If you know the answer for sure, go back a day, or another day, etc. At some number of days you will have an idea of what you ate but not be entirely sure. You certainly knew what you ate when you were eating it, but now?
  - rTX5CMRXIfFG 3y ago
    If I don’t know that now but knew it then, then knowledge is still binary is it not?
    - RoyalHenOil 3y ago
      You don't have perfect recollection one day and then no recollection whatsoever the next day. Memories become vague as they deteriorate. For example, you may remember something you learned in 4th grade, but you probably don't remember exactly how the teacher explained it or what questions about it were on the test.
      - master-lincoln 3y ago
        Ok, but then I don't know how I was taught that thing, even though I know the thing. Still binary
      - deleted 3y ago
        [deleted]
      - 3y ago
zubairq 3y ago
Some good points, programming t00 much can sometimes distort my thinking !
TibbityFlanders 3y ago
https://today.ucsd.edu/story/a-new-replication-crisis-research-that-is-less-likely-be-true-is-cited-more#:~:text=In%20psychology%2C%20only%2039%20percent%20of%20the%20100%20experiments%20successfully%20replicated https://today.ucsd.edu/story/a-new-replication-crisis-resear.... > Papers that cannot be replicated are cited 153 times more because their findings are interesting, according to a new UC San Diego study > In psychology, only 39 percent of the 100 experiments successfully replicated.
- rmbyrro 3y ago
  Was this study replicated?
  - deleted 3y ago
    [deleted]
vinay_ys 3y ago
> Experts Recognize, Beginners Reason > System 1 is fast and driven by recognition, relying upon pattern recognition in long-term memory, while system 2 is slower and focused on reasoning, requiring more processing in working memory. Interestingly, today, LLMs are augmentation for someone's weak system 1, and allowing them to focus solely on strengthening their system 2. LLMs and popular/cheap/generalizable AI today suck at system 2. So, if you are really good at system 2 and suck at system 1, the next decade is going to be amazing for you.
- deleted 3y ago
  [deleted]
- TibbityFlanders 3y ago
  (removed)
  - Always42 3y ago
    can you post enough about your local LLM setup for me to google/rep. this? There is so much out there for LLM's parsing is a pain.
    - TibbityFlanders 3y ago
      For audio generation I recommend Bark. I am getting 14 seconds of audio that is about a third of eleven labs quality in about 2 minutes. This is happening on a Windows 10 Dell, with 32gb of RAM, an i5, and an Nvidia 1050 GeForce with 4gb of vram. I'm also able to decently run local LLMs because of llama.cpp and other libraries that can share models been ram and vram. There are other tools that can help with this as well including Ollama. I suggest subscribing to r/localLLAMA. I also suggest using Bing Copilot in Edge with allowed access to the page you're viewing. I often use it to find new GitHub libraries and to give me first steps to be able to start using a new framework.
- rramadass 3y ago
  Also see Recognition-Primed Decision Model: https://en.wikipedia.org/wiki/Recognition-primed_decision https://en.wikipedia.org/wiki/Recognition-primed_decision
abaymado 3y ago
> Research into chess found little or no effect of learning it on other academic and cognitive skills, and the same is true for music instruction and cognitive training.This inability to transfer problem-solving skills is why "brain training" is ineffective for developing general intelligence. I would disagree with this premise, deep work and the forbidden word "discipline" are problem solving skills that are learned and need constant training. They are just as important as any other specific skill needed for the subject. Thus, making some problem-solving skills indeed free flowing from subject to subject.
- watwut 3y ago
  I guess that then implication is that learning chess ot music does not make you more disciplined either.
  - sitkack 3y ago
    Learning any subject in any domain, deeply has value, and that learning is transferable. I think when the analysis is done, saying that chess doesn’t make one smarter per se, the researchers are starting from a baseline where student a and student B are equal in all regards, and in addition to student B has learned to chess. While student a might not know chess they could have spent an equal amount of time learning something else and it’s that time learning things that’s important not necessarily the subject matter.
rmrf100 3y ago
sometimes it's really astonishing to see that, children grow up day by day then quikly understond the logic which hard to them before.
xjay 3y ago
Warning from Daniel Kahneman on "System 1" and "System 2". [1] > I'm going to use "System 1" and "System 2", absolutely as homunculi. [...] They don't exist. [...] Don't look for them in the brain, because they are not two systems in the brain, of which one does one, and the other does the other. So why am I using this terrible language? I'm using it because I think it's helpful. It fits the way our minds work, and to explain the background of that decision--of why I use "System 1" and "System 2"--I refer you to a very good book. [...] It's by Joshua Foer and it's called "Moonwalking with Einstein". [2] [1] https://www.youtube.com/watch?v=CjVQJdIrDJ0&t=1224s https://www.youtube.com/watch?v=CjVQJdIrDJ0&t=1224s [2] https://en.wikipedia.org/wiki/Moonwalking_with_Einstein https://en.wikipedia.org/wiki/Moonwalking_with_Einstein The article does put these terms in quotes.
- BeetleB 3y ago
  His book Thinking, Fast and Slow makes it clear that these are conveniences and not real systems in the brain.
aksss 3y ago
> “Higher capacity enables faster learning, but our unlimited long-term memory removes limitations on how much we could ultimately learn in total.1 Expert programmers may have low or high working memory capacity but it is the contents of their long-term memory that make them experts.” I’ve always told “kids” that you can learn a lot about systems but with programming and IT systems in general, there is just no substitute for getting the raw mileage of having seen many permutations, iterations, and manifestations. It’s not a dig, but a statement made in the context of encouraging new people to stick with it and not beat themselves up too much when they inevitably get overwhelmed by the scope of their unknowns or roll a critical miss. It’s all about learning, all the time.
auggierose 3y ago
I like the paper, and I think it makes many valuable points. I especially like the semantic wave model, because that's exactly how you learn mathematics. And it also explains why (high-quality) abstractions are so important. But of course everything has to be taken with a grain of salt. For example, their recommendations at the end on how to access papers is not very good. Ever heard of Sci-Hub and VPNs? It is obvious why they cannot mention this in their paper, but it is also equally obvious then that if there was evidence linking race or gender with programming ability, they would not mention it, for pretty much the same reasons. I also don't like their example of achieving a Nobel Prize as something that practically no one can attain. Yes, that's true, but that is because Nobel Prizes are artificially limited to a few people a year. I think many, many more people can achieve that level of expertise than just a few per year.
- tsimionescu 3y ago
  It should be noted that the very idea of studying links between "race" (skin color) and intelligence is deeply suspect. No one is studying things like correlation between hair wavyness and intelligence, or penis length and intelligence, etc. And yet skin color is exactly the same kind of trait: something you were born with and that is tied to the genetic baggage of your parents, with no remotely plausible direct link with intelligence (unlike, say, skull size, which at least had some plausible priors). You can even (somewhat) change the color of your skin if you really want. Now, there may be, at least in principle, families, and by extension populations, who are more or less intelligent, on average, than others. But "race" has little to do with that: these types of studies only look at skin color as a proxy for population, and that is obviously silly on genetic grounds. Dark skin is a dominant trait, the children of a lighter skinned parent and a darker skin parent will usually have darker skin and be assigned the racial category "black" in such research, even though genetically they are just as much a member of the lighter skinned population as the darker skinned one. Even worse, this often persists across a few generations, so a child with 1 dark skinned grand parent and 3 light skinned ones will often be dark skinned themselves, and thus be called "black" in many such studies. So, the reason you should be very very much concerned with citing studies that find links between "race" and intelligence is that the very premise is wrong in the vast majority of the literature.
rand1239 3y ago
I wonder whether including this article in GPT-5 training data would improve the performance of the model.
- Jensson 3y ago
  Why would it? It just learns by making this kind of text more likely to appear. If you want it to be smart it needs to see problems and strategies to solve them, not see explanations. Regurgitating explanations isn't useful, following problem solving patterns is. So to learn from this article it would need to see the kind of thinking required to write the article.
  - rand1239 3y ago
    Okay. Are humans different than this? Thoughts -> Action -> Thoughts Thoughts are generated by brain. You don't control how the neurons are fired in the brain. As far as you are concerened you experience the thoughts, the sounds, images etc which. So what exactly is the point of this article?
    - Jensson 3y ago
      When you read something you typically imagine a lot of examples that the thing you read could apply to. The way we train LLMs today doesn't do this, it just reads the text without thinking further about it. That means to make an LLM learn you have to feed it examples instead of descriptions, like examples of problems being solved etc. Maybe we could make LLM training do such things in the future, but it doesn't do it today and it is hard to do that in practice since generating examples on the fly for descriptions isn't easy to do in an intelligent way. I think that is a core part of generalizing knowledge so probably one of the keys we need to get to AGI.
remram 3y ago
Why are "key insights" presented as an ugly GIF? Just so it can't be seen by screen readers? Or maybe robots? If you really want to put your text in an image, can you pick a decent font and not make it blurry? Puzzling.
- auggierose 3y ago
  Read the PDF instead. I always do, cannot be bothered to read ACMs low quality HTML.
akerr 3y ago
Generally, these are things that teachers learn during their training. Everyone has an education and has an opinion of education yet it doesn’t mean understanding how learning happens. In my opinion, good teachers teach the how of learning too.
KineticLensman 3y ago
The corollary to learning is teaching, and this isn't addressed substantially in the article. There is a whole industry associated with teaching / training that goes substantially beyond consideration of individual learning styles. E.g. how to structure a course, training objectives and key learning points, how to do assessment of students (during the training itself, or as summary exams / tests), how to give feedback, and many other points. There is also a load of theory around competence retention (how different types of knowledge and skill fall off over time) and the limits of learnability (e.g. the point at which you should stop trying to cram in something that is very forgettable and supplement / replace training with job aides such as checklists). Awareness of these things can make a big difference between well structured training material and a stream-of-consciousness YouTube 'tutorial'.
- wolverine876 3y ago
  Thanks. What are the authoritative resources?
  - KineticLensman 3y ago
    For a reasonable overview, take a look at https://en.wikipedia.org/wiki/Instructional_design https://en.wikipedia.org/wiki/Instructional_design For an example of a training system design approach that has been taken to an insane level of detail and organisational complexity, take a look at the UK MOD's version, the Defence Systems Approach to Training. https://assets.publishing.service.gov.uk/media/652fac86927459000df9590b/JSP_822_Vol_2_-_Individual_Training_V6_Oct_2023.pdf https://assets.publishing.service.gov.uk/media/652fac8692745... For competence retention, have a look at http://icicte.org/ICICTE2015Proceedings(Papers)/7.2%20Cahillane_MacLean&Smy.pdf http://icicte.org/ICICTE2015Proceedings(Papers)/7.2%20Cahill...
    - wolverine876 3y ago
      Thanks! Have a wonderful holiday!
  - cjmcqueen 3y ago
keiferski 3y ago
Another curious feature of human memory is "spreading activation."1 Our memories are stored in interconnected neural pathways. When we try to remember something, we activate a pathway of neurons to access the targeted information. However, activation is not contained within one pathway. Some of the activation energy spreads to other connected pathways, like heat radiating from a hot water pipe. This spreading activation leaves related pathways primed for activation for hours.1 He frames this as negative in the next paragraph, but this sounds like the mechanism by which memory palaces work. https://en.wikipedia.org/wiki/Method_of_loci https://en.wikipedia.org/wiki/Method_of_loci
nullandvoid 3y ago
On the topic of how our "Activation" pathways (used when we recall information) stay primed for hours after learning. This had me wondering on the power of building a "warm up" exercise when we're trying to solve a problem - can we brute force an optimal activation hot paths for better problem solving (obviously it would be highly individual - but presumably such a thing exists given this fact). It seems you would need a routine per category of problem, but none the less there may be more value than we think in spending 5 minutes just asking/answering some probing questions around the domain in question, before trying to solve the problem.
praveen9920 3y ago
Now I can relate why doing few leet code problems just before interview helps me do better. My activation pathways stay active for few hours . Hope to use this in various other places to improve
hasoleju 3y ago
Me and my wife often discuss on how we should help our children to learn. Especially if we see them making errors. Should we tell them that they made an error and show them the correct solution or should we wait to let them notice their error on their own. The last part about the mindset of the learner gave me an interesting perspective. The article explains the growth mindset and fixed mindset. The article suggest to nurture a growth mindset by rewarding successes and tolerating failures. Pointing out failures too often might make the learner switch to a fixed mindset.
- Jensson 3y ago
  > Pointing out failures too often might make the learner switch to a fixed mindset. Why? If they see they can learn to fix those errors doesn't it lead to growth mindset? Growth comes from learning that you can improve, and improvement comes from understanding that you aren't perfect and make errors and then learning to avoid those errors. People who got coddled and think they don't make errors don't have a growth mindset, they just think they are perfect as they are and there is nothing to improve. That is as close to fixed mindset you can get. You probably mean that pointing out errors is negative feedback and might make the kid feel bad. But it has little to do with growth or fixed mindset.
  - hasoleju 3y ago
    You are right. The key point is to only point out the error and let the learner come up with the correct solution by himself. This way they have positive feedback. The article mentions that learners can switch between growth and fixed mindset if they get frustrated while learning. >In reality, as we face setbacks and experience failure, people skew toward a fixed mindset because we are not sure where the boundaries of our abilities lie.
acbart 3y ago
All three of these authors are amazing Computer Science Education researchers. Obviously, it's a well-cited and well-argued article as well, but I feel like it's worth pointing out that I expected nothing less from them. I actually learned a few things here - I had never heard of the Semantic Wave before. What a great share, thank you very much!
2devnull 3y ago
“If you want to judge programming ability, assess programming ability.” Well there is a problem with that but I will leave it to others to work out what the problem is. (Hint: How do you measure programming ability without, you know some sort of measurement. :)
- Jensson 3y ago
  You test them on programming problems, not "how many piano tuners are there in New York" style questions. Implementing algorithms is still programming so its better than the brain teasers they used before those.
  - 2devnull 3y ago
    By programming problems do you mean leetcode? Algorithm questions? Should they know sql, or assembly, or maybe JavaScript? Saying they should “test programming” is not saying anything at all. The people who wanted to ask brain teasers wanted them exactly because they thought they were a good measure of programming ability. That someone may think leetcode is what counts as programming questions is just a statement of their opinion about what programming is.
belter 3y ago
From the article... "Experts are not always the best at training beginners." "To emphasize a specific point: Do not test candidates with brain-teaser puzzles." "...to get candidates to solve interview problems in a room on their own before presenting the solution, as the added pressure from an interviewer observing or requiring talking while solving it adds to cognitive load and stress in a way that impairs performance..." https://youtu.be/bn_KRzohcAo https://youtu.be/bn_KRzohcAo

507 points by jarmitage 3y ago

500 error

hug of death: https://web.archive.org/web/20231227005023/https://cacm.acm.org/magazines/2024/1/278891-10-things-software-developers-should-learn-about-learning/fulltext https://web.archive.org/web/20231227005023/https://cacm.acm....

emmanueloga_ 3y ago
It seems they have clouflare support, but must be misconfigured somehow? Also rails? 1.4 secs seems like a long response time, even for a rails app. I guess even the pros get websites wrong sometimes :-p Cache-Control: private, max-age=0, must-revalidate Cf-Cache-Status: DYNAMIC Cf-Ray: 83be9f774919fa8a-SJC Content-Encoding: gzip Content-Type: text/html; charset=utf-8 Date: Wed, 27 Dec 2023 03:54:35 GMT Server: cloudflare Status: 200 OK Vary: Accept-Encoding X-Powered-By: Phusion Passenger X-Runtime: 1.42775

readthenotes1 3y ago

Iirc, System 1/2 thinking has been largely de-based (i.e., it could be a right model, but the evidence for that is tenuous) https://www.psychologytoday.com/us/blog/a-hovercraft-full-of-eels/202103/the-false-dilemma-system-1-vs-system-2 https://www.psychologytoday.com/us/blog/a-hovercraft-full-of... Makes me wonder if item (0) should be "Suspect every one" (with a nod to Maria Gambrelli, of course)

jarmitage 3y ago
Article footnote b. states: "b. Parts of Kahneman's book were undermined by psychology's "replication crisis," which affected some of its findings, but not the idea of system 1 and 2." Would have been a better footnote if additional references were provided for the latest in that area of discourse.

darkclouds 3y ago

[dead]

mewpmewp2 3y ago

The article looks very accurate to me. Having read all of that, I find myself agreeing with most of it, which I think usually would not be the case for me with an article like this.

thebenedict 3y ago
I was also pleasantly surprised by the quality of this article, especially the nuance around the value of a growth mindset. It's consistent with my experience of having had to work pretty hard to learn programming.
- mewpmewp2 3y ago
  It seems like a clickbait title combined with nuanced points of high quality and relevancy.
- mobiuscog 3y ago
  > the value of a growth mindset This was buzzword bingo 2023. Hopefully it dies in 2024.
deleted 3y ago
[deleted]

bedobi 3y ago

allow me to shamelessly plug something I wrote on the topic https://gist.github.com/androidfred/75629dfda63180b6f0a0eaa4655ec5f0 https://gist.github.com/androidfred/75629dfda63180b6f0a0eaa4... no data, no research :P just anecdote and opinon

eszed 3y ago
I really like what you've written. It jibes with my own experience and opinions. If I were to expand it a bit further, I'd point out that there are teaching methodologies that seem to be derived from these paradigms. In one, an apprentice copies the master as well as they are able, sometimes with no explicit instruction at all, until they've filled in the details for themselves; in the other the student memorizes facts and methods, often by rote, until they build up the big picture for themselves. I've learned within (and from) both approaches, and find each - when rigidly followed - to be highly frustrating! In my own pedagogy I try to hold both in mind, and calibrate students' learning paths accordingly. When they're mired in detail, I re-orient them towards the end goal; when they're not sure what to do, or how to do it, I guide them through the next step. It's a lot more effort, because you have to pay attention to them, and not only the subject, which most teachers would prefer not (or don't know how) to do. (On a side note, I'll say that - in the field(s?) where I'm an expert - nearly all of the pleasure comes from refining the last 2% of the details. It's never going to be perfect, but it can always be incrementally better. It's not "productive", certainly in a commercial sense, but it's immensely satisfying.)

alphazard 3y ago

In section 7: > Problem Solving is Not a Generic Skill There is going to be some difference between solving problems in a specific domain and solving problems generally (which is what TFA argues for). And since we really care about the specific domain of software engineering, it makes sense to pry open that difference when possible. However problem solving in the general case is very close to fluid intelligence and IQ. Some interpretations claim that intelligence in humans is just problem solving, and that problem solving is most of what is captured by g [0]. All problem solving will be positively correlated with all other problem solving, and you would never expect to see someone good at one, but not the other. In section 9 they cite the research on programming ability and its (expected) relation to general intelligence. I'm not sure how much of a distinction there is to draw here. Psychometrics has historically been filled with attempts to factor out additional clusters from things like g e.g. multiple intelligences. Those findings often fail to replicate. Section 7 seems more like an attempt to draw a distinction without a difference. While section 9 seems like a standard summary of the research (like most things, a mixture of innate intelligence and cumulative experience). [0] https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)

Kinrany 3y ago
Their point is that you can't learn general problem solving. IQ is not a skill.
- ozim 3y ago
  I would say you can learn general problem solving. There are strategies that help with that an IQ helps with finding patterns. That said it still would be junior level problem solving in unknown domain. Expert will always run circles around newcomers. I most likely could switch fields but I don’t want to spend 5 years getting experience. Even if I have decent generic solving skills it might take me less time but still- I would have to be really interested in the topic.
  - Jensson 3y ago
    > Expert will always run circles around newcomers. That isn't true, there are many problems less than mediocre experts fails to solve that a smart junior can manage to solve. Experts are great at common problems but not great at rarer ones, non-standard problems depend more on natural ability than experience. Of course smart experts can also solve those problems, but you didn't say a smart expert, you just said experts in general. The ability to generalize based on experience requires intelligence. That is why you have so many expert programmers who can't solve problems well, because they lack the ability to generalize their experience well enough to apply it to new problems.

quickthrower2 3y ago

> Long-term memory is where information is permanently stored and is functionally limitless; in that sense, it functions somewhat like a computer's disk storage. Doesn't jibe with my experience.

jpoesen 3y ago
Having trouble retrieving the knowledge does not mean the knowledge is no longer there.

revskill 3y ago

To me, intelligence in general is ability of solving things in a generic and abstract way.

deleted 3y ago
[deleted]

wisemang 3y ago

I appreciate the succinct detail they go into regarding how the concept of “learning styles” has been debunked. I’ve always considered myself more of a visual learner, and seeing posts here and elsewhere about this supposedly not being a thing has never sat well with me (though I never delved into the actual research). Turns out I do in fact agree with this explanation; that it’s more about what’s being taught that should dictate the how: > While learners have preferred styles, effective instruction matches the content, not learning styles. A science class should use graphs to present data rather than verbal descriptions, regardless of visual or auditory learning styles, just like cooking classes should use hands-on practices rather than reading, whether learners prefer a kinesthetic style or not.

oatmeal1 3y ago
Courses teaching music theory through visual depictions, take note!
- NetOpWibby 3y ago
  This is why I failed music class in high-school. I was the only non-band student so the work was accelerated.
  - jacobolus 3y ago
    This is why trying to rank/judge/grade everyone by a uniform standard is almost universally terrible. Students should be encouraged to try their best in every subject, allowed to make the mistakes they are naturally going to make at whatever level they are currently, and helped to improve over time. Punishing people for being less prepared than peers who did more practice or for making ordinary and expected mistakes actively gets in the way of their learning, as well as making them feel terrible. It's pretty bad for the students who are more prepared as well, as many of them internalize the idea that they are inherently good at some things and inherently bad at others, which is sometimes temporarily gratifying but often stops them from pushing themselves to try anything new or hard.

tbwriting 3y ago

This is a fantastic piece. If you find yourself thinking “okay, so the brain is different — now what? What should I actually do to learn better?” I wrote about this last year. Pardon the clickbaity title, HackerNoon changed it up on me. https://hackernoon.com/the-four-rs-how-to-become-a-good-programmer https://hackernoon.com/the-four-rs-how-to-become-a-good-prog...

james-revisoai 3y ago
I'd be interested in discussing this with you - we think alike, although, having background in psychology too, I was at first sceptical of your use of terms like "Rearranging". But I like your summary and how you orient it to the programming domain but also reference examples of good learning app techniques. I am working on an app that makes many decisions in this area, and is truly trying to attempt the psychology of learning ethically to university studying (including parts of computer science you can learn without actually "doing" the coding etc). I have a few ideas and new learning user flows I would love to get your feedback on as well as have a wider discussion and nerd out a bit on psychology studies I think you'll find revealing, would you be up to talk? As a quick peak: My own thesis on student learning broadly begins with three e's: "Effectiveness, Enjoyment(or motivation), and Environment", as I believe the first two are necessary, and the third one an augmentation, as properties of effective learning systems for people. This is based on combining my psychology knowledge (mostly the first and last two E's) with my experience volunteering and being a TA and hearing from students who through 90% of the semester struggle with the middle e - Enjoyment - more than others, or to the point where it prevents applying the others(not motivated to use effective techniques, not motivated to go to the library when procrastinating, etc) correctly. I am super interested in combining this with the teachers view and how they work too (e.g. curriculum design, personalised tutoring)

rTX5CMRXIfFG 3y ago

> a fact does not exist in a binary state of either definitively known or unknown; it can exist in intermediate states. We can forget things we previously knew, and knowledge can be unreliable, especially when recently learned. I’m sorry, what? If I only vaguely understood physics and believed now that the earth is flat, that would neither count as knowledge in some intermediate state. Knowing is binary—you either know or you don’t, no matter how strongly believe what you think.

Cerium 3y ago
An example: What did you eat for dinner? If you know the answer for sure, go back a day, or another day, etc. At some number of days you will have an idea of what you ate but not be entirely sure. You certainly knew what you ate when you were eating it, but now?
- rTX5CMRXIfFG 3y ago
  If I don’t know that now but knew it then, then knowledge is still binary is it not?
  - RoyalHenOil 3y ago
    You don't have perfect recollection one day and then no recollection whatsoever the next day. Memories become vague as they deteriorate. For example, you may remember something you learned in 4th grade, but you probably don't remember exactly how the teacher explained it or what questions about it were on the test.
    - master-lincoln 3y ago
      Ok, but then I don't know how I was taught that thing, even though I know the thing. Still binary
    - deleted 3y ago
      [deleted]
    - 3y ago

zubairq 3y ago

Some good points, programming t00 much can sometimes distort my thinking !

TibbityFlanders 3y ago

https://today.ucsd.edu/story/a-new-replication-crisis-research-that-is-less-likely-be-true-is-cited-more#:~:text=In%20psychology%2C%20only%2039%20percent%20of%20the%20100%20experiments%20successfully%20replicated https://today.ucsd.edu/story/a-new-replication-crisis-resear.... > Papers that cannot be replicated are cited 153 times more because their findings are interesting, according to a new UC San Diego study > In psychology, only 39 percent of the 100 experiments successfully replicated.

rmbyrro 3y ago
Was this study replicated?
- deleted 3y ago
  [deleted]

vinay_ys 3y ago

> Experts Recognize, Beginners Reason > System 1 is fast and driven by recognition, relying upon pattern recognition in long-term memory, while system 2 is slower and focused on reasoning, requiring more processing in working memory. Interestingly, today, LLMs are augmentation for someone's weak system 1, and allowing them to focus solely on strengthening their system 2. LLMs and popular/cheap/generalizable AI today suck at system 2. So, if you are really good at system 2 and suck at system 1, the next decade is going to be amazing for you.

deleted 3y ago
[deleted]
TibbityFlanders 3y ago
(removed)
- Always42 3y ago
  can you post enough about your local LLM setup for me to google/rep. this? There is so much out there for LLM's parsing is a pain.
  - TibbityFlanders 3y ago
    For audio generation I recommend Bark. I am getting 14 seconds of audio that is about a third of eleven labs quality in about 2 minutes. This is happening on a Windows 10 Dell, with 32gb of RAM, an i5, and an Nvidia 1050 GeForce with 4gb of vram. I'm also able to decently run local LLMs because of llama.cpp and other libraries that can share models been ram and vram. There are other tools that can help with this as well including Ollama. I suggest subscribing to r/localLLAMA. I also suggest using Bing Copilot in Edge with allowed access to the page you're viewing. I often use it to find new GitHub libraries and to give me first steps to be able to start using a new framework.
rramadass 3y ago
Also see Recognition-Primed Decision Model: https://en.wikipedia.org/wiki/Recognition-primed_decision https://en.wikipedia.org/wiki/Recognition-primed_decision

abaymado 3y ago

> Research into chess found little or no effect of learning it on other academic and cognitive skills, and the same is true for music instruction and cognitive training.This inability to transfer problem-solving skills is why "brain training" is ineffective for developing general intelligence. I would disagree with this premise, deep work and the forbidden word "discipline" are problem solving skills that are learned and need constant training. They are just as important as any other specific skill needed for the subject. Thus, making some problem-solving skills indeed free flowing from subject to subject.

watwut 3y ago
I guess that then implication is that learning chess ot music does not make you more disciplined either.
- sitkack 3y ago
  Learning any subject in any domain, deeply has value, and that learning is transferable. I think when the analysis is done, saying that chess doesn’t make one smarter per se, the researchers are starting from a baseline where student a and student B are equal in all regards, and in addition to student B has learned to chess. While student a might not know chess they could have spent an equal amount of time learning something else and it’s that time learning things that’s important not necessarily the subject matter.

rmrf100 3y ago

sometimes it's really astonishing to see that, children grow up day by day then quikly understond the logic which hard to them before.

xjay 3y ago

Warning from Daniel Kahneman on "System 1" and "System 2". [1] > I'm going to use "System 1" and "System 2", absolutely as homunculi. [...] They don't exist. [...] Don't look for them in the brain, because they are not two systems in the brain, of which one does one, and the other does the other. So why am I using this terrible language? I'm using it because I think it's helpful. It fits the way our minds work, and to explain the background of that decision--of why I use "System 1" and "System 2"--I refer you to a very good book. [...] It's by Joshua Foer and it's called "Moonwalking with Einstein". [2] [1] https://www.youtube.com/watch?v=CjVQJdIrDJ0&t=1224s https://www.youtube.com/watch?v=CjVQJdIrDJ0&t=1224s [2] https://en.wikipedia.org/wiki/Moonwalking_with_Einstein https://en.wikipedia.org/wiki/Moonwalking_with_Einstein The article does put these terms in quotes.

BeetleB 3y ago
His book Thinking, Fast and Slow makes it clear that these are conveniences and not real systems in the brain.

aksss 3y ago

> “Higher capacity enables faster learning, but our unlimited long-term memory removes limitations on how much we could ultimately learn in total.1 Expert programmers may have low or high working memory capacity but it is the contents of their long-term memory that make them experts.” I’ve always told “kids” that you can learn a lot about systems but with programming and IT systems in general, there is just no substitute for getting the raw mileage of having seen many permutations, iterations, and manifestations. It’s not a dig, but a statement made in the context of encouraging new people to stick with it and not beat themselves up too much when they inevitably get overwhelmed by the scope of their unknowns or roll a critical miss. It’s all about learning, all the time.

auggierose 3y ago

I like the paper, and I think it makes many valuable points. I especially like the semantic wave model, because that's exactly how you learn mathematics. And it also explains why (high-quality) abstractions are so important. But of course everything has to be taken with a grain of salt. For example, their recommendations at the end on how to access papers is not very good. Ever heard of Sci-Hub and VPNs? It is obvious why they cannot mention this in their paper, but it is also equally obvious then that if there was evidence linking race or gender with programming ability, they would not mention it, for pretty much the same reasons. I also don't like their example of achieving a Nobel Prize as something that practically no one can attain. Yes, that's true, but that is because Nobel Prizes are artificially limited to a few people a year. I think many, many more people can achieve that level of expertise than just a few per year.

tsimionescu 3y ago
It should be noted that the very idea of studying links between "race" (skin color) and intelligence is deeply suspect. No one is studying things like correlation between hair wavyness and intelligence, or penis length and intelligence, etc. And yet skin color is exactly the same kind of trait: something you were born with and that is tied to the genetic baggage of your parents, with no remotely plausible direct link with intelligence (unlike, say, skull size, which at least had some plausible priors). You can even (somewhat) change the color of your skin if you really want. Now, there may be, at least in principle, families, and by extension populations, who are more or less intelligent, on average, than others. But "race" has little to do with that: these types of studies only look at skin color as a proxy for population, and that is obviously silly on genetic grounds. Dark skin is a dominant trait, the children of a lighter skinned parent and a darker skin parent will usually have darker skin and be assigned the racial category "black" in such research, even though genetically they are just as much a member of the lighter skinned population as the darker skinned one. Even worse, this often persists across a few generations, so a child with 1 dark skinned grand parent and 3 light skinned ones will often be dark skinned themselves, and thus be called "black" in many such studies. So, the reason you should be very very much concerned with citing studies that find links between "race" and intelligence is that the very premise is wrong in the vast majority of the literature.

rand1239 3y ago

I wonder whether including this article in GPT-5 training data would improve the performance of the model.

Jensson 3y ago
Why would it? It just learns by making this kind of text more likely to appear. If you want it to be smart it needs to see problems and strategies to solve them, not see explanations. Regurgitating explanations isn't useful, following problem solving patterns is. So to learn from this article it would need to see the kind of thinking required to write the article.
- rand1239 3y ago
  Okay. Are humans different than this? Thoughts -> Action -> Thoughts Thoughts are generated by brain. You don't control how the neurons are fired in the brain. As far as you are concerened you experience the thoughts, the sounds, images etc which. So what exactly is the point of this article?
  - Jensson 3y ago
    When you read something you typically imagine a lot of examples that the thing you read could apply to. The way we train LLMs today doesn't do this, it just reads the text without thinking further about it. That means to make an LLM learn you have to feed it examples instead of descriptions, like examples of problems being solved etc. Maybe we could make LLM training do such things in the future, but it doesn't do it today and it is hard to do that in practice since generating examples on the fly for descriptions isn't easy to do in an intelligent way. I think that is a core part of generalizing knowledge so probably one of the keys we need to get to AGI.

remram 3y ago

Why are "key insights" presented as an ugly GIF? Just so it can't be seen by screen readers? Or maybe robots? If you really want to put your text in an image, can you pick a decent font and not make it blurry? Puzzling.

auggierose 3y ago
Read the PDF instead. I always do, cannot be bothered to read ACMs low quality HTML.

akerr 3y ago

Generally, these are things that teachers learn during their training. Everyone has an education and has an opinion of education yet it doesn’t mean understanding how learning happens. In my opinion, good teachers teach the how of learning too.

KineticLensman 3y ago

The corollary to learning is teaching, and this isn't addressed substantially in the article. There is a whole industry associated with teaching / training that goes substantially beyond consideration of individual learning styles. E.g. how to structure a course, training objectives and key learning points, how to do assessment of students (during the training itself, or as summary exams / tests), how to give feedback, and many other points. There is also a load of theory around competence retention (how different types of knowledge and skill fall off over time) and the limits of learnability (e.g. the point at which you should stop trying to cram in something that is very forgettable and supplement / replace training with job aides such as checklists). Awareness of these things can make a big difference between well structured training material and a stream-of-consciousness YouTube 'tutorial'.

wolverine876 3y ago
Thanks. What are the authoritative resources?
- KineticLensman 3y ago
  For a reasonable overview, take a look at https://en.wikipedia.org/wiki/Instructional_design https://en.wikipedia.org/wiki/Instructional_design For an example of a training system design approach that has been taken to an insane level of detail and organisational complexity, take a look at the UK MOD's version, the Defence Systems Approach to Training. https://assets.publishing.service.gov.uk/media/652fac86927459000df9590b/JSP_822_Vol_2_-_Individual_Training_V6_Oct_2023.pdf https://assets.publishing.service.gov.uk/media/652fac8692745... For competence retention, have a look at http://icicte.org/ICICTE2015Proceedings(Papers)/7.2%20Cahillane_MacLean&Smy.pdf http://icicte.org/ICICTE2015Proceedings(Papers)/7.2%20Cahill...
  - wolverine876 3y ago
    Thanks! Have a wonderful holiday!
- cjmcqueen 3y ago

keiferski 3y ago

Another curious feature of human memory is "spreading activation."1 Our memories are stored in interconnected neural pathways. When we try to remember something, we activate a pathway of neurons to access the targeted information. However, activation is not contained within one pathway. Some of the activation energy spreads to other connected pathways, like heat radiating from a hot water pipe. This spreading activation leaves related pathways primed for activation for hours.1 He frames this as negative in the next paragraph, but this sounds like the mechanism by which memory palaces work. https://en.wikipedia.org/wiki/Method_of_loci https://en.wikipedia.org/wiki/Method_of_loci

nullandvoid 3y ago

On the topic of how our "Activation" pathways (used when we recall information) stay primed for hours after learning. This had me wondering on the power of building a "warm up" exercise when we're trying to solve a problem - can we brute force an optimal activation hot paths for better problem solving (obviously it would be highly individual - but presumably such a thing exists given this fact). It seems you would need a routine per category of problem, but none the less there may be more value than we think in spending 5 minutes just asking/answering some probing questions around the domain in question, before trying to solve the problem.

praveen9920 3y ago

Now I can relate why doing few leet code problems just before interview helps me do better. My activation pathways stay active for few hours . Hope to use this in various other places to improve

hasoleju 3y ago

Me and my wife often discuss on how we should help our children to learn. Especially if we see them making errors. Should we tell them that they made an error and show them the correct solution or should we wait to let them notice their error on their own. The last part about the mindset of the learner gave me an interesting perspective. The article explains the growth mindset and fixed mindset. The article suggest to nurture a growth mindset by rewarding successes and tolerating failures. Pointing out failures too often might make the learner switch to a fixed mindset.

Jensson 3y ago
> Pointing out failures too often might make the learner switch to a fixed mindset. Why? If they see they can learn to fix those errors doesn't it lead to growth mindset? Growth comes from learning that you can improve, and improvement comes from understanding that you aren't perfect and make errors and then learning to avoid those errors. People who got coddled and think they don't make errors don't have a growth mindset, they just think they are perfect as they are and there is nothing to improve. That is as close to fixed mindset you can get. You probably mean that pointing out errors is negative feedback and might make the kid feel bad. But it has little to do with growth or fixed mindset.
- hasoleju 3y ago
  You are right. The key point is to only point out the error and let the learner come up with the correct solution by himself. This way they have positive feedback. The article mentions that learners can switch between growth and fixed mindset if they get frustrated while learning. >In reality, as we face setbacks and experience failure, people skew toward a fixed mindset because we are not sure where the boundaries of our abilities lie.

acbart 3y ago

All three of these authors are amazing Computer Science Education researchers. Obviously, it's a well-cited and well-argued article as well, but I feel like it's worth pointing out that I expected nothing less from them. I actually learned a few things here - I had never heard of the Semantic Wave before. What a great share, thank you very much!

2devnull 3y ago

“If you want to judge programming ability, assess programming ability.” Well there is a problem with that but I will leave it to others to work out what the problem is. (Hint: How do you measure programming ability without, you know some sort of measurement. :)

Jensson 3y ago
You test them on programming problems, not "how many piano tuners are there in New York" style questions. Implementing algorithms is still programming so its better than the brain teasers they used before those.
- 2devnull 3y ago
  By programming problems do you mean leetcode? Algorithm questions? Should they know sql, or assembly, or maybe JavaScript? Saying they should “test programming” is not saying anything at all. The people who wanted to ask brain teasers wanted them exactly because they thought they were a good measure of programming ability. That someone may think leetcode is what counts as programming questions is just a statement of their opinion about what programming is.

belter 3y ago

From the article... "Experts are not always the best at training beginners." "To emphasize a specific point: Do not test candidates with brain-teaser puzzles." "...to get candidates to solve interview problems in a room on their own before presenting the solution, as the added pressure from an interviewer observing or requiring talking while solving it adds to cognitive load and stress in a way that impairs performance..." https://youtu.be/bn_KRzohcAo https://youtu.be/bn_KRzohcAo