4 ms·

I wrote this and would love to know what you think about it. Am particularly interested in hearing any reasons why it's wrong

going_ham 4y ago
Thank you for writing it. I have been feeling something similar. The ability to truly understand requires the ability to know few terms and what they mean. I thought it was only me who felt it like so. Most of my friends who venture on creative tasks are able to recall a lot of things because of practice and without having to be distracted! But does it effect the field like programming? When programming, if I can remember the context, then I can easily search it (research paper, books, documentation, forums etc) Now, with Co-Pilot, isn't it effectively beneficial to understand a topic and develop a general problem solving framework for ourselves so that we can let the AI do it's thing? What are your opinions regarding it?
- p-christ 4y ago
  Even with programming I would argue you'd be a much better programmer if you can remember more. Obviously sometimes you're going to have to look things up but the more you remember the more problems you'll be able to solve. Specifically you won't be able to solve a programming problem if the answer requires you combining over 4 things you don't have in your long-term memory (even if you can look them up). This is the main reason why Jeff Dean is a better programmer than me even though we both have access to google - he has more knowledge & experience of programming in his memory than me that means that even though we can both look things up he is able to solve way more problems than me. Co-pilot slightly changes the type of thing that's valuable to remember, but it doesn't change the importance of remembering things. I think, as you implied, co-pilot probably makes remembering some types of syntax or boilerplate less important.
  - going_ham 4y ago
    Ohh, that's a cool opinion!!! Thank you ^_^
vladsanchez 4y ago
Slash-Dotted!? Site doesn't load. (shrug)
- p-christ 4y ago
  how you mean? this link doesn't load for you? https://saveall.ai/blog/learning-is-remembering https://saveall.ai/blog/learning-is-remembering
  - vladsanchez 4y ago
    Just loaded, after 20 attempts. ;)
    - p-christ 4y ago
      hmmm very interesting, not sure why that could be happening! I can see loads of people are reading it now but i would have thought our server (arranged on vercel) could handle it
kardianos 4y ago
Learning is memory, in a sense. More broadly learning the the physical structure that process leaves on your brain. When you truly learn, you do less remembering and more simulating. The process of simulating actions (multi-step addition algorithms in elementary school or wrote memorization of math-facts) is the process that changes the structure of your mind that isn't dependent on "memorization".
- p-christ 4y ago
  that's really interesting, can you explain this bit a bit more? Or let me know what to google to learn more about this? "The process of simulating actions is the process that changes the structure of your mind"
  - kardianos 4y ago
    I would point you to two different motivating examples: * Childhood development. A child who has had early childhood adverse experiences mind is effected. A fMRI or chemical detection can see physical differences in the brain. But the brain is plastic (it can change) over time. Works on this subject have been published by Dr. Dan Siegel https://drdansiegel.com/books https://drdansiegel.com/books among others. The key insight here is that academic learning is not significantly different at a core aspect then behavioral learning. * Physical sports/martial arts depend on a reaction time much smaller then what is afforded by going through the full frontal cortex. "muscle-memory" isn't real (it isn't "memory" as you think of it). What you have in these cases are "short-circuits" (this implies structural changes) that are able to act before you are consciously aware of what is going on. The same applies to math facts and other fundamentals, you move things away from memory that needs to be retrieved and into reaction. Reading C-syntax for programmers or signing your name is something that has been turned into structure that doesn't need "memory". I think your initial premise is correct. We have limited memory. How do we overcome that? We write. Writing is important because with it we can overcome our natural memory limitations. You cannot think about complexities (well or clearly) if you cannot write. The danger of writing is that you can produce something that is both irrational and nearly impenetrable to the casual reader. For example: > Foucault's use of the concept is descriptive, that is, analytical and explanatory, and at the same time normative and critical: he describes the grip biopolitics have on individuals through technologies of power in a way that makes manifest the repression at work in these biopolitical processes. The above, taken directly from an "academic" published journal, could be said to have meaning. Unfortunately, each one of these words has an alternate meaning that is not normative to English, making the entire (actual) meaning opaque. "contecpt" "analytical", "explanatory", "normative", "critical", "biopolitics", "individuals", "technologies", "power", "manifest", "repression", "processes" are all defined differently then a standard English dictionary. So even if you can get past the convoluted sentence structure, the intended meaning will still elude you. But the answer to memory limitations is clear writing using common definitions of words. I bring this up because as you extend your memory beyond what it innately has, the more likely you are to fool yourself (and others) with sophistry.
synergy20 4y ago
assuming our brain is a CPU, the 4-working-queue-short-memory is basically our L1-cache. We need L2 and hard drive for longer term memory, otherwise the system can not really function well. however long-term storage is just one vital factor, another one is the 'deep learning' neurons that understand the content it stores and more important to connect the dots among various neurons. we need both: understand and store. Neurons do both for us.
- quarok 4y ago
  yes, we need understanding - the best way to get things into memory is to make connections between neurons through understanding. And this keeps it in memory for longer than if you just memorise it — and it will enable you to deploy the information in new contexts. My takeaway from this article is that if you only focus on understanding (and so do not commit it to memory), you cannot reason using this information in unfamiliar contexts later on, once you've forgotten it. So the best thing to do is to: 1. Make sure you understand something thoroughly 2. Test yourself on it using spaced repetition to ensure you keep it forever For transparency: I'm one of the cofounders of Save All (linked site) alongside Petros the author
anothernewdude 4y ago
There's probably something to be said about batch sizes and stability.
sennraf 4y ago
Creating a tool to help others learn is awesome. I think what the article misses is that there is a ton of knowledge we can't 'write down' and therefore memorization is not enough. For example learning to solve Integrals. Yes, there are some rules and tricks one can memorize which helps but I would argue the only way to get good at solving integrals is to interact, resp. solve them. Another point are second order effects of how one learns, for example curiosity and resilience. There might be a long time negative effect on motivation of a topic, when there is too much focus on 'memorization' (It certainly was that for me in my French class;)). I really enjoyed the 'bad reputation' part and agree that it is sadly viewed as not important enough by many.
- Qualadore 4y ago
  The spacing effect can also be applied to solving problems, like "distributed practice" as opposed to "massed practice" with calculus homework [0]. Speaking completely personally, forgetting a lot of information is the easiest way to ruin my motivation to learn a topic. [0] https://link.springer.com/article/10.1007/s10648-022-09677-2 https://link.springer.com/article/10.1007/s10648-022-09677-2
GistNoesis 4y ago
There are plenty of different way to learn, and the optimal probably depend on the material to learn. For example, in machine learning, there is something called stochastic gradient descent, where to learn you present a single random element at a time from the dataset. In the end it will have learned of all concepts, by becoming more and more confident in each individual concepts. For example to learn QM, you pick a random QM wikipedia article, and try to push through the article, even though there are some things you don't understand. Then you do the same thing, for a different unrelated QM article. For learning tennis, you don't learn specifically forehand, then learning backhand, but you alternate them at random so that you have a single unified way of playing with smooth transitions, instead of having to switch between different "modes" of thinking. Sure more memory can allow some speed-space trade-off in learning ability, but using your memory too much may make you miss some fluency that may have emerged. For example the old-school of machine learning was using databases and K-near neighbors, which used a lot of memory and was slow. But the new-school of machine learning are using constant memory algorithm and compressing the data in it, and it can learn to generate all the pictures in the world with only 4 Gb of weights. Learning is imagining, once you bootstrap your imagination, its bandwidth to synthesize new examples from which you can learn from, is much greater than the bandwidth of looking up new data material to learn from.
carapace 4y ago
It's not bad, but you're limiting yourself "out the gate" so to speak. If you really want to explore memory and learning you must study and practice (self-)hypnosis. Most of what we think we know about how the brain and mind work is wrong. E.g. memory: "our working memory has a maximum capacity of roughly 4" is not an absolute rule. Your brain, right now as you read this, is tracking 100,000's of variables and recording a trace of everything you are seeing, hearing, smelling, tasting, touching, and doing. Almost all of this is done unconsciously, and for good reason. Anyhow, you can learn to have better "interfaces" to your automatic unconscious abilities and leverage them to e.g. remember instantly and durably any mathematical equation, etc. - - - - As an aside, the strategy you sketched out at the start of your article would be workable if you have great recall, however an even more better strategy would be to recapitulate the discoveries of physics in roughly historical order. Your understanding of quantum mechanics would be much deeper and richer, and you would be following a story (an epic story made up of so many fascinating smaller stories, and one that is still going on! Albeit things have calmed down this last century or so, but no one thinks we have reached the climax yet.) That would be the way to do it: start with the Greeks and the Alchemists and proceed to follow the trail(s) of how we as a species sussed out the mysteries of the physical Universe.
aappleby 4y ago
Copy-pasting my top-level comment: I am utterly baffled that no response in this thread so far has taken issue with the statement "As you probably know intuitively, it won't work." For me, this _does_ work and I have proven it many many times over the years by adding entire categories of technical knowledge to my repertoire. And not superficially, either - I get paid very well to do things professionally that I taught myself by reading Wikipedia. If my experience were commonplace, the "it won't work" statement would be highly contentious in the comments here. Since it isn't, I guess I can deduce that I must be an outlier.

going_ham 4y ago

Thank you for writing it. I have been feeling something similar. The ability to truly understand requires the ability to know few terms and what they mean. I thought it was only me who felt it like so. Most of my friends who venture on creative tasks are able to recall a lot of things because of practice and without having to be distracted! But does it effect the field like programming? When programming, if I can remember the context, then I can easily search it (research paper, books, documentation, forums etc) Now, with Co-Pilot, isn't it effectively beneficial to understand a topic and develop a general problem solving framework for ourselves so that we can let the AI do it's thing? What are your opinions regarding it?

p-christ 4y ago
Even with programming I would argue you'd be a much better programmer if you can remember more. Obviously sometimes you're going to have to look things up but the more you remember the more problems you'll be able to solve. Specifically you won't be able to solve a programming problem if the answer requires you combining over 4 things you don't have in your long-term memory (even if you can look them up). This is the main reason why Jeff Dean is a better programmer than me even though we both have access to google - he has more knowledge & experience of programming in his memory than me that means that even though we can both look things up he is able to solve way more problems than me. Co-pilot slightly changes the type of thing that's valuable to remember, but it doesn't change the importance of remembering things. I think, as you implied, co-pilot probably makes remembering some types of syntax or boilerplate less important.
- going_ham 4y ago
  Ohh, that's a cool opinion!!! Thank you ^_^

vladsanchez 4y ago

Slash-Dotted!? Site doesn't load. (shrug)

p-christ 4y ago
how you mean? this link doesn't load for you? https://saveall.ai/blog/learning-is-remembering https://saveall.ai/blog/learning-is-remembering
- vladsanchez 4y ago
  Just loaded, after 20 attempts. ;)
  - p-christ 4y ago
    hmmm very interesting, not sure why that could be happening! I can see loads of people are reading it now but i would have thought our server (arranged on vercel) could handle it

kardianos 4y ago

Learning is memory, in a sense. More broadly learning the the physical structure that process leaves on your brain. When you truly learn, you do less remembering and more simulating. The process of simulating actions (multi-step addition algorithms in elementary school or wrote memorization of math-facts) is the process that changes the structure of your mind that isn't dependent on "memorization".

p-christ 4y ago
that's really interesting, can you explain this bit a bit more? Or let me know what to google to learn more about this? "The process of simulating actions is the process that changes the structure of your mind"
- kardianos 4y ago
  I would point you to two different motivating examples: * Childhood development. A child who has had early childhood adverse experiences mind is effected. A fMRI or chemical detection can see physical differences in the brain. But the brain is plastic (it can change) over time. Works on this subject have been published by Dr. Dan Siegel https://drdansiegel.com/books https://drdansiegel.com/books among others. The key insight here is that academic learning is not significantly different at a core aspect then behavioral learning. * Physical sports/martial arts depend on a reaction time much smaller then what is afforded by going through the full frontal cortex. "muscle-memory" isn't real (it isn't "memory" as you think of it). What you have in these cases are "short-circuits" (this implies structural changes) that are able to act before you are consciously aware of what is going on. The same applies to math facts and other fundamentals, you move things away from memory that needs to be retrieved and into reaction. Reading C-syntax for programmers or signing your name is something that has been turned into structure that doesn't need "memory". I think your initial premise is correct. We have limited memory. How do we overcome that? We write. Writing is important because with it we can overcome our natural memory limitations. You cannot think about complexities (well or clearly) if you cannot write. The danger of writing is that you can produce something that is both irrational and nearly impenetrable to the casual reader. For example: > Foucault's use of the concept is descriptive, that is, analytical and explanatory, and at the same time normative and critical: he describes the grip biopolitics have on individuals through technologies of power in a way that makes manifest the repression at work in these biopolitical processes. The above, taken directly from an "academic" published journal, could be said to have meaning. Unfortunately, each one of these words has an alternate meaning that is not normative to English, making the entire (actual) meaning opaque. "contecpt" "analytical", "explanatory", "normative", "critical", "biopolitics", "individuals", "technologies", "power", "manifest", "repression", "processes" are all defined differently then a standard English dictionary. So even if you can get past the convoluted sentence structure, the intended meaning will still elude you. But the answer to memory limitations is clear writing using common definitions of words. I bring this up because as you extend your memory beyond what it innately has, the more likely you are to fool yourself (and others) with sophistry.

synergy20 4y ago

assuming our brain is a CPU, the 4-working-queue-short-memory is basically our L1-cache. We need L2 and hard drive for longer term memory, otherwise the system can not really function well. however long-term storage is just one vital factor, another one is the 'deep learning' neurons that understand the content it stores and more important to connect the dots among various neurons. we need both: understand and store. Neurons do both for us.

quarok 4y ago
yes, we need understanding - the best way to get things into memory is to make connections between neurons through understanding. And this keeps it in memory for longer than if you just memorise it — and it will enable you to deploy the information in new contexts. My takeaway from this article is that if you only focus on understanding (and so do not commit it to memory), you cannot reason using this information in unfamiliar contexts later on, once you've forgotten it. So the best thing to do is to: 1. Make sure you understand something thoroughly 2. Test yourself on it using spaced repetition to ensure you keep it forever For transparency: I'm one of the cofounders of Save All (linked site) alongside Petros the author

anothernewdude 4y ago

There's probably something to be said about batch sizes and stability.

sennraf 4y ago

Creating a tool to help others learn is awesome. I think what the article misses is that there is a ton of knowledge we can't 'write down' and therefore memorization is not enough. For example learning to solve Integrals. Yes, there are some rules and tricks one can memorize which helps but I would argue the only way to get good at solving integrals is to interact, resp. solve them. Another point are second order effects of how one learns, for example curiosity and resilience. There might be a long time negative effect on motivation of a topic, when there is too much focus on 'memorization' (It certainly was that for me in my French class;)). I really enjoyed the 'bad reputation' part and agree that it is sadly viewed as not important enough by many.

Qualadore 4y ago
The spacing effect can also be applied to solving problems, like "distributed practice" as opposed to "massed practice" with calculus homework [0]. Speaking completely personally, forgetting a lot of information is the easiest way to ruin my motivation to learn a topic. [0] https://link.springer.com/article/10.1007/s10648-022-09677-2 https://link.springer.com/article/10.1007/s10648-022-09677-2

GistNoesis 4y ago

There are plenty of different way to learn, and the optimal probably depend on the material to learn. For example, in machine learning, there is something called stochastic gradient descent, where to learn you present a single random element at a time from the dataset. In the end it will have learned of all concepts, by becoming more and more confident in each individual concepts. For example to learn QM, you pick a random QM wikipedia article, and try to push through the article, even though there are some things you don't understand. Then you do the same thing, for a different unrelated QM article. For learning tennis, you don't learn specifically forehand, then learning backhand, but you alternate them at random so that you have a single unified way of playing with smooth transitions, instead of having to switch between different "modes" of thinking. Sure more memory can allow some speed-space trade-off in learning ability, but using your memory too much may make you miss some fluency that may have emerged. For example the old-school of machine learning was using databases and K-near neighbors, which used a lot of memory and was slow. But the new-school of machine learning are using constant memory algorithm and compressing the data in it, and it can learn to generate all the pictures in the world with only 4 Gb of weights. Learning is imagining, once you bootstrap your imagination, its bandwidth to synthesize new examples from which you can learn from, is much greater than the bandwidth of looking up new data material to learn from.

carapace 4y ago

It's not bad, but you're limiting yourself "out the gate" so to speak. If you really want to explore memory and learning you must study and practice (self-)hypnosis. Most of what we think we know about how the brain and mind work is wrong. E.g. memory: "our working memory has a maximum capacity of roughly 4" is not an absolute rule. Your brain, right now as you read this, is tracking 100,000's of variables and recording a trace of everything you are seeing, hearing, smelling, tasting, touching, and doing. Almost all of this is done unconsciously, and for good reason. Anyhow, you can learn to have better "interfaces" to your automatic unconscious abilities and leverage them to e.g. remember instantly and durably any mathematical equation, etc. - - - - As an aside, the strategy you sketched out at the start of your article would be workable if you have great recall, however an even more better strategy would be to recapitulate the discoveries of physics in roughly historical order. Your understanding of quantum mechanics would be much deeper and richer, and you would be following a story (an epic story made up of so many fascinating smaller stories, and one that is still going on! Albeit things have calmed down this last century or so, but no one thinks we have reached the climax yet.) That would be the way to do it: start with the Greeks and the Alchemists and proceed to follow the trail(s) of how we as a species sussed out the mysteries of the physical Universe.

aappleby 4y ago

Copy-pasting my top-level comment: I am utterly baffled that no response in this thread so far has taken issue with the statement "As you probably know intuitively, it won't work." For me, this _does_ work and I have proven it many many times over the years by adding entire categories of technical knowledge to my repertoire. And not superficially, either - I get paid very well to do things professionally that I taught myself by reading Wikipedia. If my experience were commonplace, the "it won't work" statement would be highly contentious in the comments here. Since it isn't, I guess I can deduce that I must be an outlier.