12 ms·

I'll have to admit something about myself -- I've never been able to learn how to do anything from just reading its first principles. I need to have a working e

by anon4 12y ago

I'll have to admit something about myself -- I've never been able to learn how to do anything from just reading its first principles. I need to have a working example that I can then adapt and change and observe changes in the result. This has been especially true in programming. I've been writing programs in some capacity for 18 years now and not once in that time have I been able to simply read the reference manual of anything and following that write working code on my first try. Well, obviously I've been able to do it for trivial things, like after the Python 2->3 switch when map/filter/range started returning iterables instead of lists, simply being told "map/filter/range/etc. now return an iterable and not a list" is sufficient to explain the change in behaviour, but that's only because I know all the concepts behind it. You could say the explanation is only one level removed from my intuitive knowledge. Things start getting exponentially harder the more levels removed they get, i.e. if it's a new concept explained using new concepts previously defined, but for which I have no intuitive knowledge what they mean.

GauntletWizard 12y ago
Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'. Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key to science. You have to observe, change, guess, run, observe, change, observe until you can successfully predict what will happen for each change you make. The scientific method is taught to everyone, even if most ignore it and skip steps. 'Guess/check' is the correct way to do real-world engineering and design, not pure-logic simulation, because we still don't know all the first principles.
- deleted 12y ago
  [deleted]
- avmich 12y ago
  > Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However, both experimenting and reasoning can be flawed, and part of the trick is to learn how to do that properly. Computer science is difficult.
  - GauntletWizard 12y ago
    Programming is a branch of engineering. Very little programming is done by proof; It's done by experimentation, debugger, and tests are human-reasoned and hopefully cover all codepaths and critical junctures (for every x > 0, you test x=1, x=0, x=-1). Proof is of critical importance - It's how you find those junctures, for example, how you optimize, how you design and how you reason about runtime, etc. But programming as practiced in the field is engineering first, mathematics second.
    - dasil003 12y ago
      Reminds me of the famous Knuth quote: > Beware of bugs in the above code; I have only proved it correct, not tried it. Deep thinking and mathematical proofs are tremendously useful when thinking about a well-defined and scoped problem at a more or less fixed level of abstraction. The problem with software engineering is that in practice all abstractions are leaky, and so you inevitably find yourself dealing with issues that are too far-flung and random to be mathematically tractable. The best you can do is be rigorous about the core problem you are solving, but there is always some amount of hammering and duct tape to build it into a non-trivial real-world system. It's possible to attack this problem asymptotically with a rigorous engineering process such as NASA employs, but we don't because it's simply not cost effective for the majority of software.
      - Osiris 12y ago
        because it's simply not cost effective for the majority of software. That's a great point. We utilize best practices, like automated tests, to narrow the gap between pure experimentation and mathematical proof. Proofs in most code would be difficult to impossible because most of use libraries that use other libraries, etc., so we just do the best we can given the time and monetary constraints that we have.
- Dewie 12y ago
  > Experimentation is the key to science. A sort of humanitarian science, since almost everything we touch have been made by other humans. But somehow STEM people are famous for looking down their nose on the non-hard sciences.
- ObviousScience 12y ago
  Any time I have to develop something from first principles, I do it 2-3 times in different ways, and only then make a serious attempt at actually writing it. It's still terrible, of course, but at least has a chance of working since I have a bit of experience in what can go wrong.
  - jacquesm 12y ago
    That's exactly my method. What I've noticed in the last couple of years that there are more and more programmers that could not start from a 'blank page' if their life depended on it. That's a sad thing in my opinion, programming is a creative job and creation sometimes needs to start from nothing or extremely little.
- vinceguidry 12y ago
  It's not really worthwhile to try to code from first principles, because the context can always multiply in complexity. You want to make a tic-tac-toe game. What language are you going to program it in? Is it going to be cross-platform? Are you going to code an AI engine? Trying to do anything in computing from first principles is like trying to code without mistakes. You're losing the value of iterative design.
  - yoklov 12y ago
    I disagree. It's worth working things out from first principals as a method of practice, because when you have to do something completely novel, you won't be totally lost (If you never have to do this in your work, then I guess it doesn't apply). Also, if you never work things out yourself, we'll only ever have one way to do things. What if a better approach exists but nobody's ever tried it? It's good to double/sanity check what you came up with afterwards, and if the only way you can think of to do something is obviously seriously flawed, then don't go through with it and look it up instead.
- sklogic 12y ago
  Engineering is not a science, there is very little space for discovery there. In engineering you're applying science, not creating a new scientific knowledge.
  - TheOtherHobbes 12y ago
    Engineering lives on a continuum from almost pure maths to pure cookbook canned solutions, with a much smaller arrow going in the opposite direction from unexplained observations to theorising and model building. Things like Shannon's description of information-as-entropy were certainly a mathematical discovery about engineered systems, and led directly to a lot of coding and data compression theory. There wasn't a whole lot of interest in quantum channels at the time, so the fact that the theory fed something back to mainstream physics was a bonus.
Gravityloss 12y ago
Gall's law is related to this. I think it's reasonable that people want to create software components that have been proven to work and then just forget about their internal details and copy-pas... apply them. It's also what Bret Victor has been getting at. Very often the component's input-output mapping is much more relevant than implementation details. And you can get a feel for that by fiddling with the components.
- prawn 12y ago
  Gall's Law for anyone who didn't know or couldn't remember it: "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system." – John Gall (1975)
  - fsloth 12y ago
    Thanks for the reference, this is the best quote I've found in a long while. Strangely, I was unfamiliar with this.
  - ylem 12y ago
    Incidentally, he passed away recently. He had a varied career and was actually my pediatrician and encouraged my interest in astronomy. I only learned at his death about his contributions to systems theory...
- ThrustVectoring 12y ago
  A microwave oven's input-output mapping is much more relevant than how it works. And you can get a feel for that by plugging it in and pressing the buttons. It's a very general point - how to interact with things is often dependent on the surface, not the volume. This is the generative insight for the systems approach to engineering.
- chipsy 12y ago
  I think it's crucial to the maintenance of our superstition that we don't know which parts of the system are actually important, and we keep misleading ourselves into the wrong assumption. For example - style considerations. Important? Yes? Maybe? Not at the moment when it reaches the computer, but important when humans have to grok it. So then should we "lead" development by promoting style? Will that get us somewhere? Probably not as much as some other strategy. But it's still sort of important and this blogger made a nice post the other day about style, so...
engi_nerd 12y ago
This is probably why I am an engineer instead of a pure scientist. I want to see something working, and be free to play with it, take it apart, put it back together, combine it with other things I know, etc. I do this with computer programs, with engineering systems, with mathematics. The more I do this in a given area the more intuition I build up and the more easily I can then learn something else from first principles if I need to. But I really do prefer to get my hands dirty first. Other people don't do this -- chalk that up to different learning styles. I try to guard against cargo-culting (I abhor it).
itchyouch 12y ago
As you've pointed out, "no intuitive knowledge", the first principles of a language/api/framework come implicitly attached with design patterns that are probably fairly deep for whatever reason. Your usage of them gets you familiar with the framework, thus your ability to work with them. I would also concur that practice is necessary before being able to construct anything from scratch. The cognition of programming languages is very much like regular human languages. It takes a tremendous amount of disciplined, repetitive practice and observation before the mastery of a language allows it to just flow effortlessly out of ones fingers. A simple concrete example is the difference in articulative ability in constructing the same exact sentence for an essay between a 12, 18 and 25 year old post-doc. For the post-doc who has written a countless number of 10+ page essays versus the 12 year old in 6th grade who has more-than-likely never written anything beyond 1-2 pages, the ability is obvious and striking. The patterns and structures come naturally to the well-practiced individual as the Chinese parable of ZhaungZhi teaches us the the concept of achieving flow (WuWei). The difficulty with mastering programming the way we speak english (or any other primary language for that matter) is that many projects only ever need to construct the equivalent of a proper sentence once, then that sentence is committed to source control for eternity without ever needing to be constructed again, thus the cognitive muscles creating programs tend to not achieve flow, but the muscles of identifying, locating, copying, pasting & modifying do tend to achieve flow. To exacerbate our inability to construct things from scratch, we also look to reduce the monotony and abstract away the difficulty and reduce what actually needs to be written and kept track of from our keyboards via APIs and abstractions that segregate duty. Of course I'm not advocating that good architecture be thrown out for the sake of practice, but in a sense, programming could use the equivalent of musical scales, where the well known patterns, and language-native constructs are exercised daily (frequently enough) to the point of trivial mastery. The argument against such kinds of practice from my peers & others on this board has been, "duh, that's what API documentation and computers are for." "This stuff can be looked up." "This is why whiteboard programming exercises are not indicative of programming ability." "See, why learn math, when I can just type x+y into the console and get the result?" However, I'd argue that the effortless mastery that comes from daily practice comes with tremendous benefit. The mastery allows one to focus on the architecture problem set rather than the details of what to name a method (assuming that good method names come naturally from practice). Or be encumbered by whether the first or second argument in the split method should be the separator. Would it be acceptable as a passenger of a bus if the driver claimed that they did not need to know whether the left or right pedal was the gas or brake pedal since that could be looked up via Google in the API documentation for the bus? Of course the scope of learning how to drive can be mastered in a couple of months, but programming is more equivalent to spoken language and perhaps mastery of playing musical instruments. I would argue that much of the arguments for or against certain interview techniques/questions/strategies really stems on what the interviewers and interviewees see as a signal for effortless mastery. Esoteric pet questions test for intimacy with a narrow topic, but also get to assume that other deep-related knowledge follows along. Whiteboard interviews are looking for problem solving ability, but also effortless mastery of a language and basic data structures that someone with 1-2+ years in a language probably should have. But at this point, I digress.
rtpg 12y ago
teaching from first principles doesn't hold up experimentally. Every documented case of teaching e.g. mathematics or languages from first principles without first going through practical examples or the like has resulted in failures. I'm pretty sure nobody actually learns from first principles,but understand first principles from things they've already seen
- sklogic 12y ago
  Teaching mathematics not in a constructive way is extremely harmful. One have to wipe out all the previously acquired superstitions, all that high school crap out of the students minds before reinstating a proper, constructive knowledge there. Which raises the question - what was the point of filling the minds with all that crap in the first place? Same thing with languages, there are schools which approach foreign languages formally, starting with grammar and all that. Some of that schools are known for training spies, for example, which is sort of an indication of their quality.
marak830 12y ago
Thankyou! Also to everyone else who replied. I thought i was a bad programmer. I am cobstantly in a cycle of read, try, adjust, repeat until it works, rebuild it now i understand better how it works.
sklogic 12y ago
It's the ability which is best trained by studying mathematics (even if totally unrelated to programming). Do not listen to those saying it's not possible. We do implement programming languages based on their specifications, we implement protocols by reading RFCs, we implement numeric algorithms by reading pseudocode in the papers, etc. It's all trivial and mechanical. The latter notion is very important, one have to understand that there is very rarely a place for "creativity" and even thinking. You simply translate specification from one formal language into another, following simple rules. Thinking too much considered harmful.
- icebraining 12y ago
  We do implement programming languages based on their specifications, we implement protocols by reading RFCs, we implement numeric algorithms by reading pseudocode in the papers, etc. We implement spherical-cow versions of languages, protocols, etc by reading those papers and specs. Then we spend months or years fixing it to implement the actual languages and protocols.
  - sklogic 12y ago
    No. We spend months and years fixing the issues with the specs (especially if they're committee-driven, they tend to be crappy). Yet it's easy to keep your implementation conformant to the specification without any hand weaving and looking at "examples".
    - icebraining 12y ago
      You can keep it conformant, but that's not enough, because the real world isn't. Try writing a conformant HTML parser that rejects invalid input, and then run a crawler that uses it. You'll reject half the web!
      - sklogic 12y ago
        In such a case the right way to go is to build a formal model of the informal expectations first. I.e., split scientific and engineering phases. Collect your data, build a theory, validate it, and only then engineer an implementation of it, mechanically, with zero mental activity. And what you seem to suggest, start writing a parser and then experiment with various real-world inputs until you're satisfied, is certainly not a very productive way of doing things.

by anon4 12y ago

GauntletWizard 12y ago

Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'. Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key to science. You have to observe, change, guess, run, observe, change, observe until you can successfully predict what will happen for each change you make. The scientific method is taught to everyone, even if most ignore it and skip steps. 'Guess/check' is the correct way to do real-world engineering and design, not pure-logic simulation, because we still don't know all the first principles.

deleted 12y ago
[deleted]
avmich 12y ago
> Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However, both experimenting and reasoning can be flawed, and part of the trick is to learn how to do that properly. Computer science is difficult.
- GauntletWizard 12y ago
  Programming is a branch of engineering. Very little programming is done by proof; It's done by experimentation, debugger, and tests are human-reasoned and hopefully cover all codepaths and critical junctures (for every x > 0, you test x=1, x=0, x=-1). Proof is of critical importance - It's how you find those junctures, for example, how you optimize, how you design and how you reason about runtime, etc. But programming as practiced in the field is engineering first, mathematics second.
  - dasil003 12y ago
    Reminds me of the famous Knuth quote: > Beware of bugs in the above code; I have only proved it correct, not tried it. Deep thinking and mathematical proofs are tremendously useful when thinking about a well-defined and scoped problem at a more or less fixed level of abstraction. The problem with software engineering is that in practice all abstractions are leaky, and so you inevitably find yourself dealing with issues that are too far-flung and random to be mathematically tractable. The best you can do is be rigorous about the core problem you are solving, but there is always some amount of hammering and duct tape to build it into a non-trivial real-world system. It's possible to attack this problem asymptotically with a rigorous engineering process such as NASA employs, but we don't because it's simply not cost effective for the majority of software.
    - Osiris 12y ago
      because it's simply not cost effective for the majority of software. That's a great point. We utilize best practices, like automated tests, to narrow the gap between pure experimentation and mathematical proof. Proofs in most code would be difficult to impossible because most of use libraries that use other libraries, etc., so we just do the best we can given the time and monetary constraints that we have.
Dewie 12y ago
> Experimentation is the key to science. A sort of humanitarian science, since almost everything we touch have been made by other humans. But somehow STEM people are famous for looking down their nose on the non-hard sciences.
ObviousScience 12y ago
Any time I have to develop something from first principles, I do it 2-3 times in different ways, and only then make a serious attempt at actually writing it. It's still terrible, of course, but at least has a chance of working since I have a bit of experience in what can go wrong.
- jacquesm 12y ago
  That's exactly my method. What I've noticed in the last couple of years that there are more and more programmers that could not start from a 'blank page' if their life depended on it. That's a sad thing in my opinion, programming is a creative job and creation sometimes needs to start from nothing or extremely little.
vinceguidry 12y ago
It's not really worthwhile to try to code from first principles, because the context can always multiply in complexity. You want to make a tic-tac-toe game. What language are you going to program it in? Is it going to be cross-platform? Are you going to code an AI engine? Trying to do anything in computing from first principles is like trying to code without mistakes. You're losing the value of iterative design.
- yoklov 12y ago
  I disagree. It's worth working things out from first principals as a method of practice, because when you have to do something completely novel, you won't be totally lost (If you never have to do this in your work, then I guess it doesn't apply). Also, if you never work things out yourself, we'll only ever have one way to do things. What if a better approach exists but nobody's ever tried it? It's good to double/sanity check what you came up with afterwards, and if the only way you can think of to do something is obviously seriously flawed, then don't go through with it and look it up instead.
sklogic 12y ago
Engineering is not a science, there is very little space for discovery there. In engineering you're applying science, not creating a new scientific knowledge.
- TheOtherHobbes 12y ago
  Engineering lives on a continuum from almost pure maths to pure cookbook canned solutions, with a much smaller arrow going in the opposite direction from unexplained observations to theorising and model building. Things like Shannon's description of information-as-entropy were certainly a mathematical discovery about engineered systems, and led directly to a lot of coding and data compression theory. There wasn't a whole lot of interest in quantum channels at the time, so the fact that the theory fed something back to mainstream physics was a bonus.

Gravityloss 12y ago

Gall's law is related to this. I think it's reasonable that people want to create software components that have been proven to work and then just forget about their internal details and copy-pas... apply them. It's also what Bret Victor has been getting at. Very often the component's input-output mapping is much more relevant than implementation details. And you can get a feel for that by fiddling with the components.

prawn 12y ago
Gall's Law for anyone who didn't know or couldn't remember it: "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system." – John Gall (1975)
- fsloth 12y ago
  Thanks for the reference, this is the best quote I've found in a long while. Strangely, I was unfamiliar with this.
- ylem 12y ago
  Incidentally, he passed away recently. He had a varied career and was actually my pediatrician and encouraged my interest in astronomy. I only learned at his death about his contributions to systems theory...
ThrustVectoring 12y ago
A microwave oven's input-output mapping is much more relevant than how it works. And you can get a feel for that by plugging it in and pressing the buttons. It's a very general point - how to interact with things is often dependent on the surface, not the volume. This is the generative insight for the systems approach to engineering.
chipsy 12y ago
I think it's crucial to the maintenance of our superstition that we don't know which parts of the system are actually important, and we keep misleading ourselves into the wrong assumption. For example - style considerations. Important? Yes? Maybe? Not at the moment when it reaches the computer, but important when humans have to grok it. So then should we "lead" development by promoting style? Will that get us somewhere? Probably not as much as some other strategy. But it's still sort of important and this blogger made a nice post the other day about style, so...

engi_nerd 12y ago

This is probably why I am an engineer instead of a pure scientist. I want to see something working, and be free to play with it, take it apart, put it back together, combine it with other things I know, etc. I do this with computer programs, with engineering systems, with mathematics. The more I do this in a given area the more intuition I build up and the more easily I can then learn something else from first principles if I need to. But I really do prefer to get my hands dirty first. Other people don't do this -- chalk that up to different learning styles. I try to guard against cargo-culting (I abhor it).

itchyouch 12y ago

As you've pointed out, "no intuitive knowledge", the first principles of a language/api/framework come implicitly attached with design patterns that are probably fairly deep for whatever reason. Your usage of them gets you familiar with the framework, thus your ability to work with them. I would also concur that practice is necessary before being able to construct anything from scratch. The cognition of programming languages is very much like regular human languages. It takes a tremendous amount of disciplined, repetitive practice and observation before the mastery of a language allows it to just flow effortlessly out of ones fingers. A simple concrete example is the difference in articulative ability in constructing the same exact sentence for an essay between a 12, 18 and 25 year old post-doc. For the post-doc who has written a countless number of 10+ page essays versus the 12 year old in 6th grade who has more-than-likely never written anything beyond 1-2 pages, the ability is obvious and striking. The patterns and structures come naturally to the well-practiced individual as the Chinese parable of ZhaungZhi teaches us the the concept of achieving flow (WuWei). The difficulty with mastering programming the way we speak english (or any other primary language for that matter) is that many projects only ever need to construct the equivalent of a proper sentence once, then that sentence is committed to source control for eternity without ever needing to be constructed again, thus the cognitive muscles creating programs tend to not achieve flow, but the muscles of identifying, locating, copying, pasting & modifying do tend to achieve flow. To exacerbate our inability to construct things from scratch, we also look to reduce the monotony and abstract away the difficulty and reduce what actually needs to be written and kept track of from our keyboards via APIs and abstractions that segregate duty. Of course I'm not advocating that good architecture be thrown out for the sake of practice, but in a sense, programming could use the equivalent of musical scales, where the well known patterns, and language-native constructs are exercised daily (frequently enough) to the point of trivial mastery. The argument against such kinds of practice from my peers & others on this board has been, "duh, that's what API documentation and computers are for." "This stuff can be looked up." "This is why whiteboard programming exercises are not indicative of programming ability." "See, why learn math, when I can just type x+y into the console and get the result?" However, I'd argue that the effortless mastery that comes from daily practice comes with tremendous benefit. The mastery allows one to focus on the architecture problem set rather than the details of what to name a method (assuming that good method names come naturally from practice). Or be encumbered by whether the first or second argument in the split method should be the separator. Would it be acceptable as a passenger of a bus if the driver claimed that they did not need to know whether the left or right pedal was the gas or brake pedal since that could be looked up via Google in the API documentation for the bus? Of course the scope of learning how to drive can be mastered in a couple of months, but programming is more equivalent to spoken language and perhaps mastery of playing musical instruments. I would argue that much of the arguments for or against certain interview techniques/questions/strategies really stems on what the interviewers and interviewees see as a signal for effortless mastery. Esoteric pet questions test for intimacy with a narrow topic, but also get to assume that other deep-related knowledge follows along. Whiteboard interviews are looking for problem solving ability, but also effortless mastery of a language and basic data structures that someone with 1-2+ years in a language probably should have. But at this point, I digress.

rtpg 12y ago

teaching from first principles doesn't hold up experimentally. Every documented case of teaching e.g. mathematics or languages from first principles without first going through practical examples or the like has resulted in failures. I'm pretty sure nobody actually learns from first principles,but understand first principles from things they've already seen

sklogic 12y ago
Teaching mathematics not in a constructive way is extremely harmful. One have to wipe out all the previously acquired superstitions, all that high school crap out of the students minds before reinstating a proper, constructive knowledge there. Which raises the question - what was the point of filling the minds with all that crap in the first place? Same thing with languages, there are schools which approach foreign languages formally, starting with grammar and all that. Some of that schools are known for training spies, for example, which is sort of an indication of their quality.

marak830 12y ago

Thankyou! Also to everyone else who replied. I thought i was a bad programmer. I am cobstantly in a cycle of read, try, adjust, repeat until it works, rebuild it now i understand better how it works.

sklogic 12y ago

It's the ability which is best trained by studying mathematics (even if totally unrelated to programming). Do not listen to those saying it's not possible. We do implement programming languages based on their specifications, we implement protocols by reading RFCs, we implement numeric algorithms by reading pseudocode in the papers, etc. It's all trivial and mechanical. The latter notion is very important, one have to understand that there is very rarely a place for "creativity" and even thinking. You simply translate specification from one formal language into another, following simple rules. Thinking too much considered harmful.

icebraining 12y ago
We do implement programming languages based on their specifications, we implement protocols by reading RFCs, we implement numeric algorithms by reading pseudocode in the papers, etc. We implement spherical-cow versions of languages, protocols, etc by reading those papers and specs. Then we spend months or years fixing it to implement the actual languages and protocols.
- sklogic 12y ago
  No. We spend months and years fixing the issues with the specs (especially if they're committee-driven, they tend to be crappy). Yet it's easy to keep your implementation conformant to the specification without any hand weaving and looking at "examples".
  - icebraining 12y ago
    You can keep it conformant, but that's not enough, because the real world isn't. Try writing a conformant HTML parser that rejects invalid input, and then run a crawler that uses it. You'll reject half the web!
    - sklogic 12y ago
      In such a case the right way to go is to build a formal model of the informal expectations first. I.e., split scientific and engineering phases. Collect your data, build a theory, validate it, and only then engineer an implementation of it, mechanically, with zero mental activity. And what you seem to suggest, start writing a parser and then experiment with various real-world inputs until you're satisfied, is certainly not a very productive way of doing things.