18 ms·

Why programming is a good medium for expressing poorly understood ideas (1967)

142 points by andrelaszlo 11y ago

deleted 11y ago
[deleted]
westoncb 11y ago
It is a little tough to slog through now, though I'm sure it would've been pretty enthralling in '67. I liked this from the conclusion quite a bit, though: A computer is like a violin. You can imagine a novice trying first a phonograph and then a violin. The latter, he says, sounds terrible. That is the argument we have heard from our humanists and most of our computer scientists. Computer programs are good, they say, for particular purposes, but they aren't flexible. Neither is a violin, or a typewriter, until you learn how to use it. I also liked what seems to be a pretty central point of the article (which I admittedly only skimmed): just because some medium (e.g. computer programming) has very precise requirements (e.g. in terms of syntax) for the forms expressions can take on, does not imply that the content of what can be expressed is similarly constrained.
- theoh 11y ago
  Tacit knowledge is a useful concept not mentioned by Minsky. I don't think programming is really a good way of expressing a "poorly understood and sloppily formulated idea" of this type: a third party's visual experience and verbal description of a skillful but complex human action such as shearing a sheep. The old philosophical distinction between the discursive and the intuitive is key here. Can you systematically write a program which exactly replicates your intuitive responses to some set of possible situations? I'd say it's difficult or impossible, because of the complexity of the brain and our lack of perfect introspection into what goes on in our minds. Not sure Minsky really has a lot to offer a contemporary AGI readership. His work feels more than a little mechanistic.
  - mbrock 11y ago
    Maybe you're familiar with Dreyfus's critique of "cognitivism" in AI, as explained in his book What Computers Can't Do. His recent article "Why Heideggerian AI Failed" is also relevant... and his book on Heidegger, Being-in-the-World, which has some philosophical background for his anti-cognitivism.
    - theoh 11y ago
      I wasn't familiar with Dreyfus' work. Thanks. Recently I noticed a parallel between Antoine de Saint-Exupery's comments on technology and Heidegger's notions of presentness-at-hand and readiness-to-hand. It's in his essay "The Tool", which also includes the famous line about perfection being achieved when there is "nothing left to take away." Saint-Exupery has a profoundly romantic attitude to technology: I'm not sure about the order of publication but it is a fact that Heidegger applauded Saint-Exupery's Petit Prince, and apparently Sartre felt the two men's philosophies were related. http://www.wesjones.com/wind%20sand%20stars/wind%20sand%20stars%20-%2003%20-%20the%20tool.xml http://www.wesjones.com/wind%20sand%20stars/wind%20sand%20st... (Apparently the controls of the Lockheed Lightning, the aircraft type in which Saint-Exupery died, were too complex for his taste, turning the pilot into an "accountant" rather than receding from conscious attention, as the primitive instruments of earlier, simpler planes had. Saint-Exupery clearly saw the purpose of "the tool" as facilitating his experience (and responding to his personal "authority") as a free spirit of the air, without much thought given to the constraining military command hierarchy or the pilot's job of killing.)
      - mbrock 11y ago
        Thanks for the link! I didn't know that was the context for that famous quote, and it was a nice read. It seems that we so seldom get to experience that kind of perfection in software, which is always tangled up and half nonsense. It really is a wonder that anything works. To me, the ideal of "abstraction" in software has to do with this search for unity, on a conceptual level. Unix with its files and processes seems to come pretty close to being this kind of dependable, understandable, cohesive abstraction... which is why I love shell scripting so much. Sure, it's not the most beautiful language, but the basic building blocks make sense in some almost timeless way. Digressing further, I'd really love to work on a cleaner shell language. Like, Perl is an exploration of what happens if you write your scripts in a souped-up awk, but what I really want is a bash for us new kids, with good support for nested data structures, functional patterns, some more clever ways of representing background processes and FIFOs, modules and packages, etc etc. I forget how I got from Heidegger to FIFOs, but yeah, a ready-to-hand shell that works well with common tasks like serving HTTP and building JSON and stuff. I love the Unix philosophy but people aren't really learning shell these days... something like it seems like a way forward... composing simple programs... ah!
- TheOtherHobbes 11y ago
  I disagree. The content and expression become constrained because a description of the problem domain has to be forced to fit inside them. Usually you can only make this happen with a drastically over-simplified model. One of the tragedies of AI and CS is that it's easy to delude yourself that you've understood something because you can make a toy model of it - when in fact non-toy models that are comprehensive enough to be useful to a domain expert tend to be sprawling, irregular, complicated, and not very coherent. The problem leads to a kind of paradox: Most human behaviour can't be simulated with an algorithm that's simple enough for humans to understand. Violins turn out to be a good example. It's trivially easy to sample a violin sound. But sampling is nowhere close to being able to model the musical data bandwidth in a performance by a real string quartet.
  - skybrian 11y ago
    This isn't limited to CS. It's true of all science. And yet, building models is better than not building them. "All models are wrong but some are useful" https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong
noir_lord 11y ago
My programming lecturer used to say "programming is the process of breaking a problem down so a high speed idiot can understand it, if you don't understand the problem then you can't explain it". He also said that a programmers most useful tool was a pencil and a 5mm square A4 pad. After years of solving thorny problems by getting away from the screen and scribbling in a pad I can't say he was wrong.
- dozzie 11y ago
  He was very wise on this matter. The best code I've written was thanks to first designing it on paper.
  - noir_lord 11y ago
    He was an insanely smart guy, Type 2 Unixbeard (the skinny one with the pony tail rather than Type 1, the portly one with the T-shirt of a bankrupt tech company). Back in the 80's he worked on the software that ran/runs digital switching for (iirc) BT, high scale, high reliability stuff in C.
    - meowface 11y ago
      I had only one Unixbeard professor, and he was the best one I had in my program. He was also a Type 2.
      - collyw 11y ago
        I wish I had had a Unixbeard prof.
  - 13of40 11y ago
    I don't write on paper anymore, but I still write all my comments first, then fill in the code.
    - eru 11y ago
      Haskell has a mode, where you everything is a comment by default, and lines of code have to be marked as such.
whelp 11y ago
I love ancient wisdoms.
eruditely 11y ago
> Underlying our approach to this subject is our conviction that "computer science" is not a science and that its significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects. Mathematics provides a framework for dealing precisely with notions of "what is." Computation provides a framework for dealing precisely with notions of "how to." https://mitpress.mit.edu/sicp/front/node3.html https://mitpress.mit.edu/sicp/front/node3.html From SICP preface, which is the quote that actually matters, this is inspired by Minsky's quote. I hate that the quote that went public were all the other less inspired quotes from SICP.
- jacobolus 11y ago
  Watch Hal Abelson explain this on video, right at the start of the first 1986 SICP lecture: https://www.youtube.com/watch?v=2Op3QLzMgSY https://www.youtube.com/watch?v=2Op3QLzMgSY
jbpetersen 11y ago
Another page where I'm all too happy to use stylebot and make the CSS less awful. Couldn't figure out how to resize the text unfortunately.
mrfusion 11y ago
I'd love if all the formulas on Wikipedia were written as pseudo code with meaningful variable names. I'd understand them so much faster.
- contravariant 11y ago
  I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program, and most variables don't have a fixed meaning. Can't really think of one that could be improved by writing it as pseudo code.
  - weinzierl 11y ago
    > I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program, Even every proof can be expressed as a program, see Curry–Howard correspondence [1]. > and most variables don't have a fixed meaning. I think this is kind of the point of the original comment. For a physicist PV=nRT is concise and clear but for everyone else? The letters P, V, n, R, T can have many meanings but in the context of this formula their meaning is fixed. Why not write: pressure * volume = AVOGADRO * BOLTZMAN * temperature > Can't really think of one that could be improved by writing it as pseudo code. See example above, but I think it strongly depends on what one is familiar with. [1] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
    - vlasev 11y ago
      There are a lot of conventions in physics and math and they come to the rescue. In the CONTEXT of gases, it's easy to realize that V must be volume, T must be temperature, etc. The formula PV = nRT is for physicists, not for "anyone else".
    - contravariant 11y ago
      >Even every proof can be expressed as a program, see Curry–Howard correspondence Well, that is true, but that's still pretty far removed from any common programming language. And readability doesn't usually improve. >For a physicist PV=nRT is concise and clear but for everyone else? You've somewhat got a point there, physics would be a lot easier if all formulas and variables were actually defined when needed. But symbolic notation is still a necessity, simply because symbolic manipulation is really powerful, and it's a pain to have to write 'BOLTZMAN_CONSTANT' every single time, instead of just k_B. Also, your notation might work for simple equations like the ideal gas equation, but an equation that's only slightly more complicated, the van der Waals equation, would become quite hideous: (pressure + number_of_particles^2 * attraction / volume^2) * (volume / number_of_particles + excluded_volume) == BOLTZMANN_CONSTANT * number_of_particles * temperature I really wouldn't want to have to write that out several times just to do some calculations with it.
dschiptsov 11y ago
The beauty is in that a weighted graph of lambdas could be build out of conses even back then. The idea that code is data (and it could be generated by a program itself) was already there, but they have dropped this right intuition and turned back too early, and the dark ages of Java and packer's mindset followed.
bobbylox 11y ago
"We must not fall into anthropomorphic ways of thinking about machines; they do only what their programs say; they can't be original or creative." In debunking this fallacy, I would have thought that the "Computers Are More Flexible Than You Think" argument wold only need to go halfway, and "Brains Are Less Flexible Than We Think" could take us the rest of the way there. Of course, in 1967 I imagine the support for the second argument was much thinner than it is now.

142 points by andrelaszlo 11y ago

deleted 11y ago

[deleted]

It is a little tough to slog through now, though I'm sure it would've been pretty enthralling in '67. I liked this from the conclusion quite a bit, though: A computer is like a violin. You can imagine a novice trying first a phonograph and then a violin. The latter, he says, sounds terrible. That is the argument we have heard from our humanists and most of our computer scientists. Computer programs are good, they say, for particular purposes, but they aren't flexible. Neither is a violin, or a typewriter, until you learn how to use it. I also liked what seems to be a pretty central point of the article (which I admittedly only skimmed): just because some medium (e.g. computer programming) has very precise requirements (e.g. in terms of syntax) for the forms expressions can take on, does not imply that the content of what can be expressed is similarly constrained.

theoh 11y ago
Tacit knowledge is a useful concept not mentioned by Minsky. I don't think programming is really a good way of expressing a "poorly understood and sloppily formulated idea" of this type: a third party's visual experience and verbal description of a skillful but complex human action such as shearing a sheep. The old philosophical distinction between the discursive and the intuitive is key here. Can you systematically write a program which exactly replicates your intuitive responses to some set of possible situations? I'd say it's difficult or impossible, because of the complexity of the brain and our lack of perfect introspection into what goes on in our minds. Not sure Minsky really has a lot to offer a contemporary AGI readership. His work feels more than a little mechanistic.
- mbrock 11y ago
  Maybe you're familiar with Dreyfus's critique of "cognitivism" in AI, as explained in his book What Computers Can't Do. His recent article "Why Heideggerian AI Failed" is also relevant... and his book on Heidegger, Being-in-the-World, which has some philosophical background for his anti-cognitivism.
  - theoh 11y ago
    I wasn't familiar with Dreyfus' work. Thanks. Recently I noticed a parallel between Antoine de Saint-Exupery's comments on technology and Heidegger's notions of presentness-at-hand and readiness-to-hand. It's in his essay "The Tool", which also includes the famous line about perfection being achieved when there is "nothing left to take away." Saint-Exupery has a profoundly romantic attitude to technology: I'm not sure about the order of publication but it is a fact that Heidegger applauded Saint-Exupery's Petit Prince, and apparently Sartre felt the two men's philosophies were related. http://www.wesjones.com/wind%20sand%20stars/wind%20sand%20stars%20-%2003%20-%20the%20tool.xml http://www.wesjones.com/wind%20sand%20stars/wind%20sand%20st... (Apparently the controls of the Lockheed Lightning, the aircraft type in which Saint-Exupery died, were too complex for his taste, turning the pilot into an "accountant" rather than receding from conscious attention, as the primitive instruments of earlier, simpler planes had. Saint-Exupery clearly saw the purpose of "the tool" as facilitating his experience (and responding to his personal "authority") as a free spirit of the air, without much thought given to the constraining military command hierarchy or the pilot's job of killing.)
    - mbrock 11y ago
      Thanks for the link! I didn't know that was the context for that famous quote, and it was a nice read. It seems that we so seldom get to experience that kind of perfection in software, which is always tangled up and half nonsense. It really is a wonder that anything works. To me, the ideal of "abstraction" in software has to do with this search for unity, on a conceptual level. Unix with its files and processes seems to come pretty close to being this kind of dependable, understandable, cohesive abstraction... which is why I love shell scripting so much. Sure, it's not the most beautiful language, but the basic building blocks make sense in some almost timeless way. Digressing further, I'd really love to work on a cleaner shell language. Like, Perl is an exploration of what happens if you write your scripts in a souped-up awk, but what I really want is a bash for us new kids, with good support for nested data structures, functional patterns, some more clever ways of representing background processes and FIFOs, modules and packages, etc etc. I forget how I got from Heidegger to FIFOs, but yeah, a ready-to-hand shell that works well with common tasks like serving HTTP and building JSON and stuff. I love the Unix philosophy but people aren't really learning shell these days... something like it seems like a way forward... composing simple programs... ah!
TheOtherHobbes 11y ago
I disagree. The content and expression become constrained because a description of the problem domain has to be forced to fit inside them. Usually you can only make this happen with a drastically over-simplified model. One of the tragedies of AI and CS is that it's easy to delude yourself that you've understood something because you can make a toy model of it - when in fact non-toy models that are comprehensive enough to be useful to a domain expert tend to be sprawling, irregular, complicated, and not very coherent. The problem leads to a kind of paradox: Most human behaviour can't be simulated with an algorithm that's simple enough for humans to understand. Violins turn out to be a good example. It's trivially easy to sample a violin sound. But sampling is nowhere close to being able to model the musical data bandwidth in a performance by a real string quartet.
- skybrian 11y ago
  This isn't limited to CS. It's true of all science. And yet, building models is better than not building them. "All models are wrong but some are useful" https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong

noir_lord 11y ago

My programming lecturer used to say "programming is the process of breaking a problem down so a high speed idiot can understand it, if you don't understand the problem then you can't explain it". He also said that a programmers most useful tool was a pencil and a 5mm square A4 pad. After years of solving thorny problems by getting away from the screen and scribbling in a pad I can't say he was wrong.

dozzie 11y ago
He was very wise on this matter. The best code I've written was thanks to first designing it on paper.
- noir_lord 11y ago
  He was an insanely smart guy, Type 2 Unixbeard (the skinny one with the pony tail rather than Type 1, the portly one with the T-shirt of a bankrupt tech company). Back in the 80's he worked on the software that ran/runs digital switching for (iirc) BT, high scale, high reliability stuff in C.
  - meowface 11y ago
    I had only one Unixbeard professor, and he was the best one I had in my program. He was also a Type 2.
    - collyw 11y ago
      I wish I had had a Unixbeard prof.
- 13of40 11y ago
  I don't write on paper anymore, but I still write all my comments first, then fill in the code.
  - eru 11y ago
    Haskell has a mode, where you everything is a comment by default, and lines of code have to be marked as such.

whelp 11y ago

I love ancient wisdoms.

eruditely 11y ago

> Underlying our approach to this subject is our conviction that "computer science" is not a science and that its significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects. Mathematics provides a framework for dealing precisely with notions of "what is." Computation provides a framework for dealing precisely with notions of "how to." https://mitpress.mit.edu/sicp/front/node3.html https://mitpress.mit.edu/sicp/front/node3.html From SICP preface, which is the quote that actually matters, this is inspired by Minsky's quote. I hate that the quote that went public were all the other less inspired quotes from SICP.

jacobolus 11y ago
Watch Hal Abelson explain this on video, right at the start of the first 1986 SICP lecture: https://www.youtube.com/watch?v=2Op3QLzMgSY https://www.youtube.com/watch?v=2Op3QLzMgSY

jbpetersen 11y ago

Another page where I'm all too happy to use stylebot and make the CSS less awful. Couldn't figure out how to resize the text unfortunately.

mrfusion 11y ago

I'd love if all the formulas on Wikipedia were written as pseudo code with meaningful variable names. I'd understand them so much faster.

contravariant 11y ago
I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program, and most variables don't have a fixed meaning. Can't really think of one that could be improved by writing it as pseudo code.
- weinzierl 11y ago
  > I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program, Even every proof can be expressed as a program, see Curry–Howard correspondence [1]. > and most variables don't have a fixed meaning. I think this is kind of the point of the original comment. For a physicist PV=nRT is concise and clear but for everyone else? The letters P, V, n, R, T can have many meanings but in the context of this formula their meaning is fixed. Why not write: pressure * volume = AVOGADRO * BOLTZMAN * temperature > Can't really think of one that could be improved by writing it as pseudo code. See example above, but I think it strongly depends on what one is familiar with. [1] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
  - vlasev 11y ago
    There are a lot of conventions in physics and math and they come to the rescue. In the CONTEXT of gases, it's easy to realize that V must be volume, T must be temperature, etc. The formula PV = nRT is for physicists, not for "anyone else".
  - contravariant 11y ago
    >Even every proof can be expressed as a program, see Curry–Howard correspondence Well, that is true, but that's still pretty far removed from any common programming language. And readability doesn't usually improve. >For a physicist PV=nRT is concise and clear but for everyone else? You've somewhat got a point there, physics would be a lot easier if all formulas and variables were actually defined when needed. But symbolic notation is still a necessity, simply because symbolic manipulation is really powerful, and it's a pain to have to write 'BOLTZMAN_CONSTANT' every single time, instead of just k_B. Also, your notation might work for simple equations like the ideal gas equation, but an equation that's only slightly more complicated, the van der Waals equation, would become quite hideous: (pressure + number_of_particles^2 * attraction / volume^2) * (volume / number_of_particles + excluded_volume) == BOLTZMANN_CONSTANT * number_of_particles * temperature I really wouldn't want to have to write that out several times just to do some calculations with it.

dschiptsov 11y ago

The beauty is in that a weighted graph of lambdas could be build out of conses even back then. The idea that code is data (and it could be generated by a program itself) was already there, but they have dropped this right intuition and turned back too early, and the dark ages of Java and packer's mindset followed.

bobbylox 11y ago

"We must not fall into anthropomorphic ways of thinking about machines; they do only what their programs say; they can't be original or creative." In debunking this fallacy, I would have thought that the "Computers Are More Flexible Than You Think" argument wold only need to go halfway, and "Brains Are Less Flexible Than We Think" could take us the rest of the way there. Of course, in 1967 I imagine the support for the second argument was much thinner than it is now.