8 ms·

We Don't Know How to Compute (2011) [video]

126 points by ducktective 3y ago

AndrewOMartin 3y ago
This is a video of a talk from Gerald Sussman in 2011 at the Strange Loop conference. Abstract below. Though we have been building and programming computing machines for about 60 years and have learned a great deal about composition and abstraction, we have just begun to scratch the surface. A mammalian neuron takes about ten milliseconds to respond to a stimulus. A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it. The human genome -- the information required to build a human from a single, undifferentiated eukariotic cell -- is about 1GB. The instructions to build a mammal are written in very dense code, and the program is extremely flexible. Only small patches to the human genome are required to build a cow or a dog rather than a human. Bigger patches result in a frog or a snake. We don't have any idea how to make a description of such a complex machine that is both dense and flexible. New design principles and new linguistic support are needed. I will address this issue and show some ideas that can perhaps get us to the next phase of engineering design. Gerald Sussman Massachusetts Institute of Technology
- danybittel 3y ago
  And yet the computer is so much better in many areas than a human. It so weird to compare the two. Why would you even want that? We have already plenty of brains on earth. Who sais, that once a computer works like a brain.. it doesn't also come with it's downsides? There are lot's of problems in compute.. making it more brain like, more linguistic isn't going to solve that. I believe the whole notion of "a computer as an assistant", is doing more damage than good.
  - moffkalast 3y ago
    Well an assistant is traditionally the one who eventually replaces the master, so that notion is not conceptually wrong. Do people still think there will be any humans left alive in a few hundred years? We've gone as far as we could as biological constructs, switching to a more adaptable architecture is just the next step forward. Our bodies are overfitted for living on this one specific planet which is greatly limiting the real goal of life: to expand to new areas and replicate.
    - kweingar 3y ago
      > Do people still think there will be any humans left alive in a few hundred years? I will go on the record and predict there will be some human beings alive in a few hundred years.
      - moffkalast 3y ago
        Alright there will always be some that stubbornly cling on, but I would bet that most entities that could be considered intelligent beings won't be human by that point, just out of practicality.
        imtringued 3y ago
        Stupid flesh and blood humans with their finite needs! Let's build cyborgs with infinite information processing capability so their needs can never be saturated!
      - TeMPOraL 3y ago
        Question is, would they be alive by choice, and would they want to continue being alive?
        kweingar 3y ago
        Are you and I alive by choice? I don’t know what this means. I will go further and boldly predict that some of these future people will like living.
        TeMPOraL 3y ago
        Ok, I'll spell out what I meant. One of the possible scenarios where, centuries after GAI was created, there would still be humans (recognizable to us as such) alive, is the same as the reason why horses and pigs and cows are alive today - because they're bred as inputs to some industrial process.
  - pfarrell 3y ago
    My takeaway is that Sussman is looking at his field after a lifetime of contribution. He's looking for signposts on where we can go next. He's not valuing one over the other, but looking at the brain and saying, "Here's something that exists that's operating on problems in a way we can't begin to approach using current software/hardware techniques". He sees some hints available to us in the noise reduction things he talks about at the end. In that way, the talk is more him challenging the audience to question our approaches and implied constraints rather than him trying to present us with an answer. It's one of my favorite videos (the way too loud HI at the beginning, notwithstanding).
- unstuck3958 3y ago
  > So the computational depth of this behavior is only a few tens of steps. One of the primary "criticisms" of pre-LLM machine learning was that it is inefficient compared to, say, a human, because it requires a plethora of examples to classify something just to match the performance of a human who has only seen a few examples. That argument no longer holds the same value since the discovery of few-shot learning features of LLMs (though they still have a long way to go). Which makes me think: maybe the reason behind human brain's apparent "efficiency" is the complexity that hides beneath the billions of neurons we have. Disclaimer: I am an amateur and probably have only a litte idea of what I'm talking about.
  - imtringued 3y ago
    Sorry but there is no need for a whole brain to outperform ANNs in learning speed. https://bigthink.com/neuropsych/brain-cells-chip-play-pong/ https://bigthink.com/neuropsych/brain-cells-chip-play-pong/
- deleted 3y ago
  [deleted]
- MichaelZuo 3y ago
  F1 drivers can instinctively respond to a minor bump on a curve and make a decision in probably ~100 milliseconds, so the lower bound is just 10 computational steps. A more complex decision such as swerving to avoid a wheel that just came off the car in front, probably takes around ~500 milliseconds, or 50 computational steps. Considering that modern computers need billions of computational steps to start a calculator app and draw it on the desktop, we have a long way to go. Though there has been regression on this point, a 1990 NeXTStation Color could do the same with ~50 million cycles on a 1120×832 display. (on 12 MB of RAM and 1.5 MB of VRAM.)
  - TeMPOraL 3y ago
    I think the trick is, our computational steps are massively parallel. Take the visual system. By the time you notice - consciously - the pothole you're about to drive into, it's already processed into high-level concepts. Our visual senses aren't camera feeds - the image we are aware of isn't raw input from the eyes, but rather it's generated out of some combination of raw inputs, low-level hacks to hide limited FOV and discontinuities caused by saccades, and higher-level interpretations based on how we feel, what seems to fit the context based on our prior experiences, etc. I think it makes sense to assume that this process has many extra off-ramps that can trigger high-level reactions while bypassing conscious awareness. I.e. imagine that half-way through the post-processing stack, the relevant parts of the brain are somewhat convinced you're looking at a fast-moving object about to hit you in the face. There's a point at which it makes sense for the brain to make your body start a dodging move, even if it may be a misprediction, rather than waste a few dozen milliseconds to make sure, at the risk of getting a ball (or a rock, or a fist) up your nose.
    - marcosdumay 3y ago
      > I think the trick is, our computational steps are massively parallel. Yes, one trick is that. Our computers have to serialize everything. We don't know how to build or program massively parallel computers with distributed memory. But the GP already touched on that point.
      - 3y ago
- lostmsu 3y ago
  > A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it. IMHO, that makes no sense. Assuming we have a really large state machine with all many possible situations in a state of driving on a straight highway with a few vehicles with more or less constant relative velocity, determining if we need to make a turn is definitely doable in a few computation steps.
  - leroy-is-here 3y ago
    We attempted the symbolic route of AI programming and it didn’t work because it is explosive in computation time and exhaustive to model in code. We can’t program our AI “models” now because they are based on chains of probabilities. Therefore they can run in real time. They don’t have any real logic that we specifically programmed, only what the model was able to infer and then encode. Perhaps there should be a marriage between these two attempts of AI.
    - marcosdumay 3y ago
      > it is explosive in computation time Hum... The numerical route for AI necessarily has the same complexity in computation time. Those two have equivalent semantics. If you can get a result with one, you can get the same result with the other, it can just differ on how easy it is to program. By the way, symbolic AI can learn too, you don't need to model things in code.
      - leroy-is-here 3y ago
        I was talking about attempts such as expert systems. We attempted to symbolically encode things as sums of their parts in plain terms, e.g. a bicycle is two wheels, an axle, a chain, some peddles, etc. but in more detail than that. The problem is the level of detail required for general knowledge: firstly it is hard to codify, secondly as it grows computation becomes exponentially more expensive. We as humans have the intelligence to simply see for fractions of a second and then just know — we can know the answers to math problems and how a person is feeling. The symbolic AI programming was an attempt to codify what symbols we are seeing but they are so diverse and nuanced it became apparent this was an ineffective scientific route.
- karmakaze 3y ago
  Not really a fair comparison--250 ms is reaction time. A deliberate conscious response would be much longer. "Reaction times can reflect habits rather than computations"[0]. [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5582865/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5582865/
- bionhoward 3y ago
  Seems like fractals are the difference maker here because if you don’t use a fractal you have to program the whole thing whereas if you use fractals (embryology) then your code can be massively recursive. Alternative splicing and epigenetics (holy shit, I can’t believe the iPhone autocorrect doesn’t know the word “epigenetics” in ‘23) also multiply the flexibility of the genome. Also, phosphorylation: proteins are not Boolean switches, they have arbitrary numbers of activation states. Ternary logic is already way more dense than Boolean so when you go even further like into 6-ary, 9-ary logics you’re a lot more free to program stuff. Also the network motifs are a lot simpler than first appear. The metabolic map looks like spaghetti on the wall but each point is composed of reusable blocks like feedforward and feedback etc (see Uri Alon’s book and “Wetware” book and Synthetic Biology: A primer for more detail) I’m biased as a molecular biologist but genetics and systems biology are the most incredible thing. Natural nanotech
  - MisterTea 3y ago
    > Natural nanotech When I was a kid in the 80's there were a number of gross/gore/horror trading cards for kids - Garbage Pail Kids being the prime example. One of the sets I had was set in some far flung future, I can barely remember the series but part of the theme were living, biological machines. The only card I can clearly remember featured a flying prisoner transport where the prisoners were seated in the ships cavernous stomach and "if upset during transport could accidentally digest the prisoners." (Honorable mention to HL2 featuring "synthtech" - the drop ship, strider and gun ships were living biological machines.) This stuck in my head as the idea of designing machines which grow like plants or raised like animals was an incredibly amazing and interesting concept. But as I got older and educated the idea them seemed as far flung as the cards themselves. Though reading you post makes me think, maybe they aren't so far flung after all. Though I still fell that a creature which grows steel or titanium skeletons is still quite firmly in the realm of sci-fi.
- alcover 3y ago
  > The human genome [...] is about 1GB I don't get how this is impressive or relevant on its own. Sure, it sounds small in face of the final result : a full-grown human. But this 1GB program is born out of and expressed into the full OS that is the physical world.
CraigJPerry 3y ago
We do keep looking under the proverbial lamppost for the missing keys[1] when it comes to computation. Compilers for the most popular languages are pretty amazing these days. Yet it’s still on the developer to make the most of them. One aspect might be performance: - structure your problem in such a way (carefully manage dependency relationships, eliminate branches, etc). that the compiler can choose to emit vector instructions - manage memory access patterns (e.g. prefer columnar and SoA access patterns over cache unfriendly pointer chasing structures) - understand how to maximally exploit cache hierarchies for your problem (e.g. threading to exploit the other L1 caches rather than relying on slower L3 etc.) Another aspect might be data security rules, e.g. tagging and labelling with the ability to enforce rules for the developer. But there are literally hundreds of aspects to deal with. Our languages today look at the problem space through a toilet roll tube with the other eye closed. The future of computation is guaranteed to be rich, vibrant and exciting. [1] https://en.m.wikipedia.org/wiki/Streetlight_effect https://en.m.wikipedia.org/wiki/Streetlight_effect
- messe 3y ago
  I think every suggestion you have there would fall under that very same lamppost: a linear model of computation going only one direction. Sussman details in this talk a different model he calls propagators that deal with information flow back and forth, allowing for some interesting ways of working with "fuzzy" and incomplete data, while still allowing for cheap backtracking if a contradiction arises. I haven't read the Radul & Sussman's paper "The Art of the Propagator"[1] yet, but I fully intend to now having watched the talk. [1]: https://dspace.mit.edu/bitstream/handle/1721.1/44215/MIT-CSAIL-TR-2009-002.pdf https://dspace.mit.edu/bitstream/handle/1721.1/44215/MIT-CSA...
- mgaunard 3y ago
  Few of the things we care about need to be fast, and fewer of the things that need to be fast are held up by number crunching performance. There are already a plethora of languages designed for numerical computing. Just use that if that's what you're doing.
namelosw 3y ago
GJS and SICP inspired me to love programming - before that, I only knew the rigid and cold C++ "Pyramid". Lisp is more dynamic and "organic" to me. (And I like his Babylonian wizard costume more than a Harry Potter one!) The past decades were dull for me in terms of programming languages, compared to the era of Scheme and Prolog. But the computing industry advanced steadily: multi-core programming, map-reduce clusters, CUDA, etc. And these led to these crazy generative neural networks we see today. I recall GJS gave several talks on this "We Don't Know How to Compute". I can't find the source and I skimmed this video and couldn't find it, but I remember he talked about how Gecko (or Lizard I'm not sure) could be transplanted with extra arms and still function. It strikes me as how much we don't know about computing, if we view biology and other things as computation. Now, with the AI hype, these frontiers seem open again to exploration. For example, you can give AI agents goals and tools and let them act on their own, it's still clunky but it works and it's improving fast every single day. It's just that we haven't figure out the patterns, implications and best practices yet. Exciting times!
jFriedensreich 3y ago
could someone explain why people laugh when he sais he went by the lab that said "laboratory for experimental histology"?
- messe 3y ago
  I know the transcript says "experimental histology", but I think he actually says was "experimental epistemology", which is a branch of philosophy dealing with the nature of knowledge—hence the laugh at "experimental".
  - messe 3y ago
    I reached out to find out, and received this reply: > It was indeed "experimental epistemology," > GJS
  - jFriedensreich 3y ago
    thanks that is also the only thing that makes sense, i just could not hear it after being primed by the transcript
    - messe 3y ago
      Technically "histology"—the microscopic study of tissues—also makes sense in the context of the lab (studying neurons), but it would be far less funny of a joke.
lbriner 3y ago
I find it funny that in the age of "User experience" my experience of most software is terrible. I don't use all the programs out there but apart from games, which often need to perform well to sell in the first place I have constant problems with Windows, MacOS, MS Office, most web sites, Visual Studio, browsers sometimes, virus scanners, network equipment, fast food touchscreens, Android, tablets etc. it's like we have rushed to quickly to demonstrate our super-powers at the expense of stuff that just works. I remember when desktop apps were written in desktop frameworks in the days when your PC was a Pentium 90 with 64MB of RAM and they opened quickly, they responded immediately to commands and they felt native. Despite the fact that these apps today do very little of value differently than 30 years ago bu the hardware is 1000s of times more powerful, they seem to make untold "metrics" network connections, they have no snap, they feel like massively incorrect abstractions over the hardware, they create weird user-experience and the people who sell them often don't know how to support them. I feel we have lost our way as an Industry being seduced by what the spoiled rich companies do and applying it to our own little kingdoms instead of doing what we need to do well. We have created false demons to slay and have ended up with such a mess its actually a little embarrassing. I guess while we lack any recognised/required industry qualifications, like Law or Medicine, we are just all doing our own things for our own reasons and unreasonably expecting everything to keep getting better. I mean, have you used Azure Devops? Who decided that it was reasonable to make a basic web app into a front-end monster that makes 100s of ajax calls?
- marcosdumay 3y ago
  > when your PC was a Pentium 90 with 64MB of RAM and they opened quickly, they responded immediately to commands and they felt native Well, no, I don't remember that. I remember them locking all the time (up to the point where moving the mouse made them slower) and taking ages to load. I do agree that it's absurd that the applications that do the same tasks today still take about as long to load and lock about as much, even though computers are 1000s of times more powerful. But they were never fast.
  - Jtsummers 3y ago
    I remember using Word on an early Pentium and being able to type faster than it could keep up with, at a whopping 30wpm at the time. I'd type a paragraph or two, and wait. To make it responsive it had to be the only thing open. And if I could type at my current speed on that thing (~100-120wpm, keyboard dependent) I doubt even that would be enough. We just expected and did less (at once) with computers at the time. If I close everything on my computer today and only run one application, it's pretty damned responsive (unless it's MS Teams).
dgb23 3y ago
IMO the three most important reasons we use computers are: - correctness - performance - automation (We basically still suck at all three.) As I understood it, the talk mainly addresses the third point. We write inflexible concretions that require a buttload of work, maintenance and resources to achieve mediocre things. Sussman is pushing programming and software design into directions that break away with this limitation, often through powerful general principles and techniques. The book "Software Design and Flexibility" explores this and provides a variety of examples and advice.
squirrel 3y ago
Isn't Sussman's idea similar to the tuple spaces of languages like Linda? https://en.wikipedia.org/wiki/Tuple_space https://en.wikipedia.org/wiki/Tuple_space
agentultra 3y ago
This is such an interesting talk! However I think the take on EWD's contributions to computer science could use a different take. On maintainability and extension of systems it is possible to achieve a system like Sussman's propagators while still maintaining proofs of correctness. It's true that overly specific proofs that don't work on the more general principles are harder to maintain in step with a changing program; however the trade-off is that you don't know if the change you made (to the highly dynamic system) will have the desired effect (or at least, you can't prove it doesn't). For a small circuit diagram it might be simple enough that there are obviously no errors. However the prevailing approach to software development is to make software so complex there are no obvious errors. Biological systems seem to get around this by encoding a lot of redundant information, preserving a ton of errors, and it takes millions of years to sort them out. I think though that in some cases it does make sense to trade off some correctness when prototyping and experimenting with ideas. However, in my limited experience, taking that idea to the extreme can lead to systems that only the original author can understand. Mathematics may be impressionistic but the formalization of mathematics, though tedious and pedantic, isn't! It wouldn't be until a few years later when systems like Lean are shared to the wider world (and Coq continues to improve) that show how maintaining more general proof-level engineering can be done on a larger scale without sacrificing flexibility. Sussman is legendary. Loved this talk!

126 points by ducktective 3y ago

This is a video of a talk from Gerald Sussman in 2011 at the Strange Loop conference. Abstract below. Though we have been building and programming computing machines for about 60 years and have learned a great deal about composition and abstraction, we have just begun to scratch the surface. A mammalian neuron takes about ten milliseconds to respond to a stimulus. A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it. The human genome -- the information required to build a human from a single, undifferentiated eukariotic cell -- is about 1GB. The instructions to build a mammal are written in very dense code, and the program is extremely flexible. Only small patches to the human genome are required to build a cow or a dog rather than a human. Bigger patches result in a frog or a snake. We don't have any idea how to make a description of such a complex machine that is both dense and flexible. New design principles and new linguistic support are needed. I will address this issue and show some ideas that can perhaps get us to the next phase of engineering design. Gerald Sussman Massachusetts Institute of Technology

danybittel 3y ago
And yet the computer is so much better in many areas than a human. It so weird to compare the two. Why would you even want that? We have already plenty of brains on earth. Who sais, that once a computer works like a brain.. it doesn't also come with it's downsides? There are lot's of problems in compute.. making it more brain like, more linguistic isn't going to solve that. I believe the whole notion of "a computer as an assistant", is doing more damage than good.
- moffkalast 3y ago
  Well an assistant is traditionally the one who eventually replaces the master, so that notion is not conceptually wrong. Do people still think there will be any humans left alive in a few hundred years? We've gone as far as we could as biological constructs, switching to a more adaptable architecture is just the next step forward. Our bodies are overfitted for living on this one specific planet which is greatly limiting the real goal of life: to expand to new areas and replicate.
  - kweingar 3y ago
    > Do people still think there will be any humans left alive in a few hundred years? I will go on the record and predict there will be some human beings alive in a few hundred years.
    - moffkalast 3y ago
      Alright there will always be some that stubbornly cling on, but I would bet that most entities that could be considered intelligent beings won't be human by that point, just out of practicality.
      - imtringued 3y ago
        Stupid flesh and blood humans with their finite needs! Let's build cyborgs with infinite information processing capability so their needs can never be saturated!
    - TeMPOraL 3y ago
      Question is, would they be alive by choice, and would they want to continue being alive?
      - kweingar 3y ago
        Are you and I alive by choice? I don’t know what this means. I will go further and boldly predict that some of these future people will like living.
        TeMPOraL 3y ago
        Ok, I'll spell out what I meant. One of the possible scenarios where, centuries after GAI was created, there would still be humans (recognizable to us as such) alive, is the same as the reason why horses and pigs and cows are alive today - because they're bred as inputs to some industrial process.
- pfarrell 3y ago
  My takeaway is that Sussman is looking at his field after a lifetime of contribution. He's looking for signposts on where we can go next. He's not valuing one over the other, but looking at the brain and saying, "Here's something that exists that's operating on problems in a way we can't begin to approach using current software/hardware techniques". He sees some hints available to us in the noise reduction things he talks about at the end. In that way, the talk is more him challenging the audience to question our approaches and implied constraints rather than him trying to present us with an answer. It's one of my favorite videos (the way too loud HI at the beginning, notwithstanding).
unstuck3958 3y ago
> So the computational depth of this behavior is only a few tens of steps. One of the primary "criticisms" of pre-LLM machine learning was that it is inefficient compared to, say, a human, because it requires a plethora of examples to classify something just to match the performance of a human who has only seen a few examples. That argument no longer holds the same value since the discovery of few-shot learning features of LLMs (though they still have a long way to go). Which makes me think: maybe the reason behind human brain's apparent "efficiency" is the complexity that hides beneath the billions of neurons we have. Disclaimer: I am an amateur and probably have only a litte idea of what I'm talking about.
- imtringued 3y ago
  Sorry but there is no need for a whole brain to outperform ANNs in learning speed. https://bigthink.com/neuropsych/brain-cells-chip-play-pong/ https://bigthink.com/neuropsych/brain-cells-chip-play-pong/
deleted 3y ago
[deleted]
MichaelZuo 3y ago
F1 drivers can instinctively respond to a minor bump on a curve and make a decision in probably ~100 milliseconds, so the lower bound is just 10 computational steps. A more complex decision such as swerving to avoid a wheel that just came off the car in front, probably takes around ~500 milliseconds, or 50 computational steps. Considering that modern computers need billions of computational steps to start a calculator app and draw it on the desktop, we have a long way to go. Though there has been regression on this point, a 1990 NeXTStation Color could do the same with ~50 million cycles on a 1120×832 display. (on 12 MB of RAM and 1.5 MB of VRAM.)
- TeMPOraL 3y ago
  I think the trick is, our computational steps are massively parallel. Take the visual system. By the time you notice - consciously - the pothole you're about to drive into, it's already processed into high-level concepts. Our visual senses aren't camera feeds - the image we are aware of isn't raw input from the eyes, but rather it's generated out of some combination of raw inputs, low-level hacks to hide limited FOV and discontinuities caused by saccades, and higher-level interpretations based on how we feel, what seems to fit the context based on our prior experiences, etc. I think it makes sense to assume that this process has many extra off-ramps that can trigger high-level reactions while bypassing conscious awareness. I.e. imagine that half-way through the post-processing stack, the relevant parts of the brain are somewhat convinced you're looking at a fast-moving object about to hit you in the face. There's a point at which it makes sense for the brain to make your body start a dodging move, even if it may be a misprediction, rather than waste a few dozen milliseconds to make sure, at the risk of getting a ball (or a rock, or a fist) up your nose.
  - marcosdumay 3y ago
    > I think the trick is, our computational steps are massively parallel. Yes, one trick is that. Our computers have to serialize everything. We don't know how to build or program massively parallel computers with distributed memory. But the GP already touched on that point.
    - 3y ago
lostmsu 3y ago
> A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it. IMHO, that makes no sense. Assuming we have a really large state machine with all many possible situations in a state of driving on a straight highway with a few vehicles with more or less constant relative velocity, determining if we need to make a turn is definitely doable in a few computation steps.
- leroy-is-here 3y ago
  We attempted the symbolic route of AI programming and it didn’t work because it is explosive in computation time and exhaustive to model in code. We can’t program our AI “models” now because they are based on chains of probabilities. Therefore they can run in real time. They don’t have any real logic that we specifically programmed, only what the model was able to infer and then encode. Perhaps there should be a marriage between these two attempts of AI.
  - marcosdumay 3y ago
    > it is explosive in computation time Hum... The numerical route for AI necessarily has the same complexity in computation time. Those two have equivalent semantics. If you can get a result with one, you can get the same result with the other, it can just differ on how easy it is to program. By the way, symbolic AI can learn too, you don't need to model things in code.
    - leroy-is-here 3y ago
      I was talking about attempts such as expert systems. We attempted to symbolically encode things as sums of their parts in plain terms, e.g. a bicycle is two wheels, an axle, a chain, some peddles, etc. but in more detail than that. The problem is the level of detail required for general knowledge: firstly it is hard to codify, secondly as it grows computation becomes exponentially more expensive. We as humans have the intelligence to simply see for fractions of a second and then just know — we can know the answers to math problems and how a person is feeling. The symbolic AI programming was an attempt to codify what symbols we are seeing but they are so diverse and nuanced it became apparent this was an ineffective scientific route.
karmakaze 3y ago
Not really a fair comparison--250 ms is reaction time. A deliberate conscious response would be much longer. "Reaction times can reflect habits rather than computations"[0]. [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5582865/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5582865/
bionhoward 3y ago
Seems like fractals are the difference maker here because if you don’t use a fractal you have to program the whole thing whereas if you use fractals (embryology) then your code can be massively recursive. Alternative splicing and epigenetics (holy shit, I can’t believe the iPhone autocorrect doesn’t know the word “epigenetics” in ‘23) also multiply the flexibility of the genome. Also, phosphorylation: proteins are not Boolean switches, they have arbitrary numbers of activation states. Ternary logic is already way more dense than Boolean so when you go even further like into 6-ary, 9-ary logics you’re a lot more free to program stuff. Also the network motifs are a lot simpler than first appear. The metabolic map looks like spaghetti on the wall but each point is composed of reusable blocks like feedforward and feedback etc (see Uri Alon’s book and “Wetware” book and Synthetic Biology: A primer for more detail) I’m biased as a molecular biologist but genetics and systems biology are the most incredible thing. Natural nanotech
- MisterTea 3y ago
  > Natural nanotech When I was a kid in the 80's there were a number of gross/gore/horror trading cards for kids - Garbage Pail Kids being the prime example. One of the sets I had was set in some far flung future, I can barely remember the series but part of the theme were living, biological machines. The only card I can clearly remember featured a flying prisoner transport where the prisoners were seated in the ships cavernous stomach and "if upset during transport could accidentally digest the prisoners." (Honorable mention to HL2 featuring "synthtech" - the drop ship, strider and gun ships were living biological machines.) This stuck in my head as the idea of designing machines which grow like plants or raised like animals was an incredibly amazing and interesting concept. But as I got older and educated the idea them seemed as far flung as the cards themselves. Though reading you post makes me think, maybe they aren't so far flung after all. Though I still fell that a creature which grows steel or titanium skeletons is still quite firmly in the realm of sci-fi.
alcover 3y ago
> The human genome [...] is about 1GB I don't get how this is impressive or relevant on its own. Sure, it sounds small in face of the final result : a full-grown human. But this 1GB program is born out of and expressed into the full OS that is the physical world.

CraigJPerry 3y ago

We do keep looking under the proverbial lamppost for the missing keys[1] when it comes to computation. Compilers for the most popular languages are pretty amazing these days. Yet it’s still on the developer to make the most of them. One aspect might be performance: - structure your problem in such a way (carefully manage dependency relationships, eliminate branches, etc). that the compiler can choose to emit vector instructions - manage memory access patterns (e.g. prefer columnar and SoA access patterns over cache unfriendly pointer chasing structures) - understand how to maximally exploit cache hierarchies for your problem (e.g. threading to exploit the other L1 caches rather than relying on slower L3 etc.) Another aspect might be data security rules, e.g. tagging and labelling with the ability to enforce rules for the developer. But there are literally hundreds of aspects to deal with. Our languages today look at the problem space through a toilet roll tube with the other eye closed. The future of computation is guaranteed to be rich, vibrant and exciting. [1] https://en.m.wikipedia.org/wiki/Streetlight_effect https://en.m.wikipedia.org/wiki/Streetlight_effect

messe 3y ago
I think every suggestion you have there would fall under that very same lamppost: a linear model of computation going only one direction. Sussman details in this talk a different model he calls propagators that deal with information flow back and forth, allowing for some interesting ways of working with "fuzzy" and incomplete data, while still allowing for cheap backtracking if a contradiction arises. I haven't read the Radul & Sussman's paper "The Art of the Propagator"[1] yet, but I fully intend to now having watched the talk. [1]: https://dspace.mit.edu/bitstream/handle/1721.1/44215/MIT-CSAIL-TR-2009-002.pdf https://dspace.mit.edu/bitstream/handle/1721.1/44215/MIT-CSA...
mgaunard 3y ago
Few of the things we care about need to be fast, and fewer of the things that need to be fast are held up by number crunching performance. There are already a plethora of languages designed for numerical computing. Just use that if that's what you're doing.

namelosw 3y ago

GJS and SICP inspired me to love programming - before that, I only knew the rigid and cold C++ "Pyramid". Lisp is more dynamic and "organic" to me. (And I like his Babylonian wizard costume more than a Harry Potter one!) The past decades were dull for me in terms of programming languages, compared to the era of Scheme and Prolog. But the computing industry advanced steadily: multi-core programming, map-reduce clusters, CUDA, etc. And these led to these crazy generative neural networks we see today. I recall GJS gave several talks on this "We Don't Know How to Compute". I can't find the source and I skimmed this video and couldn't find it, but I remember he talked about how Gecko (or Lizard I'm not sure) could be transplanted with extra arms and still function. It strikes me as how much we don't know about computing, if we view biology and other things as computation. Now, with the AI hype, these frontiers seem open again to exploration. For example, you can give AI agents goals and tools and let them act on their own, it's still clunky but it works and it's improving fast every single day. It's just that we haven't figure out the patterns, implications and best practices yet. Exciting times!

jFriedensreich 3y ago

could someone explain why people laugh when he sais he went by the lab that said "laboratory for experimental histology"?

messe 3y ago
I know the transcript says "experimental histology", but I think he actually says was "experimental epistemology", which is a branch of philosophy dealing with the nature of knowledge—hence the laugh at "experimental".
- messe 3y ago
  I reached out to find out, and received this reply: > It was indeed "experimental epistemology," > GJS
- jFriedensreich 3y ago
  thanks that is also the only thing that makes sense, i just could not hear it after being primed by the transcript
  - messe 3y ago
    Technically "histology"—the microscopic study of tissues—also makes sense in the context of the lab (studying neurons), but it would be far less funny of a joke.

lbriner 3y ago

I find it funny that in the age of "User experience" my experience of most software is terrible. I don't use all the programs out there but apart from games, which often need to perform well to sell in the first place I have constant problems with Windows, MacOS, MS Office, most web sites, Visual Studio, browsers sometimes, virus scanners, network equipment, fast food touchscreens, Android, tablets etc. it's like we have rushed to quickly to demonstrate our super-powers at the expense of stuff that just works. I remember when desktop apps were written in desktop frameworks in the days when your PC was a Pentium 90 with 64MB of RAM and they opened quickly, they responded immediately to commands and they felt native. Despite the fact that these apps today do very little of value differently than 30 years ago bu the hardware is 1000s of times more powerful, they seem to make untold "metrics" network connections, they have no snap, they feel like massively incorrect abstractions over the hardware, they create weird user-experience and the people who sell them often don't know how to support them. I feel we have lost our way as an Industry being seduced by what the spoiled rich companies do and applying it to our own little kingdoms instead of doing what we need to do well. We have created false demons to slay and have ended up with such a mess its actually a little embarrassing. I guess while we lack any recognised/required industry qualifications, like Law or Medicine, we are just all doing our own things for our own reasons and unreasonably expecting everything to keep getting better. I mean, have you used Azure Devops? Who decided that it was reasonable to make a basic web app into a front-end monster that makes 100s of ajax calls?

marcosdumay 3y ago
> when your PC was a Pentium 90 with 64MB of RAM and they opened quickly, they responded immediately to commands and they felt native Well, no, I don't remember that. I remember them locking all the time (up to the point where moving the mouse made them slower) and taking ages to load. I do agree that it's absurd that the applications that do the same tasks today still take about as long to load and lock about as much, even though computers are 1000s of times more powerful. But they were never fast.
- Jtsummers 3y ago
  I remember using Word on an early Pentium and being able to type faster than it could keep up with, at a whopping 30wpm at the time. I'd type a paragraph or two, and wait. To make it responsive it had to be the only thing open. And if I could type at my current speed on that thing (~100-120wpm, keyboard dependent) I doubt even that would be enough. We just expected and did less (at once) with computers at the time. If I close everything on my computer today and only run one application, it's pretty damned responsive (unless it's MS Teams).

dgb23 3y ago

IMO the three most important reasons we use computers are: - correctness - performance - automation (We basically still suck at all three.) As I understood it, the talk mainly addresses the third point. We write inflexible concretions that require a buttload of work, maintenance and resources to achieve mediocre things. Sussman is pushing programming and software design into directions that break away with this limitation, often through powerful general principles and techniques. The book "Software Design and Flexibility" explores this and provides a variety of examples and advice.

squirrel 3y ago

Isn't Sussman's idea similar to the tuple spaces of languages like Linda? https://en.wikipedia.org/wiki/Tuple_space https://en.wikipedia.org/wiki/Tuple_space

agentultra 3y ago

This is such an interesting talk! However I think the take on EWD's contributions to computer science could use a different take. On maintainability and extension of systems it is possible to achieve a system like Sussman's propagators while still maintaining proofs of correctness. It's true that overly specific proofs that don't work on the more general principles are harder to maintain in step with a changing program; however the trade-off is that you don't know if the change you made (to the highly dynamic system) will have the desired effect (or at least, you can't prove it doesn't). For a small circuit diagram it might be simple enough that there are obviously no errors. However the prevailing approach to software development is to make software so complex there are no obvious errors. Biological systems seem to get around this by encoding a lot of redundant information, preserving a ton of errors, and it takes millions of years to sort them out. I think though that in some cases it does make sense to trade off some correctness when prototyping and experimenting with ideas. However, in my limited experience, taking that idea to the extreme can lead to systems that only the original author can understand. Mathematics may be impressionistic but the formalization of mathematics, though tedious and pedantic, isn't! It wouldn't be until a few years later when systems like Lean are shared to the wider world (and Coq continues to improve) that show how maintaining more general proof-level engineering can be done on a larger scale without sacrificing flexibility. Sussman is legendary. Loved this talk!