7 ms·

Frank Rosenblatt

70 points by Thevet 4y ago

Yajirobe 4y ago
How many these so-called ‘fathers of AI/deep learning’ are there??
- jacquesm 4y ago
  Turing and Hinton rate a mention, as do Minsky and Newell. There are probably lots of others, especially given the current explosion of effort in the field.
- ly3xqhl8g9 4y ago
  Seppo Linnainmaa, first publisher of "reverse mode of automatic differentiation to efficiently compute the derivative of a differentiable composite function that can be represented as a graph, by recursively applying the chain rule to the building blocks of the function" [1] aka backpropagation. Gerardi Ostrowski was first apparently but didn't publish. Paul Werbos, "first described the process of training artificial neural networks through backpropagation of errors" [2]. [1] https://en.wikipedia.org/wiki/Seppo_Linnainmaa https://en.wikipedia.org/wiki/Seppo_Linnainmaa [2] https://en.wikipedia.org/wiki/Paul_Werbos https://en.wikipedia.org/wiki/Paul_Werbos
  - mlajtos 4y ago
    Hi Jürgen!
- seydor 4y ago
  standing on the shouderS of giantS S for plural
dang 4y ago
Related: Frank Rosenblatt's perceptron paved the way for AI 60 years too soon (2019) - https://news.ycombinator.com/item?id=30236123 https://news.ycombinator.com/item?id=30236123 - Feb 2022 (57 comments)
- mikewarot 4y ago
  He wasn't too soon, he was sabotaged by Minsky and Papert[1] who were competitors for funding. His premature death left his ideas in limbo for far too long. [1] https://en.wikipedia.org/wiki/Perceptrons_(book) https://en.wikipedia.org/wiki/Perceptrons_(book)
  - deleted 4y ago
    [deleted]
  - hungryforcodes 4y ago
    This so much !
  - YeGoblynQueenne 4y ago
    Who says this? The wikipedia article you link is very short on citations, but very long on vague invocations of "some critics", without ever citing those "some critics". Those are all the hallmarks of a rumour of the kind that spread on the internet, based on nothing else than half-digested information and a thirst for controversies. It should be noted that Minsky, himself, was a connectionist who worked on neural networks, as well as other approaches to AI. For example: https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinforcement_calculator https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinf... I don't think most people who repeat those claims about Perceptrons have even read the book and really know what it says, other than from second- or third-hand sources (i.e. something someone once posted on twitter). The entire book, in pdf format, can be downloaded from here: https://ieeexplore.ieee.org/book/8076704 https://ieeexplore.ieee.org/book/8076704 It's a whole book and not a blog post so it takes some reading. Alternatively, there is the introduction to the 1988 reissue with a short foreword by Leon Botu, focusing on the history of the book and its repercussions here: https://leon.bottou.org/publications/pdf/perceptrons-2017.pdf https://leon.bottou.org/publications/pdf/perceptrons-2017.pd... And a review of the book with more technical information here: https://www.sciencedirect.com/science/article/pii/S0019995870904092?ref=cra_js_challenge&fr=RR-1 https://www.sciencedirect.com/science/article/pii/S001999587...
    - deleted 4y ago
com2kid 4y ago
> Rat brain experiments One of the more off the wall sci-fi ideas I saw was a pulp sci-fi novel that had dog brains being used for self driving cars. The idea stuck with me that maybe sometimes we should give up on optimizing nature and just go along with nature. Of course neural networks are very much inspired by nature, but since Rosenblatt's time, science has learned that brains are even more complex than previously imagined (and no one ever though brains were simple!)
- m463 4y ago
  I think we ARE nature, organically adapting whatever tool or advantage we find and putting it to use.
  - stareatgoats 4y ago
    > I think we ARE nature Agree. But will the future autonomous and from humans totally independent AI with self-encoded self-preservation and agency be nature too? I'm inclined to say yes, if we are able to expand our definition of nature just a bit to include non-biological life forms (of which there might be many already, we just haven't noticed).
- ekkodal 4y ago
  Do you remember the name of the novel? Sounds like a fun read :)
  - jacquesm 4y ago
    I remember reading that and took a 1/2 hour to find it but so far no luck. It may have been a short story. Searching for it is hard because somehow 'Sally' by Asimov keeps popping up but that has nothing to do with it.
  - jumhyn 4y ago
    It’s not quite self-driving cars but Snow Crash has a very similar plot detail.
  - pmoriarty 4y ago
bendbro 4y ago
> transfer of unlearned behavior with brain extracts I'm surprised that it works.
- ly3xqhl8g9 4y ago
  Moths remember what they learned as caterpillars [1]. Memory transfer via RNA injections has also been achieved in snails and flatworms [2] [3]. Memory transfer and memory markets will probably be a thing sometime by 2100, perhaps even synthetic memory markets, let stable diffusion reimagine your past. [1] "M. sexta larvae can learn to associate odor cues with an aversive stimulus, and this memory persists undiminished across two larval molts, as well as into adulthood. The behavior represents true associative learning, not chemical legacy, and, as far as we know, provides the first definitive demonstration that associative memory survives metamorphosis in Lepidoptera." https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0001736 https://journals.plos.org/plosone/article?id=10.1371/journal... [2] "Glanzman said one of McConnell’s students, Al Jacobson, demonstrated the transfer of memories between flatworms via RNA injections, coincidentally while an assistant professor at UCLA. The work was published in Nature in 1966 but Jacobson never received tenure, perhaps because of doubts about his findings. The experiment was, however, replicated in rats shortly afterward." https://www.scientificamerican.com/article/memory-transferred-between-snails-challenging-standard-theory-of-how-the-brain-remembers https://www.scientificamerican.com/article/memory-transferre... Jacobson's 1966 article, https://pubmed.ncbi.nlm.nih.gov/5921188 https://pubmed.ncbi.nlm.nih.gov/5921188 [3] Memory in the flesh, https://www.theverge.com/2015/3/18/8225321/memory-research-flatworm-cannibalism-james-mcconnell-michael-levin https://www.theverge.com/2015/3/18/8225321/memory-research-f...
  - dekhn 4y ago
    Generally memory transfer by mRNA is not considered to be a well-established phenomenon.
    - ly3xqhl8g9 4y ago
      The snails paper [1] does not mention mRNA, or the RNA type at all. But sure, there are lots to learn and there are more questions than answers. [1] RNA from Trained Aplysia Can Induce an Epigenetic Engram for Long-Term Sensitization in Untrained Aplysia, https://www.eneuro.org/content/5/3/ENEURO.0038-18.2018 https://www.eneuro.org/content/5/3/ENEURO.0038-18.2018
PeterStuer 4y ago
I'm the lucky owner of a Spartan 1962 edition of Frank Rosenblatt's Principles of Neurodynamics (there was a time University Libraries sold out their inventories for peanuts, and you'd find these pearls amongst the trash). I still chuckle though at the anecdote in "The Brain Makers" book about the Blatt being used around the AI-Lab as a unit of body odor.
_a_a_a_ 4y ago
> when in 1969 Marvin Minsky and Seymour Papert published the book “Perceptrons” with a mathematical proof about the limitations of two-layer feed-forward perceptrons ... The book's only proven result — that linear functions cannot model non-linear ones — was trivial A question: IIRC Minsky & Papert proved XOR couldn't be done in that limited perceptron, but that implies (given "that linear functions cannot model non-linear ones") that XOR is non-linear. Is this right, if so how is it not? What does non-linear mean here, actually?
- AndrewOMartin 4y ago
  Draw a 2d graph, on the vertical axis put A-is-false at the bottom and A-is-true at the top. On the horizontal axis put B-is-false on the left and B-is-true on the right. Now 'fill in' this graph with an X if the combination of A and B is True according to the XOR function, and a O otherwise. You'll find you get similar symbols in opposite corners and can't separate the two groups of symbols by drawing a single straight line. Now try the same for OR, NOT, AND, NOR and NAND.
- AndrewOMartin 4y ago
  I think it's worth pointing out that "Perceptions" also says it's trivially obvious that any network with a hidden layer could be used to approximate any curve because then you're basically manually tweaking an arbitrarily complex function. This was significant at the time because no one had published a training algorithm for networks with hidden layers (e.g. backpropogation)

Frank Rosenblatt

70 points by Thevet 4y ago

Yajirobe 4y ago

How many these so-called ‘fathers of AI/deep learning’ are there??

jacquesm 4y ago
Turing and Hinton rate a mention, as do Minsky and Newell. There are probably lots of others, especially given the current explosion of effort in the field.
ly3xqhl8g9 4y ago
Seppo Linnainmaa, first publisher of "reverse mode of automatic differentiation to efficiently compute the derivative of a differentiable composite function that can be represented as a graph, by recursively applying the chain rule to the building blocks of the function" [1] aka backpropagation. Gerardi Ostrowski was first apparently but didn't publish. Paul Werbos, "first described the process of training artificial neural networks through backpropagation of errors" [2]. [1] https://en.wikipedia.org/wiki/Seppo_Linnainmaa https://en.wikipedia.org/wiki/Seppo_Linnainmaa [2] https://en.wikipedia.org/wiki/Paul_Werbos https://en.wikipedia.org/wiki/Paul_Werbos
- mlajtos 4y ago
  Hi Jürgen!
seydor 4y ago
standing on the shouderS of giantS S for plural

dang 4y ago

Related: Frank Rosenblatt's perceptron paved the way for AI 60 years too soon (2019) - https://news.ycombinator.com/item?id=30236123 https://news.ycombinator.com/item?id=30236123 - Feb 2022 (57 comments)

mikewarot 4y ago
He wasn't too soon, he was sabotaged by Minsky and Papert[1] who were competitors for funding. His premature death left his ideas in limbo for far too long. [1] https://en.wikipedia.org/wiki/Perceptrons_(book) https://en.wikipedia.org/wiki/Perceptrons_(book)
- deleted 4y ago
  [deleted]
- hungryforcodes 4y ago
  This so much !
- YeGoblynQueenne 4y ago
  Who says this? The wikipedia article you link is very short on citations, but very long on vague invocations of "some critics", without ever citing those "some critics". Those are all the hallmarks of a rumour of the kind that spread on the internet, based on nothing else than half-digested information and a thirst for controversies. It should be noted that Minsky, himself, was a connectionist who worked on neural networks, as well as other approaches to AI. For example: https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinforcement_calculator https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinf... I don't think most people who repeat those claims about Perceptrons have even read the book and really know what it says, other than from second- or third-hand sources (i.e. something someone once posted on twitter). The entire book, in pdf format, can be downloaded from here: https://ieeexplore.ieee.org/book/8076704 https://ieeexplore.ieee.org/book/8076704 It's a whole book and not a blog post so it takes some reading. Alternatively, there is the introduction to the 1988 reissue with a short foreword by Leon Botu, focusing on the history of the book and its repercussions here: https://leon.bottou.org/publications/pdf/perceptrons-2017.pdf https://leon.bottou.org/publications/pdf/perceptrons-2017.pd... And a review of the book with more technical information here: https://www.sciencedirect.com/science/article/pii/S0019995870904092?ref=cra_js_challenge&fr=RR-1 https://www.sciencedirect.com/science/article/pii/S001999587...
  - deleted 4y ago

com2kid 4y ago

> Rat brain experiments One of the more off the wall sci-fi ideas I saw was a pulp sci-fi novel that had dog brains being used for self driving cars. The idea stuck with me that maybe sometimes we should give up on optimizing nature and just go along with nature. Of course neural networks are very much inspired by nature, but since Rosenblatt's time, science has learned that brains are even more complex than previously imagined (and no one ever though brains were simple!)

m463 4y ago
I think we ARE nature, organically adapting whatever tool or advantage we find and putting it to use.
- stareatgoats 4y ago
  > I think we ARE nature Agree. But will the future autonomous and from humans totally independent AI with self-encoded self-preservation and agency be nature too? I'm inclined to say yes, if we are able to expand our definition of nature just a bit to include non-biological life forms (of which there might be many already, we just haven't noticed).
ekkodal 4y ago
Do you remember the name of the novel? Sounds like a fun read :)
- jacquesm 4y ago
  I remember reading that and took a 1/2 hour to find it but so far no luck. It may have been a short story. Searching for it is hard because somehow 'Sally' by Asimov keeps popping up but that has nothing to do with it.
- jumhyn 4y ago
  It’s not quite self-driving cars but Snow Crash has a very similar plot detail.
- pmoriarty 4y ago

bendbro 4y ago

> transfer of unlearned behavior with brain extracts I'm surprised that it works.

ly3xqhl8g9 4y ago
Moths remember what they learned as caterpillars [1]. Memory transfer via RNA injections has also been achieved in snails and flatworms [2] [3]. Memory transfer and memory markets will probably be a thing sometime by 2100, perhaps even synthetic memory markets, let stable diffusion reimagine your past. [1] "M. sexta larvae can learn to associate odor cues with an aversive stimulus, and this memory persists undiminished across two larval molts, as well as into adulthood. The behavior represents true associative learning, not chemical legacy, and, as far as we know, provides the first definitive demonstration that associative memory survives metamorphosis in Lepidoptera." https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0001736 https://journals.plos.org/plosone/article?id=10.1371/journal... [2] "Glanzman said one of McConnell’s students, Al Jacobson, demonstrated the transfer of memories between flatworms via RNA injections, coincidentally while an assistant professor at UCLA. The work was published in Nature in 1966 but Jacobson never received tenure, perhaps because of doubts about his findings. The experiment was, however, replicated in rats shortly afterward." https://www.scientificamerican.com/article/memory-transferred-between-snails-challenging-standard-theory-of-how-the-brain-remembers https://www.scientificamerican.com/article/memory-transferre... Jacobson's 1966 article, https://pubmed.ncbi.nlm.nih.gov/5921188 https://pubmed.ncbi.nlm.nih.gov/5921188 [3] Memory in the flesh, https://www.theverge.com/2015/3/18/8225321/memory-research-flatworm-cannibalism-james-mcconnell-michael-levin https://www.theverge.com/2015/3/18/8225321/memory-research-f...
- dekhn 4y ago
  Generally memory transfer by mRNA is not considered to be a well-established phenomenon.
  - ly3xqhl8g9 4y ago
    The snails paper [1] does not mention mRNA, or the RNA type at all. But sure, there are lots to learn and there are more questions than answers. [1] RNA from Trained Aplysia Can Induce an Epigenetic Engram for Long-Term Sensitization in Untrained Aplysia, https://www.eneuro.org/content/5/3/ENEURO.0038-18.2018 https://www.eneuro.org/content/5/3/ENEURO.0038-18.2018

PeterStuer 4y ago

I'm the lucky owner of a Spartan 1962 edition of Frank Rosenblatt's Principles of Neurodynamics (there was a time University Libraries sold out their inventories for peanuts, and you'd find these pearls amongst the trash). I still chuckle though at the anecdote in "The Brain Makers" book about the Blatt being used around the AI-Lab as a unit of body odor.

_a_a_a_ 4y ago

> when in 1969 Marvin Minsky and Seymour Papert published the book “Perceptrons” with a mathematical proof about the limitations of two-layer feed-forward perceptrons ... The book's only proven result — that linear functions cannot model non-linear ones — was trivial A question: IIRC Minsky & Papert proved XOR couldn't be done in that limited perceptron, but that implies (given "that linear functions cannot model non-linear ones") that XOR is non-linear. Is this right, if so how is it not? What does non-linear mean here, actually?

AndrewOMartin 4y ago
Draw a 2d graph, on the vertical axis put A-is-false at the bottom and A-is-true at the top. On the horizontal axis put B-is-false on the left and B-is-true on the right. Now 'fill in' this graph with an X if the combination of A and B is True according to the XOR function, and a O otherwise. You'll find you get similar symbols in opposite corners and can't separate the two groups of symbols by drawing a single straight line. Now try the same for OR, NOT, AND, NOR and NAND.
AndrewOMartin 4y ago
I think it's worth pointing out that "Perceptions" also says it's trivially obvious that any network with a hidden layer could be used to approximate any curve because then you're basically manually tweaking an arbitrarily complex function. This was significant at the time because no one had published a training algorithm for networks with hidden layers (e.g. backpropogation)