9 ms·

Three questions that keep me up at night

84 points by andrewnc 7y ago

thaumasiotes 7y ago
> If you don't know how to grasp an object, you can't bring food to your mouth. Counterpoint: hummingbirds. It might sound cheap, but discussions of the "basic necessities" of anything so often just overlook obvious counterexamples.
- ericjang 7y ago
  Hummingbirds actually proves my point here, on how life and death solve the symbol grounding problem. What is a "stable grasp"? An entire body of classical robotics literature is devoted to answering this problem. I work on robotic grasping research and defining what it means to grasp something successfully is a little tricky unless you have a broader context of what the grasping is for. Nature doesn't care whether hands exist or not - only that the animal can feed. That is why we see such a spectrum of appendages in the animal kingdom. Some are hand-like, others not at all, and some in between. In some sense, "hand" is a mental construct of human language.
RickJWagner 7y ago
1. What is your name? 2. What is your quest? 3. What is your favorite color? Watch out for that last one.
- noobly 7y ago
  >Watch out for that last one. This must be a reference I’m not getting?
  - umvi 7y ago
    Monty Python and the Holy Grail https://youtu.be/0D7hFHfLEyk https://youtu.be/0D7hFHfLEyk
  - core_dumped 7y ago
    Watch "Monty Python and The Holy Grail"
- pdonis 7y ago
  What is the capital of Assyria?
- CarlRJ 7y ago
  Blue! No, YELLLLLLOOoooooowwwwwwww!
- Reedx 7y ago
  1. Are you really the head of the Kwik-E-Mart? 2. Really?! 3. You? https://www.youtube.com/watch?v=FiDY2nXVLZ0 https://www.youtube.com/watch?v=FiDY2nXVLZ0
sudoaza 7y ago
Lame, only true question is will this universe die out and all information in it be lost or can it be maintained indefinitely/transfer information to the next universe iteration.
- pwdisswordfish2 7y ago
  > only true question you mean like, the last question?
  - empiricus 7y ago
    I guess, the only question that matters. Unfortunately it does not matter for us, we are too transient.
Gatsky 7y ago
Sometimes I wonder if we are the critters related to the second question. Of course the next question is how far away are we from meeting the simulation’s spec.
untog 7y ago
I know it’s just the moment we’re living in right now but as I read those questions I just shrug, they couldn’t feel less relevant. The ones on my mind are “how long will large populations be locked down?” “How many people are going to die?” and “how long until this all happens again?”
- nine_k 7y ago
  One could go even shorter term than that. But life is a long game.
- ceilingcorner 7y ago
  “The greats never sacrifice the important for the urgent. They handle the immediate problem and still make sure to secure the future.” The media is incentivized to distract you. Turn off your computer, read a book, and don’t forget to live.
  - deleted 7y ago
    [deleted]
  - diN0bot 7y ago
    no i think you misunderstand. those three questions the commenter posted aren’t about running from fear or anxiety — some would simply call “questions about how we collectively overcome massive and pressing challenges” more relevant, in the short and long term.
  - untog 7y ago
    What an absurd statement. This crisis matters. Social distancing matters. Lockdowns matter. “Stick your fingers in your ears” is awful advice.
    - m-p-3 7y ago
      But at some point, the medias are rehashing the same statements: Stay home, and follows good practices. As an individual, there's only so much you can do and there's no need to keep yourself distracted from other matters.
Red_Leaves_Flyy 7y ago
Tldr: >Q1: Can we imitate "thinking" from only observing behavior? >Q2: What is the computationally cheapest "organic building block" of an Artificial Life simulation that could lead to human-level AGI? >Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise from T-symmetric (microscopic) physical laws?
mannykannot 7y ago
> Q1: Can we imitate "thinking" from only observing behavior? If you believe that the human mind is the result of Darwinian evolution, then you implicitly believe that thinking can arise from interacting with the environment. That does not exactly answer the question, but if anything, it goes beyond it. On the other hand: "This could also be addressed via other high-data environments: 1) Observing trader orders on markets and seeing if we can recover the trader's deductive reasoning and beliefs about the future. See if we can observe rational thought (if not rational behavior). 2) Recovering intent and emotions and desire from social network activity." Before you could even begin to get anything from these streams of symbols, you would have to have some way to give them semantics - to understand what they signify to the actors. I suspect that these would be among the last things you would achieve, not a means to the end.
jvm___ 7y ago
Do we already posess the hardware required to run full AI? Deep fakes can be done on older hardware, we just didn't have the code. Does the same thing apply to AI?
Zenst 7y ago
Q1: Can we imitate "thinking" from only observing behavior? NO as you are measuring actions and not the process that instigated those actions. One example - chess - you see the move, but the thinking behind that move may elude. More so a bad move, as their thinking was that it is a good move when it is not. [EDIT format/spelling]
ethn 7y ago
Q1) By the mere copying of behaviors, no. It is clear that there are at least the a priori intuitions of space, time, and causality necessary for humans to obtain any knowledge. It is within that a priori framework where concepts and then decisions are derived from. Q2) This would require the knowledge of what is sufficient & necessary for AGI. Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this question. Read page 33 of Weiner's Cybernetics. The real insight, where in what I will now explain in brevity is at the expense of the insight's resolution, is that in the nature of microscopic phenomena there are an exorbitant quantity of fickle components ("equal particles") whose positions, accelerations, velocities are all impossible to record because of that sheer quantity by the nature of this sphere of study. More so, there are then at least second-order coupling effects between all these fickle components due to their constant interaction. As a resolve, we have developed terms of statistical understanding. Weiner cites "cloud, temperature, turbulence, etc." all being concepts which are specifically defined in a statistical context. Where as in the macroscopic sphere there are far fewer particles much further apart from each other with near certain initial masses, positions, and velocities. The result? In the microscopic systems, because their properties are precisely defined statistically the distribution which predicts the future nor past for a given system isn't unique. That is, the future state (predicted by the distribution provided by some initial system) could have also been predicted by other distributions describing many other initial systems unless by "some miracle" there is a "tightly defined statistical range". So are microscopic physics condemned to this paradox? Not necessarily, there may be some way to construct scientific objects from systems phenomena which don't require a statistical conception because they can be precisely defined from a microscopic systems point of view temporally & spatially. However, this would require a revolution in theory and likely mathematics
- lidHanteyk 7y ago
  I dislike the cybernetic approach because I find that it first commits to epistemic helplessness, saying that we know nothing of particles. Perhaps that was reasonable when QM was so young, but these days we can use QM for a better answer. This isn't complete, but it helps a lot. First, note that CPT-symmetry can be extended up to the macroscopic level. Einstein's thought experiments about time involve macroscopic entanglement. For example, given that a red ball and a blue ball are each in their own boxes, and one possesses one box, then the box is entangled to have either a red or blue ball inside. We can use linear logic to reason about these situations; one holds the multiplicative disjunction of red and blue. Second, note that in order to make MWI's numbers work, we need not just branching of possibilities, but also coalescing. Imagine that, in empty space, we suddenly have a single virtual-particle pair; the diagram would look like () with time vertical. At the beginning, we have one universe; during the split, two universes, one for each chirality of the pair; afterwards, one universe. When we split, the probabilities of each possible universe decrease. Now, imagine doing this with more pairs, and verify that the Binomial Theorem manifests. This gives the same rise to macrostate-like entropy as Boltzmann's approach to thermodynamics, but only using QM particles and QM logic; a QM state is likelier when more of the many worlds in MWI would support its existence. Putting these together, we may conclude that Loschmidt's paradox is an incomplete part of a bigger paradoxical complaint: We can see that there are time-reversible and time-irreversible behaviors, both in particles and in macroscopic systems. The time-reversible behaviors seem to govern resources; the time-irreversible behaviors seem to govern entropy. The two logics for these two sets of behaviors are distinct; linear logic and intuitionistic logic can intertwine, but here they don't appear to do so. Why not?
mberning 7y ago
I can imagine many people would not be comfortable answering that question, particularly depending on the company and their culture. I can easily see this type of question leading to dramatically different conclusions ranging from “this person is a deep thinker on varied topics” all the way to “this person wouldn’t be a good cultural fit”. Don’t think too deeply about the wrong topics.
keiferski 7y ago
#2 is explored by John Searle's Chinese Room argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril. This article is an excellent introduction: http://www.dreamsongs.com/Searle.html http://www.dreamsongs.com/Searle.html A relevant excerpt: Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics. The "Simulation, Duplication and Evolution" section of the SEP article is also useful: https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvol https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvo...
- gowld 7y ago
  Searle either misunderstood AI (and computers) or explained himself poorly (as philosophers often do), so misunderstanding Searle isn't a major problem.
  - mannykannot 7y ago
    I seriously doubt that Searle misunderstands computers, and probably not what passes for AI either then or now. I also regard that paper as one of the more accessible works of the genre, though, as always, you have to pay close attention to what claims are being made. On the other hand, at least at the time of writing the original paper, he clearly misunderstood the 'systems reply', as his response to it -- supposing that the actor had memorized the instructions -- is beside the point, and makes no difference.
- lidHanteyk 7y ago
  One important nuance to the Chinese Room is to keep in mind that, even as a black box, there must be limitations on its behavior. We can infer those limitations. We start with Aaronson's observation [0] that the Chinese Room's complexity can be analyzed. Let us assume that the Room responds in (N)P time, and in particular not EXP time. Then the dictionary must be poly-sized. Further, the dictionary is finite but the input strings are unbounded in length, so that we must be able to divide and conquer somehow. Therefore we only need log-many steps to walk the dictionary. The dictionary is ordered, and so indexed, and so walked in log-time. We can conclude that Searle's definition of intelligence is basically the ability to execute certain kinds of basic katamorphisms, akin to pregroup grammars [1] or other categorical grammars; in other words, Searle says that intelligence is merely the ability to talk convincingly like a human, and not necessarily any of the other features that we see across Mammalia. Searle might retort that the Room knows facts about the world. This would require us to augment our pregroup/category, enriching it with relational information. For example, in the utterance "I like cats", we might annotate each word with a relation or type, and then look them up in a secondary database-book. We can see here that it would then be the database-book which has an opinion on cats, since "I" becomes merely another symbol in the database. This is reminiscent of Hofstadter's critique of Searle (supported by his other work), in which he suggests that "I" is indeed merely a self-referent symbol which leads to a strange loop, or as he puts it, "I am a strange loop." Here, the strangeness of the loop is provided by the ability of "I" to be both a plain word in the textual language and also a referent symbol in the symbolic database. Edit: Forgot the main takeaway: When Searle and Turing measure intelligence based on a person's ability to respond to textual queries with textual answers, but the entire interface between the tester and the person is the ability to give textual answers to textual questions, then we aren't measuring intelligence anymore. We've struck Goodhart's Law [2] and we can imagine that the most successful people in these tests will be those who can abuse glitches in text-processing systems. [0] https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf [1] https://en.wikipedia.org/wiki/Pregroup_grammar https://en.wikipedia.org/wiki/Pregroup_grammar [2] https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law
erostrate 7y ago
Richard Feynman has a good explanation for 3. http://www.informationphilosopher.com/solutions/scientists/feynman/past_and_future.html http://www.informationphilosopher.com/solutions/scientists/f...
- ericjang 7y ago
  Thanks! I had a chat with a mathematician friend today, and we came up with a fairly simple resolution actually that I'm pretty satisfied with: the trick here is to stop thinking of particles as T-symmetric billiard balls, and just to assume that there exists microscopic T-asymmetric interactions. If we assume molecular chaos hypothesis (independent particle velocities prior to collision) and take into account probabilistic collisions occurring at the 2-particle microscopic level, then post-collision velocities are now dependent, thereby imposing an ordering to time. I have to think more about this though, and I'm still struggling with understanding the circular dependency between entropy "being time" and entropy being "caused" by "motion" (with respect to ... time?)
  - erostrate 7y ago
    Why can't you assume time symmetric billiard balls? If you take a lot of them, black and white, put all the black ones on one side and the white on the other, give them random velocities, and wait a bit, you will end up with them all mixed together. The entropy has increased even though all interactions are reversible. The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement). The way I think about this (from Feynman) is that entropy measures "how special" a macroscopic state is, on average. When you apply many microscopic independent fluctuations to a given state you are unlikely to end up with a "more special" state. That's how states are ordered at the macroscopic level giving the arrow of time a clear direction.
    - ericjang 7y ago
      The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement). I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric. It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.
intended 7y ago
In this set up, the answer to q1- Intent: No, intent can’t be inferred. If intent could be inferred- the meaningful and material portion of human privacy would cease to exist.
6510 7y ago
> Many AI researchers, myself included, believe that competitive survival of "living organisms" is the only true way to implement general intelligence. My gut translates that idea as: Progress is made though War not Markets. Which seems dubious to say the least.
- andyjohnson0 7y ago
  Isn't a market a kind of competitive environment? Given an appropriate reward function, and multiple independent actors seeking advantage, some might converge on an optimal strategy for attaining AGI. Anyway, that's the way I read it.
  - 6510 7y ago
    Sure but exchange is a more important part of it. Even a predator offers protection services.
- blast 7y ago
  When you put it that way it makes me more sympathetic to the idea, horrifying as it is. War has been one of the biggest things that led to progress, unfortunately. And we don't seem to have learned how to do without it. Walter Scheidel's work has been interesting about this. https://www.economist.com/open-future/2018/09/10/can-inequality-only-be-fixed-by-war-revolution-or-plague https://www.economist.com/open-future/2018/09/10/can-inequal...
TheMechanist 7y ago
Question 2: This is better left unanswered. We've been building superhuman AIs for at least 300 years which are exploiting reality to their advantage and have started out-competing humans for resources (including by changing laws in their favour). These things we've been building are called corporations, and they're certainly smarter than any individual, richer than any human can become and are now working hard to figure out how their humans can be replaced by more reliable components.
nojvek 7y ago
Ask HN: what are the questions that keep you up at night ? For me: How do viruses really work? How to antiviruses work? I’ve gone into basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We know how to sequence DNA and it’s pretty cheap right now. How do we simulate compiling DNA -> proteins and how they interact with each other? How do you find weak spots in a virus and figure out what proteins will inactivate them ? 2) what is the best course of action for people who lost their jobs? Taking <20k from 401k/IRA should be free of taxes and penalties. Everyone has lost >30% of value since Feb already. Sure most people don’t have 401ks, but those who do could weather this storm a bit better. Should we stop payments for all loans for 2 months. No mortgage, auto loans, student loans etc. stop those foreclosures from happening. No rent. 3) will US have the most deaths? Most likely. A large population + most states haven’t locked down. This means unless >80% of population gets infected and recovers to build herd immunity, or we have a vaccine, we’re in deep trouble. Should we infect everyone >10 and under <30 and in clusters so we build up herd immunity in weeks? And get most of the young population back to work? Like if you’re gonna be sitting at home for two weeks, may as well get sick and build up immunity while we’re at it.
GoodJokes 7y ago
Ted Chiang has a great short story in regards to #2, The Lifecycle of Software Objects

84 points by andrewnc 7y ago

> If you don't know how to grasp an object, you can't bring food to your mouth. Counterpoint: hummingbirds. It might sound cheap, but discussions of the "basic necessities" of anything so often just overlook obvious counterexamples.

ericjang 7y ago
Hummingbirds actually proves my point here, on how life and death solve the symbol grounding problem. What is a "stable grasp"? An entire body of classical robotics literature is devoted to answering this problem. I work on robotic grasping research and defining what it means to grasp something successfully is a little tricky unless you have a broader context of what the grasping is for. Nature doesn't care whether hands exist or not - only that the animal can feed. That is why we see such a spectrum of appendages in the animal kingdom. Some are hand-like, others not at all, and some in between. In some sense, "hand" is a mental construct of human language.

RickJWagner 7y ago

1. What is your name? 2. What is your quest? 3. What is your favorite color? Watch out for that last one.

noobly 7y ago
>Watch out for that last one. This must be a reference I’m not getting?
- umvi 7y ago
  Monty Python and the Holy Grail https://youtu.be/0D7hFHfLEyk https://youtu.be/0D7hFHfLEyk
- core_dumped 7y ago
  Watch "Monty Python and The Holy Grail"
pdonis 7y ago
What is the capital of Assyria?
CarlRJ 7y ago
Blue! No, YELLLLLLOOoooooowwwwwwww!
Reedx 7y ago
1. Are you really the head of the Kwik-E-Mart? 2. Really?! 3. You? https://www.youtube.com/watch?v=FiDY2nXVLZ0 https://www.youtube.com/watch?v=FiDY2nXVLZ0

sudoaza 7y ago

Lame, only true question is will this universe die out and all information in it be lost or can it be maintained indefinitely/transfer information to the next universe iteration.

pwdisswordfish2 7y ago
> only true question you mean like, the last question?
- empiricus 7y ago
  I guess, the only question that matters. Unfortunately it does not matter for us, we are too transient.

Gatsky 7y ago

Sometimes I wonder if we are the critters related to the second question. Of course the next question is how far away are we from meeting the simulation’s spec.

untog 7y ago

I know it’s just the moment we’re living in right now but as I read those questions I just shrug, they couldn’t feel less relevant. The ones on my mind are “how long will large populations be locked down?” “How many people are going to die?” and “how long until this all happens again?”

nine_k 7y ago
One could go even shorter term than that. But life is a long game.
ceilingcorner 7y ago
“The greats never sacrifice the important for the urgent. They handle the immediate problem and still make sure to secure the future.” The media is incentivized to distract you. Turn off your computer, read a book, and don’t forget to live.
- deleted 7y ago
  [deleted]
- diN0bot 7y ago
  no i think you misunderstand. those three questions the commenter posted aren’t about running from fear or anxiety — some would simply call “questions about how we collectively overcome massive and pressing challenges” more relevant, in the short and long term.
- untog 7y ago
  What an absurd statement. This crisis matters. Social distancing matters. Lockdowns matter. “Stick your fingers in your ears” is awful advice.
  - m-p-3 7y ago
    But at some point, the medias are rehashing the same statements: Stay home, and follows good practices. As an individual, there's only so much you can do and there's no need to keep yourself distracted from other matters.

Red_Leaves_Flyy 7y ago

Tldr: >Q1: Can we imitate "thinking" from only observing behavior? >Q2: What is the computationally cheapest "organic building block" of an Artificial Life simulation that could lead to human-level AGI? >Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise from T-symmetric (microscopic) physical laws?

mannykannot 7y ago

> Q1: Can we imitate "thinking" from only observing behavior? If you believe that the human mind is the result of Darwinian evolution, then you implicitly believe that thinking can arise from interacting with the environment. That does not exactly answer the question, but if anything, it goes beyond it. On the other hand: "This could also be addressed via other high-data environments: 1) Observing trader orders on markets and seeing if we can recover the trader's deductive reasoning and beliefs about the future. See if we can observe rational thought (if not rational behavior). 2) Recovering intent and emotions and desire from social network activity." Before you could even begin to get anything from these streams of symbols, you would have to have some way to give them semantics - to understand what they signify to the actors. I suspect that these would be among the last things you would achieve, not a means to the end.

jvm___ 7y ago

Do we already posess the hardware required to run full AI? Deep fakes can be done on older hardware, we just didn't have the code. Does the same thing apply to AI?

Zenst 7y ago

Q1: Can we imitate "thinking" from only observing behavior? NO as you are measuring actions and not the process that instigated those actions. One example - chess - you see the move, but the thinking behind that move may elude. More so a bad move, as their thinking was that it is a good move when it is not. [EDIT format/spelling]

ethn 7y ago

Q1) By the mere copying of behaviors, no. It is clear that there are at least the a priori intuitions of space, time, and causality necessary for humans to obtain any knowledge. It is within that a priori framework where concepts and then decisions are derived from. Q2) This would require the knowledge of what is sufficient & necessary for AGI. Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this question. Read page 33 of Weiner's Cybernetics. The real insight, where in what I will now explain in brevity is at the expense of the insight's resolution, is that in the nature of microscopic phenomena there are an exorbitant quantity of fickle components ("equal particles") whose positions, accelerations, velocities are all impossible to record because of that sheer quantity by the nature of this sphere of study. More so, there are then at least second-order coupling effects between all these fickle components due to their constant interaction. As a resolve, we have developed terms of statistical understanding. Weiner cites "cloud, temperature, turbulence, etc." all being concepts which are specifically defined in a statistical context. Where as in the macroscopic sphere there are far fewer particles much further apart from each other with near certain initial masses, positions, and velocities. The result? In the microscopic systems, because their properties are precisely defined statistically the distribution which predicts the future nor past for a given system isn't unique. That is, the future state (predicted by the distribution provided by some initial system) could have also been predicted by other distributions describing many other initial systems unless by "some miracle" there is a "tightly defined statistical range". So are microscopic physics condemned to this paradox? Not necessarily, there may be some way to construct scientific objects from systems phenomena which don't require a statistical conception because they can be precisely defined from a microscopic systems point of view temporally & spatially. However, this would require a revolution in theory and likely mathematics

lidHanteyk 7y ago
I dislike the cybernetic approach because I find that it first commits to epistemic helplessness, saying that we know nothing of particles. Perhaps that was reasonable when QM was so young, but these days we can use QM for a better answer. This isn't complete, but it helps a lot. First, note that CPT-symmetry can be extended up to the macroscopic level. Einstein's thought experiments about time involve macroscopic entanglement. For example, given that a red ball and a blue ball are each in their own boxes, and one possesses one box, then the box is entangled to have either a red or blue ball inside. We can use linear logic to reason about these situations; one holds the multiplicative disjunction of red and blue. Second, note that in order to make MWI's numbers work, we need not just branching of possibilities, but also coalescing. Imagine that, in empty space, we suddenly have a single virtual-particle pair; the diagram would look like () with time vertical. At the beginning, we have one universe; during the split, two universes, one for each chirality of the pair; afterwards, one universe. When we split, the probabilities of each possible universe decrease. Now, imagine doing this with more pairs, and verify that the Binomial Theorem manifests. This gives the same rise to macrostate-like entropy as Boltzmann's approach to thermodynamics, but only using QM particles and QM logic; a QM state is likelier when more of the many worlds in MWI would support its existence. Putting these together, we may conclude that Loschmidt's paradox is an incomplete part of a bigger paradoxical complaint: We can see that there are time-reversible and time-irreversible behaviors, both in particles and in macroscopic systems. The time-reversible behaviors seem to govern resources; the time-irreversible behaviors seem to govern entropy. The two logics for these two sets of behaviors are distinct; linear logic and intuitionistic logic can intertwine, but here they don't appear to do so. Why not?

mberning 7y ago

I can imagine many people would not be comfortable answering that question, particularly depending on the company and their culture. I can easily see this type of question leading to dramatically different conclusions ranging from “this person is a deep thinker on varied topics” all the way to “this person wouldn’t be a good cultural fit”. Don’t think too deeply about the wrong topics.

keiferski 7y ago

#2 is explored by John Searle's Chinese Room argument, which is unfortunately completely misunderstood by most technologists working in AI, to their own peril. This article is an excellent introduction: http://www.dreamsongs.com/Searle.html http://www.dreamsongs.com/Searle.html A relevant excerpt: Searle goes on to contend that intentional behavior must be in response to real contingencies, not formal simulations of reality. If so, it would seem to follow that the capacity for intentional behavior is the result of what might be called phylogenic contingencies, and so would only be found in products of real evolution. In short, what Searle is saying is that, just like other biological activities such as digestion and photosynthesis, thought is intrinsically dependent on the biochemistry of its origin. Just as a formal simulation of digestion is not really digesting, a formal simulation of thought is not really thinking. And to tie this to the gender thought experiment, a formal simulation of gender is not gender, because it lacks biological structural and functional characteristics. The "Simulation, Duplication and Evolution" section of the SEP article is also useful: https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvol https://plato.stanford.edu/entries/chinese-room/#SimuDuplEvo...

gowld 7y ago
Searle either misunderstood AI (and computers) or explained himself poorly (as philosophers often do), so misunderstanding Searle isn't a major problem.
- mannykannot 7y ago
  I seriously doubt that Searle misunderstands computers, and probably not what passes for AI either then or now. I also regard that paper as one of the more accessible works of the genre, though, as always, you have to pay close attention to what claims are being made. On the other hand, at least at the time of writing the original paper, he clearly misunderstood the 'systems reply', as his response to it -- supposing that the actor had memorized the instructions -- is beside the point, and makes no difference.
lidHanteyk 7y ago
One important nuance to the Chinese Room is to keep in mind that, even as a black box, there must be limitations on its behavior. We can infer those limitations. We start with Aaronson's observation [0] that the Chinese Room's complexity can be analyzed. Let us assume that the Room responds in (N)P time, and in particular not EXP time. Then the dictionary must be poly-sized. Further, the dictionary is finite but the input strings are unbounded in length, so that we must be able to divide and conquer somehow. Therefore we only need log-many steps to walk the dictionary. The dictionary is ordered, and so indexed, and so walked in log-time. We can conclude that Searle's definition of intelligence is basically the ability to execute certain kinds of basic katamorphisms, akin to pregroup grammars [1] or other categorical grammars; in other words, Searle says that intelligence is merely the ability to talk convincingly like a human, and not necessarily any of the other features that we see across Mammalia. Searle might retort that the Room knows facts about the world. This would require us to augment our pregroup/category, enriching it with relational information. For example, in the utterance "I like cats", we might annotate each word with a relation or type, and then look them up in a secondary database-book. We can see here that it would then be the database-book which has an opinion on cats, since "I" becomes merely another symbol in the database. This is reminiscent of Hofstadter's critique of Searle (supported by his other work), in which he suggests that "I" is indeed merely a self-referent symbol which leads to a strange loop, or as he puts it, "I am a strange loop." Here, the strangeness of the loop is provided by the ability of "I" to be both a plain word in the textual language and also a referent symbol in the symbolic database. Edit: Forgot the main takeaway: When Searle and Turing measure intelligence based on a person's ability to respond to textual queries with textual answers, but the entire interface between the tester and the person is the ability to give textual answers to textual questions, then we aren't measuring intelligence anymore. We've struck Goodhart's Law [2] and we can imagine that the most successful people in these tests will be those who can abuse glitches in text-processing systems. [0] https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf [1] https://en.wikipedia.org/wiki/Pregroup_grammar https://en.wikipedia.org/wiki/Pregroup_grammar [2] https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law

erostrate 7y ago

Richard Feynman has a good explanation for 3. http://www.informationphilosopher.com/solutions/scientists/feynman/past_and_future.html http://www.informationphilosopher.com/solutions/scientists/f...

ericjang 7y ago
Thanks! I had a chat with a mathematician friend today, and we came up with a fairly simple resolution actually that I'm pretty satisfied with: the trick here is to stop thinking of particles as T-symmetric billiard balls, and just to assume that there exists microscopic T-asymmetric interactions. If we assume molecular chaos hypothesis (independent particle velocities prior to collision) and take into account probabilistic collisions occurring at the 2-particle microscopic level, then post-collision velocities are now dependent, thereby imposing an ordering to time. I have to think more about this though, and I'm still struggling with understanding the circular dependency between entropy "being time" and entropy being "caused" by "motion" (with respect to ... time?)
- erostrate 7y ago
  Why can't you assume time symmetric billiard balls? If you take a lot of them, black and white, put all the black ones on one side and the white on the other, give them random velocities, and wait a bit, you will end up with them all mixed together. The entropy has increased even though all interactions are reversible. The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement). The way I think about this (from Feynman) is that entropy measures "how special" a macroscopic state is, on average. When you apply many microscopic independent fluctuations to a given state you are unlikely to end up with a "more special" state. That's how states are ordered at the macroscopic level giving the arrow of time a clear direction.
  - ericjang 7y ago
    The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement). I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric. It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.

intended 7y ago

In this set up, the answer to q1- Intent: No, intent can’t be inferred. If intent could be inferred- the meaningful and material portion of human privacy would cease to exist.

6510 7y ago

> Many AI researchers, myself included, believe that competitive survival of "living organisms" is the only true way to implement general intelligence. My gut translates that idea as: Progress is made though War not Markets. Which seems dubious to say the least.

andyjohnson0 7y ago
Isn't a market a kind of competitive environment? Given an appropriate reward function, and multiple independent actors seeking advantage, some might converge on an optimal strategy for attaining AGI. Anyway, that's the way I read it.
- 6510 7y ago
  Sure but exchange is a more important part of it. Even a predator offers protection services.
blast 7y ago
When you put it that way it makes me more sympathetic to the idea, horrifying as it is. War has been one of the biggest things that led to progress, unfortunately. And we don't seem to have learned how to do without it. Walter Scheidel's work has been interesting about this. https://www.economist.com/open-future/2018/09/10/can-inequality-only-be-fixed-by-war-revolution-or-plague https://www.economist.com/open-future/2018/09/10/can-inequal...

TheMechanist 7y ago

Question 2: This is better left unanswered. We've been building superhuman AIs for at least 300 years which are exploiting reality to their advantage and have started out-competing humans for resources (including by changing laws in their favour). These things we've been building are called corporations, and they're certainly smarter than any individual, richer than any human can become and are now working hard to figure out how their humans can be replaced by more reliable components.

nojvek 7y ago

Ask HN: what are the questions that keep you up at night ? For me: How do viruses really work? How to antiviruses work? I’ve gone into basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We know how to sequence DNA and it’s pretty cheap right now. How do we simulate compiling DNA -> proteins and how they interact with each other? How do you find weak spots in a virus and figure out what proteins will inactivate them ? 2) what is the best course of action for people who lost their jobs? Taking <20k from 401k/IRA should be free of taxes and penalties. Everyone has lost >30% of value since Feb already. Sure most people don’t have 401ks, but those who do could weather this storm a bit better. Should we stop payments for all loans for 2 months. No mortgage, auto loans, student loans etc. stop those foreclosures from happening. No rent. 3) will US have the most deaths? Most likely. A large population + most states haven’t locked down. This means unless >80% of population gets infected and recovers to build herd immunity, or we have a vaccine, we’re in deep trouble. Should we infect everyone >10 and under <30 and in clusters so we build up herd immunity in weeks? And get most of the young population back to work? Like if you’re gonna be sitting at home for two weeks, may as well get sick and build up immunity while we’re at it.

GoodJokes 7y ago

Ted Chiang has a great short story in regards to #2, The Lifecycle of Software Objects