4 ms·

Lots of great stuff in here, this struck me: > It is already the case that for many of us when we make a decision about what book to read or what film to see o

by spydum 12y ago

Lots of great stuff in here, this struck me: > It is already the case that for many of us when we make a decision about what book to read or what film to see or what restaurant to eat at, we don’t make a reasoned decision based on personal experience, but on the recommendations of an app! So that little bit of free will that we got to exercise in our daily lives has evaporated. So the lake of free will has become a pond.

clusterfoo 12y ago
Which leads to an unsettling thought: > And culture builds on culture. (...) That cumulative process is what allows for the civilization that we now live in, but it raises a very interesting paradox and it’s interesting that it’s very rarely commented upon. > Here’s the thought experiment: (...) imagine I showed you an Apple II running World of Warcraft. If you knew anything about computers you’d say, “Impossible. Impossible!” (...) > Now, let’s think about human beings. Human beings are hardware that’s about 100,000 years old, but we run string theory, Lie algebra. We’re running 21st-century software! (...) > I happen to be one who believes that the cultural becomes so complicated at a certain point that it won’t run on our brains. And in fact, you could argue that the reason why we’ve generated computational devices is consciously or unconsciously, we’ve come to recognize that our endogenous, organic computing power is not up to the task and we have to recruit machines to represent culture, because we cannot. I think there’s good evidence for that. > I think that that ultimately might be what bounds us; that we’ll reach a point where our memory capacity and inferential power simply cannot accommodate the latest cultural artifact. At that point what happens? Does it become independent of us, or does it just stop? It’s like evolution coming to an end. --- The whole thing is worth watching really, but that question really hit home for me. Though maybe I'm more optimistic: if we automated away all of the menial -- such as deciding what's for dinner, searching for the next book to read, etc. (unless it's done for pleasure of course) -- and the political -- automate a large portion of how our institutions are run -- all of these things that stifle and fragment our creative energies, so that we could focus all of our brainpower and energy on creative endeavours, how much more could we accomplish? There's still a cognitive limit, but it seems to me that currently we are mired by so much unnecessary overhead that once we are able to remove it, we'll find that limit is a lot farther away than we expected. I'd guess that for most of us, well over 50% of our creative energy is instead spent on the unnecessary and menial. Especially since brains are not Turing machines, and the cost of overhead is not linear (50% overhead might mean more than 50% productivity loss): brains get tired over time, they burn out, they need rest, context switching is hard... that overhead burns you out and is making you less productive even when you're not actively thinking about it (what's for dinner, your personal finances, schedules, meetings, that dentist appointment, am I due for a haricut? did I already book the mechanic? my car is making that weird noise again, oh crap did I miss the new Game of Thrones last night, now I gotta catch up... etc etc etc etc) -- what a relief would it be to automate all of that and only focus on what's fun and what's important!)
- maxerickson 12y ago
  Thinking along those lines, a major role of modern society is to automate away safety. It isn't perfect, but I guess there are lots of ways in which it succeeds at it.
  - deleted 12y ago
    [deleted]
  - clusterfoo 12y ago
    And security. The biggest problem of course is that this a field ripe with potential for abuse... there's a fine line between a personalized book recommendation service, and a massive, streamlined propaganda machine. We have the knowhow to build either (very near) futures: one in which we automate all the menial aspects of our day-to-day lives and are left to explore and create... and one in which our desire for convenience is exploited by governments and corporations. The technical difficulty of achieving either is the same, the only difference between utopia and dystopia in this case is the foundation on top of which it's built: One is built on openness: open data, neutral networks, open software and standards, all massively available so that anyone with the ingenuity to do so can pitch in and help build this future. The other is built on a foundation of walled gardens and closed ecosystems: everyone building on top of platforms and technologies that are ultimately controlled by a handful of individuals whose interests we now have to trust align with our own.
    - A_COMPUTER 12y ago
      It wouldn't be propaganda if it gave you want you wanted. But the same systems that can predict what you want will give you what some unnamed face wants the masses to know, and silently reports on people whom it detects aren't complying. It's the same system, what it does depends on who owns it and how transparent their operations are.
- troels 12y ago
  > Here’s the thought experiment: (...) imagine I showed you an Apple II running World of Warcraft. If you knew anything about computers you’d say, “Impossible. Impossible!” (...) He should have look at some of the stuff that demo scene programmers churn out of a C64 these days.
convexfunction 12y ago
I'm trying to figure out an interpretation of this that actually makes sense. He's suggesting people don't even consider their prior experiences when choosing restaurants now? The subjective experience of "free will" requires a high threshold of ignorance? Maybe I'm overestimating other people but in those specific contexts, I almost never take the number of stars or a few people's reviews as the only relevant things to consider. (Or, maybe he's flattering himself by perceiving a type of non-agency in "the masses" that doesn't exist.)
- xxxyy 12y ago
  I think the point is that when the AI takes over the control of the world it does not have to be through a technological singularity, with Roko's Basilisk and all the drama. It could as well be through generations of people progressively yielding their free will to what he calls "apps". In this scenario, our choices, over the course of a century or so, become ultimately non-existent.
  - tjradcliffe 12y ago
    > In this scenario, our choices, over the course of a century or so, become ultimately non-existent. You need to explain how choosing to let apps make certain types of decision removes all choices whatsoever. As it stands, you make it sound like a carpenter who is using a nail gun instead of hammer has stopped driving nails.
    - jonas21 12y ago
      Suppose the carpenter pulls out an autonomous house-building robot and tells it to build him a house. Is he still driving nails? But to your main point, while we may be offloading only trivial decisions to apps today, the better they become at making these decisions, the more natural it will be to trust them for more significant ones. As the original article mentions, it's not much of a stretch to imagine an app that looks at your demographics and preferences and tells you who to vote for. And from there, why not apps that choose where to live, what career to pursue, or who to marry? Some day, it may even seem foolish not to defer to apps for important decisions. After all, how can one fallible, emotional person ever hope to make a better decision than a datacenter full of machines that can coolly consider all of the parameters and potential outcomes? At that point, floating through a blissfully optimized life, one might say that yes, the apps are deciding everything for me, but they're doing so only in accordance with my preferences and values. I'm still in charge; I'm still exercising free will. But in the absence of making decisions oneself, where exactly did those preferences and values come from?

by spydum 12y ago

clusterfoo 12y ago

Which leads to an unsettling thought: > And culture builds on culture. (...) That cumulative process is what allows for the civilization that we now live in, but it raises a very interesting paradox and it’s interesting that it’s very rarely commented upon. > Here’s the thought experiment: (...) imagine I showed you an Apple II running World of Warcraft. If you knew anything about computers you’d say, “Impossible. Impossible!” (...) > Now, let’s think about human beings. Human beings are hardware that’s about 100,000 years old, but we run string theory, Lie algebra. We’re running 21st-century software! (...) > I happen to be one who believes that the cultural becomes so complicated at a certain point that it won’t run on our brains. And in fact, you could argue that the reason why we’ve generated computational devices is consciously or unconsciously, we’ve come to recognize that our endogenous, organic computing power is not up to the task and we have to recruit machines to represent culture, because we cannot. I think there’s good evidence for that. > I think that that ultimately might be what bounds us; that we’ll reach a point where our memory capacity and inferential power simply cannot accommodate the latest cultural artifact. At that point what happens? Does it become independent of us, or does it just stop? It’s like evolution coming to an end. --- The whole thing is worth watching really, but that question really hit home for me. Though maybe I'm more optimistic: if we automated away all of the menial -- such as deciding what's for dinner, searching for the next book to read, etc. (unless it's done for pleasure of course) -- and the political -- automate a large portion of how our institutions are run -- all of these things that stifle and fragment our creative energies, so that we could focus all of our brainpower and energy on creative endeavours, how much more could we accomplish? There's still a cognitive limit, but it seems to me that currently we are mired by so much unnecessary overhead that once we are able to remove it, we'll find that limit is a lot farther away than we expected. I'd guess that for most of us, well over 50% of our creative energy is instead spent on the unnecessary and menial. Especially since brains are not Turing machines, and the cost of overhead is not linear (50% overhead might mean more than 50% productivity loss): brains get tired over time, they burn out, they need rest, context switching is hard... that overhead burns you out and is making you less productive even when you're not actively thinking about it (what's for dinner, your personal finances, schedules, meetings, that dentist appointment, am I due for a haricut? did I already book the mechanic? my car is making that weird noise again, oh crap did I miss the new Game of Thrones last night, now I gotta catch up... etc etc etc etc) -- what a relief would it be to automate all of that and only focus on what's fun and what's important!)

maxerickson 12y ago
Thinking along those lines, a major role of modern society is to automate away safety. It isn't perfect, but I guess there are lots of ways in which it succeeds at it.
- deleted 12y ago
  [deleted]
- clusterfoo 12y ago
  And security. The biggest problem of course is that this a field ripe with potential for abuse... there's a fine line between a personalized book recommendation service, and a massive, streamlined propaganda machine. We have the knowhow to build either (very near) futures: one in which we automate all the menial aspects of our day-to-day lives and are left to explore and create... and one in which our desire for convenience is exploited by governments and corporations. The technical difficulty of achieving either is the same, the only difference between utopia and dystopia in this case is the foundation on top of which it's built: One is built on openness: open data, neutral networks, open software and standards, all massively available so that anyone with the ingenuity to do so can pitch in and help build this future. The other is built on a foundation of walled gardens and closed ecosystems: everyone building on top of platforms and technologies that are ultimately controlled by a handful of individuals whose interests we now have to trust align with our own.
  - A_COMPUTER 12y ago
    It wouldn't be propaganda if it gave you want you wanted. But the same systems that can predict what you want will give you what some unnamed face wants the masses to know, and silently reports on people whom it detects aren't complying. It's the same system, what it does depends on who owns it and how transparent their operations are.
troels 12y ago
> Here’s the thought experiment: (...) imagine I showed you an Apple II running World of Warcraft. If you knew anything about computers you’d say, “Impossible. Impossible!” (...) He should have look at some of the stuff that demo scene programmers churn out of a C64 these days.

convexfunction 12y ago

I'm trying to figure out an interpretation of this that actually makes sense. He's suggesting people don't even consider their prior experiences when choosing restaurants now? The subjective experience of "free will" requires a high threshold of ignorance? Maybe I'm overestimating other people but in those specific contexts, I almost never take the number of stars or a few people's reviews as the only relevant things to consider. (Or, maybe he's flattering himself by perceiving a type of non-agency in "the masses" that doesn't exist.)

xxxyy 12y ago
I think the point is that when the AI takes over the control of the world it does not have to be through a technological singularity, with Roko's Basilisk and all the drama. It could as well be through generations of people progressively yielding their free will to what he calls "apps". In this scenario, our choices, over the course of a century or so, become ultimately non-existent.
- tjradcliffe 12y ago
  > In this scenario, our choices, over the course of a century or so, become ultimately non-existent. You need to explain how choosing to let apps make certain types of decision removes all choices whatsoever. As it stands, you make it sound like a carpenter who is using a nail gun instead of hammer has stopped driving nails.
  - jonas21 12y ago
    Suppose the carpenter pulls out an autonomous house-building robot and tells it to build him a house. Is he still driving nails? But to your main point, while we may be offloading only trivial decisions to apps today, the better they become at making these decisions, the more natural it will be to trust them for more significant ones. As the original article mentions, it's not much of a stretch to imagine an app that looks at your demographics and preferences and tells you who to vote for. And from there, why not apps that choose where to live, what career to pursue, or who to marry? Some day, it may even seem foolish not to defer to apps for important decisions. After all, how can one fallible, emotional person ever hope to make a better decision than a datacenter full of machines that can coolly consider all of the parameters and potential outcomes? At that point, floating through a blissfully optimized life, one might say that yes, the apps are deciding everything for me, but they're doing so only in accordance with my preferences and values. I'm still in charge; I'm still exercising free will. But in the absence of making decisions oneself, where exactly did those preferences and values come from?