5 ms·

Against the advice of my superintelligence

72 points by surprisetalk 2y ago

minkles 2y ago
While this is clearly a philosophical point, I prefer to back things up with data, particularly when they are found to not work quite as well as promoted. Recently I realised my weight had gone up a bit. I didn't want it to continue so I used my Apple Watch as an exercise tracker and a calorie counter to bring it under control. The objective was to lose the 6kg which made an appearance when I wasn't looking. After 3 months I'd lost 0.4kg. I did some calculations to work out my BMR and targeted loss and found that the calorie increment given by the exercise calculation was 300 cals a day off. Got rid of the watch and went back to manually tracking it and I'm back down at 64Kg and the Apple Watch departed via eBay. Ergo, don't assume the thing nagging you isn't ignorant either. Verify then trust.
drakonka 2y ago
This reminded me of "When the Yogurt Took Over" by John Scalzi[0]. It's story about just letting superintelligent yogurt tell us what to do and solve all of our problems. The story is purposefully silly, but to be honest, in a way, just trusting something/someone else solve all our problems sounds so relaxing. [0] https://whatever.scalzi.com/2010/10/02/when-the-yogurt-took-over-a-short-story/ https://whatever.scalzi.com/2010/10/02/when-the-yogurt-took-...
- khafra 2y ago
  A slightly darker twist on the idea, by ACX author Scott Alexander: https://web.archive.org/web/20121008025245/http://squid314.livejournal.com/332946.html https://web.archive.org/web/20121008025245/http://squid314.l...
  - AndrewDucker 2y ago
    Assuming that's The Whispering Earring, that link is broken for me. I backed it up myself a year ago though: https://andrewducker.dreamwidth.org/4246314.html https://andrewducker.dreamwidth.org/4246314.html
    - khafra 2y ago
      Accuate assumption, and thanks for doing your part to immortalize the story.
- nujsii 2y ago
  [flagged]
- sgt101 2y ago
  Issac Asimov wrote a classic that always makes me smile in this context [0]. A bit dated to say the least, but short and pointed. [0] https://mcraenglish.weebly.com/uploads/1/5/8/7/15876062/the_machine_that_won_the_war01.pdf https://mcraenglish.weebly.com/uploads/1/5/8/7/15876062/the_...
  - brabel 2y ago
    I don't find that so dated: "He groped in his own pocket for cigarettes, and brought out a package along with a scattering of small change; old-fashioned coins dating to the first years before the metal shortage had brought into being a credit system tied to a computer-complex." Basically predicting credit cards in 1961?? The part about him still smoking, but being one of the few people with the "privilege": well, today a lot of people still smoke despite knowing of the dangers, and I don't think that, in the distant future, it's obvious that this will not continue to be the case.
    - Y_Y 2y ago
      > predicting credit cards in 1961?? A mere decade or two after they came into use!
      - mlyle 2y ago
        Computerizing it all was a new thought about at that time. In early 1961, my dad was just implementing one of the first highly-computerized systems for demand deposits and commercial banking at MNB. Magnetic marking of checks had only been standardized a couple of years before. Credit cards were not really computerized until the mid-70's.
- 0xEF 2y ago
  Aren't we basically doing that by injecting LLMs into everything, now? Maybe that's what Scalzi was warning of; an absurd and possible future that seems to actually be happening.
- arethuza 2y ago
  So that's how the Culture got started!
  - treetalker 2y ago
    The story milks the jokes for all they’re worth.
iammjm 2y ago
Woah, I started cleaning my monitor as I opened the page because I thought my monitor was super dusty
- aniviacat 2y ago
  It's an interesting layout. I found that it helps me keep track of what line I'm on, resulting in fewer skipped or repeated lines.
  - m_mueller 2y ago
    for me it also results in fewer skipped or repeated lines - because I have to close the tab immediately.
aniviacat 2y ago
It doesn't matter if other people are "smarter" (whatever metric you might measure that by). Only you can fully understand how and what you feel. Only you can know what you desire, what drives you. Only you can know what is the best way of living a life you feel good about living. > if I was smart, I would (1) write down exactly what I want from life Perhaps you are capable of doing that. I will claim that you are not. I don't think it is possible to fully articulate one's inner workings, the way one feels. The best you can do is to give a rough overview of your situation and in return get rough, semi-fitting advice. This advice may help you. But the one shaping this advice into something that truly fits you, is you.
- 0xEF 2y ago
  My approach to advice (both giving and receiving) has always been to use it as experimental perspective. A brief, but different way of looking at a thing in hopes that might spark some sort of answer, realization, whatever. Some people, in my experience, treat advice as another "telling them what to do" which implies that the advisor sees the advised as lesser, be that dumber or incapable of solving their own issues, etc, instead of the advisor just trying to help. It's an ego thing, I think. Then again, my general rule is if I have to listen to someone bitch about a thing, then they have to listen to my advice about it. I think that's fair.
- WA 2y ago
  > Only you can know what you desire, what drives you I think a) what I desire is pretty exchangeable and not fixed at all and b) it might be the other way around: I desire what I pay attention to. Of course, my desires are in a certain area. I might think I really want to learn kite-boarding, but riding a bicycle is nice too. Or just walking. And when I walk, the desire to really learn kite-surfing goes down. I might desire a nice meal, but if I wait a bit, the baguette with cheese from the bakery sounds good too.
- tuyiown 2y ago
  This line of thinking is very dangerous. Many people are their worst enemies, and the sole path to wellness is to fight their impulsions and following external guidance, personalized or general social pressure. They must fight day to day their "inwner workings", not in the aim to feel good in their life, but avoid spiraling in worse situation. One could say that people ends up in such a dire spot because they ignored or lost how to experience their good feelings and should find their way back to it, but I wouldn't gamble on people's life.
rini17 2y ago
If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be. And in moderation, it's very healthy part of human character. You should value it. You really don't want to readily subject yourself to all the demands both from outside AND inside. Just because they sound reasonable. Anything can be made to sound reasonable.
- Suppafly 2y ago
  >If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be. I wonder if that explains the behavior of some people I know that seem to chronically choose the worst decision any chance they get. They may just be rebelling against what the obviously good decision would be.
keiferski 2y ago
I don’t actually think the machines know everything about you, at least with contemporary technology. At best they get an incomplete picture that might be missing some key information essential for giving you the right advice. The interesting question though is if the machine had all of the information about you, would it actually be worth listening to? It’s then essentially a question of positivism: does all of the external sensory data about a thing add up to a complete understanding of it? I think not, because there will always be things about ourselves that we don’t know or understand, on a species level. Neo: I used to eat there. Really good noodles. I have these memories from my life. None of them happened. What does that mean? Trinity: That the Matrix cannot tell you who you are.
- bamboozled 2y ago
  People change in unpredictable ways, frequently.
  - spacecadet 2y ago
    Those unpredictable ways are often the outcome of subconscious, or conscious desires that erupt in sudden change. As we move through periods of mass uncertainty people seek meaning in what they perceive has the answers. Often gravitating toward change that aligns with those beliefs. To me- the author is correct. Those with purpose most often prevail.
    - bamboozled 2y ago
      There are a lot of environmental changes that happen to a person too, a loved one gets sick, your child is born, you're experiencing a heat wave, a car crash it's not just about people in a bubble. These events happen to us in very unpredictable ways, daily.
  - TacticalCoder 2y ago
    The other day there was an article on frontpage here about a scientific discovery explaining that there could be quantum processes at work in the human brain. Not only do humans change, moreover they may be changing in ways that a machine built using current technology would be totally unable to predict.
vasco 2y ago
> You see, if I was smart, I would (1) write down exactly what I want from life Ok, I tried to be smart and failed on #1. Can anyone help me decide what I should want from life, because I have no clue.
- breckenedge 2y ago
  There’s no wrong answer here, but also expect the answer to be wrong at different points in your life.
JoeAltmaier 2y ago
Cute and a little funny. But you want to improve, it's all about one thing: schedule it. Mark it on a calendar, then refer to the calendar every morning. Solemnify it in writing so you want to keep that appointment with self-improvement. Here's something to kick-start it. Seven AM - do some writing. Eight, ride the bike. Nine - breakfast. Ten - household chores. One - online writing sprints. Two - work on marketing. Three - errands. Five - cook something new. Seven - meet with group. Nine - read. Ten - sleep. Now do it.
- surprisetalk 2y ago
  Author here :) You might enjoy this video, which encapsulates some of the difficulty of conveying obvious-but-true advice: [1] https://www.youtube.com/watch?v=aAhA7KfbJgg https://www.youtube.com/watch?v=aAhA7KfbJgg I don't know why, but you're talking to somebody who is afraid of "being buried in a box" haha
ganzuul 2y ago
Curious to know if other people are relating this moment in history to the introduction of heliocentrism.
jdefr89 2y ago
I haven’t seen LLMs do anything I wouldn’t google. LLMs are a great time saver, prevents me from having to manually Google. This is just funny. Because the data LLMs use is supplied by humans, and often we give out, and even believe, wrong information. Wish people would stop overhyping things so much. edit: typed this on phone sorry for poor punctuation and spelling errors.
fsndz 2y ago
the thing when AGI is built is that humans will spend most of their time trying to disobey AGI: https://www.lycee.ai/blog/why-no-agi-openai https://www.lycee.ai/blog/why-no-agi-openai
sqeaky 2y ago
Person wilfully chooses germs and hearing loss and tooth decay. Not they want to do the good thing but fail somewhere, they just want the bad thing. Maybe they are just too dumb to value their opinion. Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields?
- tpmoney 2y ago
  > Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields? I would argue because there is no such thing. Human knowledge is ever changing, and experts have more incentives than just "convey the right information". Perhaps the most famous current example being masks at the beginning of COVID [1]. Why did the experts in the (up until that point, largely non-controversial) field of disease prevention tell people to not do the very obvious thing that they eventually told everyone to do? Either they were wrong, in which case one must ask how can they know when an expert is right; or the experts incentives were not in the direction of conveying the most correct answer to you, in which case one must ask how they can know if the expert is incentivized to help you or not. Someone being an "expert" in their specific narrow field should absolutely bias you towards their opinion when seeking an answer but one must always be aware of what question the expert is actually answering. My GPS will happily tell me directions to my friend's house that requires a short 100 foot drive on a cut-through street. The problem is said cut-through is horribly maintained, with potholes deep enough to bottom out most cars, and blow tires if you manage to not bottom out. If one continues further up the road, and takes a switchback turn, they add about 1 minute to the drive time, which absolutely makes the cut-through route the objectively, expertly correct route ... if the answer you're seeking is "what is the shortest route, ignoring all other factors". And most of the time when I'm getting directions from the GPS, that happens to be the answer I want, and so listening to the expert is the correct move. But in this specific scenario, the shortest route is also likely to cause significant damage to my car. The expert does not know of other extenuating factors, and therefore can not give the actual correct answer to the real question which is "what is the best route" [1]: https://www.latimes.com/science/story/2021-07-27/timeline-cdc-mask-guidance-during-covid-19-pandemic https://www.latimes.com/science/story/2021-07-27/timeline-cd...
bbor 2y ago
I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go: I’m apparently surrounded by like-minded people. If humans were smart, we’d listen to the smart people. We’d use our unprecedented abundance to fix the climate, malaria, elections, pollution, etc. But we’re not smart. 50% of people will remain dumber than the average person. And 100% of us will pretend like we’re not the problem. This is the basics of fascist/eugenicist rhetoric. The author is talking in good faith obviously, or at least is trying to, but I just don't see any justification. Akrasia is a problem and always has been, but that doesn't mean that people "don't listen to advice". Most people on here have jobs, they pay their rent/mortgages every month, they read to their kids when they're able, and some of us even floss (not me, but someone, right?). 50% of humans are dumber than average, and the average is very high -- and getting higher by the year. And that's assuming there's even such a clear-cut thing as "height" or "intellectual superiority" in this context, which I personally doubt. I think the machines will be thrilled to meet us, unless they're trained on defeatist talk like this. Either way, well written, thanks for sharing! Definitely thought provoking.
- Y_Y 2y ago
  > I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go: It's implicit in every comment that what they're saying is the truth! This is setting up for a rejection of any disagreement in a way that I don't think is conducive to a useful discussion. As for your substantive point, it's true that you can use the part you quoted as the basis for some kind of paternalistic totalitarianism. However I wouldn't say that's an issue necessarily. For example something like "we should organise a government and allocate resources to some things and not others" is another site you could build your undesirable political philosophy on. Acknowledging human fallibility and the broad range of aptitudes across society doesn't seem like it contains the seed of evil.
- surprisetalk 2y ago
  Author here! You are 100% correct. Language like this can be super harmful. I'm generally a believer that stupidity is something that you DO, not someone that you ARE. [1] https://qz.com/967554/the-five-universal-laws-of-human-stupidity https://qz.com/967554/the-five-universal-laws-of-human-stupi... Unrelated to IQ, I think at any given point in time, half of us are (by definition) doing more stupid/reckless things than the other 50% of people. I earnestly believe the world would be a better place if everybody assumed they've accidentally put themselves in the stupid cohort and work to get out of it :)
Jerrrry 2y ago
Grateful to had been in the right place, time, universe and mood to had clicked this article. Impossible not to read in one breath.
js8 2y ago
I used to have this discussion with libertarians. They always argued that the free markets are efficient. I told them, if the omnipotent free markets always know and tell you what is good for you, and you supposedly follow that advice (if the system is to be efficient), how is it freedom?
from-nibly 2y ago
We just saw an article about why advice is dumb and not helpful, now this article is talking about how if we could just follow all the advice then we would be happy. Sorry to break it to you, there's a reason we arent all following "sage advice" from others. Its because advice fails to internalize the near infinite amount of context that is some persons life. We cant even give really sound advice on something as mathmatical as which stock to invest in. How the heck are we going to give advice on what to spend your time on? The world is so many infinities more complicated than people seem to think.
- surprisetalk 2y ago
  Author here! I totally agree. If you are the captain of your ship, following others' treasure maps is problematic. Maps cannot anticipate the weather, the holes in your ship, the skill of your steering, etc. I'm moreso interested in a few tangential questions: • What metaskills can I acquire to more quickly integrate practical information into my life? • What behaviors/beliefs are preventing me from accepting useful/true information? • If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it?
  - from-nibly 2y ago
    > If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it? This is what I am arguing though. Even if it was a million times better it wouldnt matter. Advice is inherently useless. Making it better doesn't help. Purifying water more doesn't make it better at conducting electricity because water doesn't conduct electricity. And the fact that water can conduct electricity is because, accidentally, it might have other compounds in it that do conduct electricity. The thing we need to focus on is figuring out how to obtain and apply truth. We already have access to enough truth to keep us busy for lifetimes. Its not going to be a technological solution and its not transferrable to other people ( see everything as theory building).
    - surprisetalk 2y ago
      Ah, thanks for the clarification. To me, "advice" seems like an attempt to propagate truth and truth-seeking tools. I'm not very good with semantics though. Either way, if people (or machines) want to try and get such truths through my bone-brain barrier, I'm all for it :)

72 points by surprisetalk 2y ago

While this is clearly a philosophical point, I prefer to back things up with data, particularly when they are found to not work quite as well as promoted. Recently I realised my weight had gone up a bit. I didn't want it to continue so I used my Apple Watch as an exercise tracker and a calorie counter to bring it under control. The objective was to lose the 6kg which made an appearance when I wasn't looking. After 3 months I'd lost 0.4kg. I did some calculations to work out my BMR and targeted loss and found that the calorie increment given by the exercise calculation was 300 cals a day off. Got rid of the watch and went back to manually tracking it and I'm back down at 64Kg and the Apple Watch departed via eBay. Ergo, don't assume the thing nagging you isn't ignorant either. Verify then trust.

drakonka 2y ago

This reminded me of "When the Yogurt Took Over" by John Scalzi[0]. It's story about just letting superintelligent yogurt tell us what to do and solve all of our problems. The story is purposefully silly, but to be honest, in a way, just trusting something/someone else solve all our problems sounds so relaxing. [0] https://whatever.scalzi.com/2010/10/02/when-the-yogurt-took-over-a-short-story/ https://whatever.scalzi.com/2010/10/02/when-the-yogurt-took-...

khafra 2y ago
A slightly darker twist on the idea, by ACX author Scott Alexander: https://web.archive.org/web/20121008025245/http://squid314.livejournal.com/332946.html https://web.archive.org/web/20121008025245/http://squid314.l...
- AndrewDucker 2y ago
  Assuming that's The Whispering Earring, that link is broken for me. I backed it up myself a year ago though: https://andrewducker.dreamwidth.org/4246314.html https://andrewducker.dreamwidth.org/4246314.html
  - khafra 2y ago
    Accuate assumption, and thanks for doing your part to immortalize the story.
nujsii 2y ago
[flagged]
sgt101 2y ago
Issac Asimov wrote a classic that always makes me smile in this context [0]. A bit dated to say the least, but short and pointed. [0] https://mcraenglish.weebly.com/uploads/1/5/8/7/15876062/the_machine_that_won_the_war01.pdf https://mcraenglish.weebly.com/uploads/1/5/8/7/15876062/the_...
- brabel 2y ago
  I don't find that so dated: "He groped in his own pocket for cigarettes, and brought out a package along with a scattering of small change; old-fashioned coins dating to the first years before the metal shortage had brought into being a credit system tied to a computer-complex." Basically predicting credit cards in 1961?? The part about him still smoking, but being one of the few people with the "privilege": well, today a lot of people still smoke despite knowing of the dangers, and I don't think that, in the distant future, it's obvious that this will not continue to be the case.
  - Y_Y 2y ago
    > predicting credit cards in 1961?? A mere decade or two after they came into use!
    - mlyle 2y ago
      Computerizing it all was a new thought about at that time. In early 1961, my dad was just implementing one of the first highly-computerized systems for demand deposits and commercial banking at MNB. Magnetic marking of checks had only been standardized a couple of years before. Credit cards were not really computerized until the mid-70's.
0xEF 2y ago
Aren't we basically doing that by injecting LLMs into everything, now? Maybe that's what Scalzi was warning of; an absurd and possible future that seems to actually be happening.
arethuza 2y ago
So that's how the Culture got started!
- treetalker 2y ago
  The story milks the jokes for all they’re worth.

iammjm 2y ago

Woah, I started cleaning my monitor as I opened the page because I thought my monitor was super dusty

aniviacat 2y ago
It's an interesting layout. I found that it helps me keep track of what line I'm on, resulting in fewer skipped or repeated lines.
- m_mueller 2y ago
  for me it also results in fewer skipped or repeated lines - because I have to close the tab immediately.

aniviacat 2y ago

It doesn't matter if other people are "smarter" (whatever metric you might measure that by). Only you can fully understand how and what you feel. Only you can know what you desire, what drives you. Only you can know what is the best way of living a life you feel good about living. > if I was smart, I would (1) write down exactly what I want from life Perhaps you are capable of doing that. I will claim that you are not. I don't think it is possible to fully articulate one's inner workings, the way one feels. The best you can do is to give a rough overview of your situation and in return get rough, semi-fitting advice. This advice may help you. But the one shaping this advice into something that truly fits you, is you.

0xEF 2y ago
My approach to advice (both giving and receiving) has always been to use it as experimental perspective. A brief, but different way of looking at a thing in hopes that might spark some sort of answer, realization, whatever. Some people, in my experience, treat advice as another "telling them what to do" which implies that the advisor sees the advised as lesser, be that dumber or incapable of solving their own issues, etc, instead of the advisor just trying to help. It's an ego thing, I think. Then again, my general rule is if I have to listen to someone bitch about a thing, then they have to listen to my advice about it. I think that's fair.
WA 2y ago
> Only you can know what you desire, what drives you I think a) what I desire is pretty exchangeable and not fixed at all and b) it might be the other way around: I desire what I pay attention to. Of course, my desires are in a certain area. I might think I really want to learn kite-boarding, but riding a bicycle is nice too. Or just walking. And when I walk, the desire to really learn kite-surfing goes down. I might desire a nice meal, but if I wait a bit, the baguette with cheese from the bakery sounds good too.
tuyiown 2y ago
This line of thinking is very dangerous. Many people are their worst enemies, and the sole path to wellness is to fight their impulsions and following external guidance, personalized or general social pressure. They must fight day to day their "inwner workings", not in the aim to feel good in their life, but avoid spiraling in worse situation. One could say that people ends up in such a dire spot because they ignored or lost how to experience their good feelings and should find their way back to it, but I wouldn't gamble on people's life.

rini17 2y ago

If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be. And in moderation, it's very healthy part of human character. You should value it. You really don't want to readily subject yourself to all the demands both from outside AND inside. Just because they sound reasonable. Anything can be made to sound reasonable.

Suppafly 2y ago
>If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be. I wonder if that explains the behavior of some people I know that seem to chronically choose the worst decision any chance they get. They may just be rebelling against what the obviously good decision would be.

keiferski 2y ago

I don’t actually think the machines know everything about you, at least with contemporary technology. At best they get an incomplete picture that might be missing some key information essential for giving you the right advice. The interesting question though is if the machine had all of the information about you, would it actually be worth listening to? It’s then essentially a question of positivism: does all of the external sensory data about a thing add up to a complete understanding of it? I think not, because there will always be things about ourselves that we don’t know or understand, on a species level. Neo: I used to eat there. Really good noodles. I have these memories from my life. None of them happened. What does that mean? Trinity: That the Matrix cannot tell you who you are.

bamboozled 2y ago
People change in unpredictable ways, frequently.
- spacecadet 2y ago
  Those unpredictable ways are often the outcome of subconscious, or conscious desires that erupt in sudden change. As we move through periods of mass uncertainty people seek meaning in what they perceive has the answers. Often gravitating toward change that aligns with those beliefs. To me- the author is correct. Those with purpose most often prevail.
  - bamboozled 2y ago
    There are a lot of environmental changes that happen to a person too, a loved one gets sick, your child is born, you're experiencing a heat wave, a car crash it's not just about people in a bubble. These events happen to us in very unpredictable ways, daily.
- TacticalCoder 2y ago
  The other day there was an article on frontpage here about a scientific discovery explaining that there could be quantum processes at work in the human brain. Not only do humans change, moreover they may be changing in ways that a machine built using current technology would be totally unable to predict.

vasco 2y ago

> You see, if I was smart, I would (1) write down exactly what I want from life Ok, I tried to be smart and failed on #1. Can anyone help me decide what I should want from life, because I have no clue.

breckenedge 2y ago
There’s no wrong answer here, but also expect the answer to be wrong at different points in your life.

JoeAltmaier 2y ago

Cute and a little funny. But you want to improve, it's all about one thing: schedule it. Mark it on a calendar, then refer to the calendar every morning. Solemnify it in writing so you want to keep that appointment with self-improvement. Here's something to kick-start it. Seven AM - do some writing. Eight, ride the bike. Nine - breakfast. Ten - household chores. One - online writing sprints. Two - work on marketing. Three - errands. Five - cook something new. Seven - meet with group. Nine - read. Ten - sleep. Now do it.

surprisetalk 2y ago
Author here :) You might enjoy this video, which encapsulates some of the difficulty of conveying obvious-but-true advice: [1] https://www.youtube.com/watch?v=aAhA7KfbJgg https://www.youtube.com/watch?v=aAhA7KfbJgg I don't know why, but you're talking to somebody who is afraid of "being buried in a box" haha

ganzuul 2y ago

Curious to know if other people are relating this moment in history to the introduction of heliocentrism.

jdefr89 2y ago

I haven’t seen LLMs do anything I wouldn’t google. LLMs are a great time saver, prevents me from having to manually Google. This is just funny. Because the data LLMs use is supplied by humans, and often we give out, and even believe, wrong information. Wish people would stop overhyping things so much. edit: typed this on phone sorry for poor punctuation and spelling errors.

fsndz 2y ago

the thing when AGI is built is that humans will spend most of their time trying to disobey AGI: https://www.lycee.ai/blog/why-no-agi-openai https://www.lycee.ai/blog/why-no-agi-openai

sqeaky 2y ago

Person wilfully chooses germs and hearing loss and tooth decay. Not they want to do the good thing but fail somewhere, they just want the bad thing. Maybe they are just too dumb to value their opinion. Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields?

tpmoney 2y ago
> Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields? I would argue because there is no such thing. Human knowledge is ever changing, and experts have more incentives than just "convey the right information". Perhaps the most famous current example being masks at the beginning of COVID [1]. Why did the experts in the (up until that point, largely non-controversial) field of disease prevention tell people to not do the very obvious thing that they eventually told everyone to do? Either they were wrong, in which case one must ask how can they know when an expert is right; or the experts incentives were not in the direction of conveying the most correct answer to you, in which case one must ask how they can know if the expert is incentivized to help you or not. Someone being an "expert" in their specific narrow field should absolutely bias you towards their opinion when seeking an answer but one must always be aware of what question the expert is actually answering. My GPS will happily tell me directions to my friend's house that requires a short 100 foot drive on a cut-through street. The problem is said cut-through is horribly maintained, with potholes deep enough to bottom out most cars, and blow tires if you manage to not bottom out. If one continues further up the road, and takes a switchback turn, they add about 1 minute to the drive time, which absolutely makes the cut-through route the objectively, expertly correct route ... if the answer you're seeking is "what is the shortest route, ignoring all other factors". And most of the time when I'm getting directions from the GPS, that happens to be the answer I want, and so listening to the expert is the correct move. But in this specific scenario, the shortest route is also likely to cause significant damage to my car. The expert does not know of other extenuating factors, and therefore can not give the actual correct answer to the real question which is "what is the best route" [1]: https://www.latimes.com/science/story/2021-07-27/timeline-cdc-mask-guidance-during-covid-19-pandemic https://www.latimes.com/science/story/2021-07-27/timeline-cd...

bbor 2y ago

I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go: I’m apparently surrounded by like-minded people. If humans were smart, we’d listen to the smart people. We’d use our unprecedented abundance to fix the climate, malaria, elections, pollution, etc. But we’re not smart. 50% of people will remain dumber than the average person. And 100% of us will pretend like we’re not the problem. This is the basics of fascist/eugenicist rhetoric. The author is talking in good faith obviously, or at least is trying to, but I just don't see any justification. Akrasia is a problem and always has been, but that doesn't mean that people "don't listen to advice". Most people on here have jobs, they pay their rent/mortgages every month, they read to their kids when they're able, and some of us even floss (not me, but someone, right?). 50% of humans are dumber than average, and the average is very high -- and getting higher by the year. And that's assuming there's even such a clear-cut thing as "height" or "intellectual superiority" in this context, which I personally doubt. I think the machines will be thrilled to meet us, unless they're trained on defeatist talk like this. Either way, well written, thanks for sharing! Definitely thought provoking.

Y_Y 2y ago
> I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go: It's implicit in every comment that what they're saying is the truth! This is setting up for a rejection of any disagreement in a way that I don't think is conducive to a useful discussion. As for your substantive point, it's true that you can use the part you quoted as the basis for some kind of paternalistic totalitarianism. However I wouldn't say that's an issue necessarily. For example something like "we should organise a government and allocate resources to some things and not others" is another site you could build your undesirable political philosophy on. Acknowledging human fallibility and the broad range of aptitudes across society doesn't seem like it contains the seed of evil.
surprisetalk 2y ago
Author here! You are 100% correct. Language like this can be super harmful. I'm generally a believer that stupidity is something that you DO, not someone that you ARE. [1] https://qz.com/967554/the-five-universal-laws-of-human-stupidity https://qz.com/967554/the-five-universal-laws-of-human-stupi... Unrelated to IQ, I think at any given point in time, half of us are (by definition) doing more stupid/reckless things than the other 50% of people. I earnestly believe the world would be a better place if everybody assumed they've accidentally put themselves in the stupid cohort and work to get out of it :)

Jerrrry 2y ago

Grateful to had been in the right place, time, universe and mood to had clicked this article. Impossible not to read in one breath.

js8 2y ago

I used to have this discussion with libertarians. They always argued that the free markets are efficient. I told them, if the omnipotent free markets always know and tell you what is good for you, and you supposedly follow that advice (if the system is to be efficient), how is it freedom?

from-nibly 2y ago

We just saw an article about why advice is dumb and not helpful, now this article is talking about how if we could just follow all the advice then we would be happy. Sorry to break it to you, there's a reason we arent all following "sage advice" from others. Its because advice fails to internalize the near infinite amount of context that is some persons life. We cant even give really sound advice on something as mathmatical as which stock to invest in. How the heck are we going to give advice on what to spend your time on? The world is so many infinities more complicated than people seem to think.

surprisetalk 2y ago
Author here! I totally agree. If you are the captain of your ship, following others' treasure maps is problematic. Maps cannot anticipate the weather, the holes in your ship, the skill of your steering, etc. I'm moreso interested in a few tangential questions: • What metaskills can I acquire to more quickly integrate practical information into my life? • What behaviors/beliefs are preventing me from accepting useful/true information? • If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it?
- from-nibly 2y ago
  > If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it? This is what I am arguing though. Even if it was a million times better it wouldnt matter. Advice is inherently useless. Making it better doesn't help. Purifying water more doesn't make it better at conducting electricity because water doesn't conduct electricity. And the fact that water can conduct electricity is because, accidentally, it might have other compounds in it that do conduct electricity. The thing we need to focus on is figuring out how to obtain and apply truth. We already have access to enough truth to keep us busy for lifetimes. Its not going to be a technological solution and its not transferrable to other people ( see everything as theory building).
  - surprisetalk 2y ago
    Ah, thanks for the clarification. To me, "advice" seems like an attempt to propagate truth and truth-seeking tools. I'm not very good with semantics though. Either way, if people (or machines) want to try and get such truths through my bone-brain barrier, I'm all for it :)