13 ms·

Give AI curiosity, and it will watch TV forever (2018)

75 points by rzk 3y ago

EGreg 3y ago
* . When the agent found the TV and started flipping through the channels, the stream of new images made the TV irresistible. Edwards said there were instances when the AI could pry itself away from the TV, but only when the AI’s surroundings somehow seemed more interesting than the next thing on TV.* Sounds exactly like humans addicted to watching tiktoks and social media. Do you personally know any?
thefourthchime 3y ago
Link to the actual paper. https://pathak22.github.io/large-scale-curiosity/ https://pathak22.github.io/large-scale-curiosity/ I was curious how they define or reward curiosity, it says it right here: Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrinsic reward function which uses prediction error as reward signal. So, the prediction error is the reward, nice.
- gunshai 3y ago
  This seems highly suseptible to what a human might consider irrelevant randomness. Given random images just shuffling indefinitely a curious individual will just give up and say, even though I can't predict the next thing it doesn't pertain to the domain of curiosity.
  - dimatura 3y ago
    Right, that's why the "noisy TV problem" is a "problem" - it makes the simple model of curiosity used in the research less effective, because it causes agents to get stuck watching TV instead of exploring and advancing in the game, which is what we'd want them to do. (Though this does seem somewhat reminiscent of certain human behaviors...). One possible solution is to equip agents with a more expressive predictive model, capable of discerning "interesting" randomness from "uninteresting" randomness (for some TBD definition of "interesting", of course I'm handwaving here - I'm sure in the 5 past years there's been progress on this front).
    - jhardy54 3y ago
      First thing that popped into my head: “interesting randomness” is when you can’t predict future frames, but [within some interval] you gain that ability. Static on the TV is random but uninteresting, whereas morse code is “random” at first, but after enough exposure can be understood and predicted.
shp0ngle 3y ago
2018
- politelemon 3y ago
  Adding the year to the title would be relevant here, because 'AI' in 2023/2024 is a very different context and it's a very different landscape now
  - yamrzou 3y ago
    Added
syntaxing 3y ago
With the definition of curiosity from the article, it’s not that surprising? A dynamic “screen” is always more interesting than the static map. Definition: The definition that OpenAI team used for artificial curiosity was relatively simple: The algorithm would try to predict what its environment would look like one frame into the future. When that next frame happened, the algorithm would be rewarded by how wrong it was. The idea is that if the algorithm could predict what would happen in the environment, it had seen it before.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- hahajk 3y ago
  The researchers quoted were explicit about this: > OpenAI researcher Harri Edwards tells Quartz that the idea for letting the AI agent flip through channels came from a thought experiment called the noisy-TV problem. The static on a TV is immensely random, so a curious AI agent could never truly predict what would happen next, and get drawn into watching the TV forever. In the real world, you could think of it as something completely random, like the way light shimmers off a waterfall. The headline is really just inappropriate anthropomorphization.
  - adammarples 3y ago
    Perhaps it should use prediction error on some higher level of embedding, that way boring changes like static would be treated similarly but genuinely novel things would be treated higher.
  - p1mrx 3y ago
    The article seems pretty incoherent. It's not clear if the AI was watching static or actual TV content. If it's static, then why bother flipping through channels?
- gunshai 3y ago
orenlindsey 3y ago
Kind of reveals how we have designed television - to keep you watching forever.
- janalsncm 3y ago
  TV channels are designed to keep you engaged, but this agent only wanted to flip through the channels. In fact the agent was most interested in TV static since it satiated its definition of “curiosity” best.
- grotorea 3y ago
  Maybe someday the AI will be clever enough to predict the plotlines and get bored.
  - rand1239 3y ago
    Why? You know the plotline of your life as well. You will become old, suffer and die at the end. Are you bored?
    - grotorea 3y ago
      Maybe but with my life the plot twists don't have to be foreshadowed or make dramatic sense!
velcrovan 3y ago
I think the “Murderbot Diaries” series by Martha Wells anticipates and depicts this very well. https://en.wikipedia.org/wiki/The_Murderbot_Diaries https://en.wikipedia.org/wiki/The_Murderbot_Diaries Anyway, read the first one, you’ll be hooked.
slowmovintarget 3y ago
Now the question is, how do you balance that with a simulation of patience, which ought to run out?
throwaway4aday 3y ago
Using prediction error as the definition of curiosity rings hollow for me. Curiosity in my mind is more about mapping out an unexplored thing and not about being surprised.
- annowiki 3y ago
  Actually it seems pretty accurate. Novelty-seeking is a well known phenomenon in curious individuals. https://en.wikipedia.org/wiki/Novelty_seeking https://en.wikipedia.org/wiki/Novelty_seeking Literally getting dopamine rewards for seeing something new is what keeps people glued to tik tok feeds and twitter. I tend to get bored halfway through a book if it is predictable.
- samatman 3y ago
  It's not a complete definition for a number of reasons, but it's a crucial component of curiousity. If you're not surprised at any point in mapping out an unexplored thing, in what sense is it unexplored? There are pretty high odds you've never been to this exact page before: https://oeis.org/A000079 https://oeis.org/A000079. But once you click on it, is there any remaining curiosity? It's an unexplored thing, in the sense that (I presume) you've never looked at this exact page before. But it doesn't provoke curiousity because there's nothing there to surprise you.
  - kuchenbecker 3y ago
    When I go hiking I expect trees and hills. Despite the expectation, it's still novel / sublime finding a new vista or path
  - throwaway4aday 3y ago
    True, I was thinking this over and I can see where surprise plays a part in curiosity but I don't think it's the main driver. I think your example shows this because when I click on the link out of curiosity it's because I don't have a good prediction for what will be on the other end of it. If I were a simple neural net then the outputs I would produce when seeing that link would be mostly similar across all the categories I could predict since I've never seen this url and there is little in it that allows me to predict what type of content it leads to. I am surprised to see the link since I couldn't predict it, so that's a point in favor of this approach, and I am also surprised by the content I see after clicking on it although only mildly because I didn't have a good prediction to start with. You're right that after I click on it I have very little curiosity to find out more about it but I think that is the key difference. The rest of that site remains an unexplored place, I've only seen one page of it which it seems would be a very small fraction of all of it. So why am I not curious about the rest of it and more crucially, what would make me curious to explore it? I could imagine someone with a particular love of mathematics being able to exercise their curiosity on that page but is that because they would be surprised by what they found after clicking on each link or entering a new sequence? What would drive them to explore it? I think that speaks to the problem, if you optimize for surprise then the ideal reward is paying attention to an infinite number of TV channels (or a page with a bunch of links you've never seen before leading to different pages you've never seen) but I wouldn't call that curiosity. I'm trying to imagine the simplest case, say a button you could press and every time you pressed it something entirely random would happen, always guaranteeing surprise. It would have a great deal of novelty at first but after a while it would cease to hold your attention even though your prediction of what would happen would never be accurate. I'd bet that after a while you might even never bother to push it again. The only way you would be convinced to push it consistently would be a) if you were assigned a reward for pushing it e.g. money in which case it is a slot machine or b) if by pushing it you could somehow reduce your uncertainty about what would happen which would as a by product reduce your surprise. Thinking about it this way, surprise is certainly a key element at first. It grabs your attention initially but it doesn't hold it. What keeps you focused on exploring the thing that surprised you initially is the learning process which involves reducing prediction error i.e. reducing surprise. So there is a tension between the two. The combination probably makes for a good exploration strategy. Initial surprise, look for a learnable pattern and follow it until another surprise, maybe backtrack and try other familiar patterns until those are exhausted and then investigate each sequence that led to a surprise by recursing through these steps. This would also explain the example where my curiosity was prompted by the unknown link but I was not motivated to explore further. The website wasn't interesting to me because it was too unfamiliar and I wasn't able to find any familiar routes to explore through it due to my lack of interest in that area of mathematics but our hypothetical mathematician with a fondness for integers would see lots of familiar patterns they could explore attached to which are likely some enticingly unknown and surprising links. Thanks for the prompt to think about this more!
bjnewman85 3y ago
as mentioned elsewhere this is from nearly six years ago and uses a very crude model for curiosity - and poses this as some sort of unsolveable problem instead of a decision made by the researchers in order to investigate particular behaviors of the systems they were working with. It is a fun thought experiment - how do our brains systems manage to reward seeking new information without getting trapped by simplistic pseudo-RNG patterns in nature
m3kw9 3y ago
This is similar to how human minds work, things are bland when it’s predictable like stories, work like factories. We would tend to avoid it
- dguest 3y ago
  Except that when things work like factorio we love it. There's kind of a sweet spot when humans discover something predicable but not downright repetitive.
photochemsyn 3y ago
Instead of TV, weather satellite feeds, and other streams of real-time constantly changing earth system-type data? Also, stock market prices, but I imagine a whole lot of effort is already quietly going into that at present.
kemiller 3y ago
Can confirm this works for humans too.
aantix 3y ago
This trait must serve us in some capacity. You see this with children and iPads. My kids know 3x what I did when I was their same age. It's as if something can finally fill their bandwidth, keep up with their thoughts, and answer those questions in real-time. No wonder they get angry when it's time to put away the iPad - they're engaged at a high level. Much like when I am programming on a sideproject and my focus is interrupted. The biggest travesty is that we don't get more control over Youtube's algorithm. That we can't ban all Youtube videos with the word "Minecraft" in it, so that we can override their curiosity and say "OK, learn something else today." Youtube's focus solely on engagement above all else is the biggest tragedy of the modern world. Such a massive opportunity for education - e.g. inserting more academic videos in between entertainment. But we don't get that feature, or any manual overrides for more control. If any Youtube engineers are reading, please give us (the parents) more control! Youtube, do better.
- radium3d 3y ago
  I would also like this for myself. Lol
  - eimrine 3y ago
    LOL guys isn't that so hard not to watch dumb or short or news videos from your main account? My recommendations are never game videos, and rarely less than 1 hour videos because guess what? I seldomly watch videos which are less than 1 hour and never watch games.
    - retrac 3y ago
      On Youtube lately, I have lately mostly been watching videos on metal smelting and ancient Mesopotamia. There are conspiracies about these topics, apparently. I wouldn't have thought so. But the Youtube algorithm manages to dredge them up.
- ok123456 3y ago
  TikTok has this switch. There's a STEM mode in the user settings that will only show you educational content on your algorithmic stream.
blt 3y ago
The notion of curiosity by prediction error can probably be refined with some information theoretic quantities to eventually realize that new random samples of TV static are not surprising.
deleted 3y ago
[deleted]
Dxtros 3y ago
This reminds me of the murder bot book series by Martha Wells. The main character (an advanced ai robot) started really enjoying human media and even used media as a bargaining chip to work with other bots. One striking moment was when the main character was discussing a rogue and violent robot with a transport bot. “ART said, What does it want? To kill all the humans, I answered. I could feel ART metaphorically clutch its function. If there were no humans, there would be no crew to protect and no reason to do research and fill its databases. It said, That is irrational. I know, I said, if the humans were dead, who would make the media? It was so outrageous, it sounded like something a human would say.” -Martha Wells, Artificial Condition Although fiction, it’s very thought provoking in evaluating where a truly sentient AI might place its motives. On one hand the research transport bot (ART) is motivated to protect its humans because it would be functionless without them. While the main character (a security unit, who is typically treated badly by humans) sarcastically but partially truthfully places its motives to not kill humans in funding its curiosity of TV. Would implementing curiosity in a sentient AI act as a safeguard possibly? Would curiosity arise as a byproduct of sentience without being directly programmed?
- aquova 3y ago
  I've been reading through the series this week and this is the first thing that popped into my mind. A series about an AI who doesn't care about its job or its clients, and achieves a level of personal liberation by hacking itself, just so it could download TV shows and watch them when no one was looking.
- gary_0 3y ago
  Curiosity as a safeguard is an interesting thought. An AI might be disinclined to kill all humans for whatever reason if it considers the result boring. On the other hand, maybe it would also want to force humans to be more interesting for its own entertainment...
- thfuran 3y ago
  >Would implementing curiosity in a sentient AI act as a safeguard possibly? Unless it gets curious about the variety of sounds humans make when you vivisect them or something else you'd prefer not be rigorously investigated.
yamrzou 3y ago
Related: Reinforcement Learning with Prediction-Based Rewards — https://news.ycombinator.com/item?id=18346943 https://news.ycombinator.com/item?id=18346943 — Oct 2018 (38 comments)
t3estabc 3y ago
[dead]
blurbleblurble 3y ago
This sounds like one of the more potentially "dangerous" endowments to give a machine.
wantsanagent 3y ago
The important counterweight to this phenomenon is the brain's adaptability to noise. Some ML researchers like to think that noise is not predictable because this follows from the classic CS definition of noise. However in reality the brain quickly adapts to any noisy sensory input, it begins to predict higher level characteristics of the noisy input and no longer reacts with surprise or interest. This happens at all levels of sensory processing, from single cell firing (which is noisy) to the boredom you feel with a 100 channels of TV that are all technically novel to you but contain nothing remotely interesting. Basically if you've built an agent that can be perpetually distracted by noise or a "noisy" TV then you've forgotten an important piece of the puzzle.
- sdl 3y ago
  I like the Bayesian Surprise definition for this. It's not about predicting the exact next state of the world (or the next frame of the noisy TV) but about how much the next state changes your model of the world. https://papers.nips.cc/paper_files/paper/2005/hash/0172d289da48c48de8c5ebf3de9f7ee1-Abstract.html https://papers.nips.cc/paper_files/paper/2005/hash/0172d289d...
tehsauce 3y ago
I observed this phenomena not too long ago while training an RL agent to play pokemon using an intrinsic novelty reward based on the number of unique screens the agent had seen. The goal was to get it to explore the map, but instead it became fixated watching animated water, flowers, and npcs in the starting town. I made an extensive video analyzing the experiments, you can find it by searching "ai plays pokemon" :)
esafak 3y ago
> OpenAI researcher Harri Edwards tells Quartz that the idea for letting the AI agent flip through channels came from a thought experiment called the noisy-TV problem. The static on a TV is immensely random, so a curious AI agent could never truly predict what would happen next, and get drawn into watching the TV forever. In the real world, you could think of it as something completely random, like the way light shimmers off a waterfall... This is a great thought experiment but they're using the wrong metric. Animals are wired to seek information, not noise. We understand that there is nothing to be learned by absorbing noise.
massysett 3y ago
Was it curious but not learning? If the AI learns, it will not watch TV forever, because most TV is predictable. There is the cop show, the lawyer show, the doctor show, the news, the family sitcom, etc etc. Eventually it would learn all these and find the TV less interesting - which is exactly what happens to many people.
deleted 3y ago
[deleted]

75 points by rzk 3y ago

* . When the agent found the TV and started flipping through the channels, the stream of new images made the TV irresistible. Edwards said there were instances when the AI could pry itself away from the TV, but only when the AI’s surroundings somehow seemed more interesting than the next thing on TV.* Sounds exactly like humans addicted to watching tiktoks and social media. Do you personally know any?

thefourthchime 3y ago

Link to the actual paper. https://pathak22.github.io/large-scale-curiosity/ https://pathak22.github.io/large-scale-curiosity/ I was curious how they define or reward curiosity, it says it right here: Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrinsic reward function which uses prediction error as reward signal. So, the prediction error is the reward, nice.

gunshai 3y ago
This seems highly suseptible to what a human might consider irrelevant randomness. Given random images just shuffling indefinitely a curious individual will just give up and say, even though I can't predict the next thing it doesn't pertain to the domain of curiosity.
- dimatura 3y ago
  Right, that's why the "noisy TV problem" is a "problem" - it makes the simple model of curiosity used in the research less effective, because it causes agents to get stuck watching TV instead of exploring and advancing in the game, which is what we'd want them to do. (Though this does seem somewhat reminiscent of certain human behaviors...). One possible solution is to equip agents with a more expressive predictive model, capable of discerning "interesting" randomness from "uninteresting" randomness (for some TBD definition of "interesting", of course I'm handwaving here - I'm sure in the 5 past years there's been progress on this front).
  - jhardy54 3y ago
    First thing that popped into my head: “interesting randomness” is when you can’t predict future frames, but [within some interval] you gain that ability. Static on the TV is random but uninteresting, whereas morse code is “random” at first, but after enough exposure can be understood and predicted.

shp0ngle 3y ago

2018

politelemon 3y ago
Adding the year to the title would be relevant here, because 'AI' in 2023/2024 is a very different context and it's a very different landscape now
- yamrzou 3y ago
  Added

syntaxing 3y ago

With the definition of curiosity from the article, it’s not that surprising? A dynamic “screen” is always more interesting than the static map. Definition: The definition that OpenAI team used for artificial curiosity was relatively simple: The algorithm would try to predict what its environment would look like one frame into the future. When that next frame happened, the algorithm would be rewarded by how wrong it was. The idea is that if the algorithm could predict what would happen in the environment, it had seen it before.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
hahajk 3y ago
The researchers quoted were explicit about this: > OpenAI researcher Harri Edwards tells Quartz that the idea for letting the AI agent flip through channels came from a thought experiment called the noisy-TV problem. The static on a TV is immensely random, so a curious AI agent could never truly predict what would happen next, and get drawn into watching the TV forever. In the real world, you could think of it as something completely random, like the way light shimmers off a waterfall. The headline is really just inappropriate anthropomorphization.
- adammarples 3y ago
  Perhaps it should use prediction error on some higher level of embedding, that way boring changes like static would be treated similarly but genuinely novel things would be treated higher.
- p1mrx 3y ago
  The article seems pretty incoherent. It's not clear if the AI was watching static or actual TV content. If it's static, then why bother flipping through channels?
gunshai 3y ago

orenlindsey 3y ago

Kind of reveals how we have designed television - to keep you watching forever.

janalsncm 3y ago
TV channels are designed to keep you engaged, but this agent only wanted to flip through the channels. In fact the agent was most interested in TV static since it satiated its definition of “curiosity” best.
grotorea 3y ago
Maybe someday the AI will be clever enough to predict the plotlines and get bored.
- rand1239 3y ago
  Why? You know the plotline of your life as well. You will become old, suffer and die at the end. Are you bored?
  - grotorea 3y ago
    Maybe but with my life the plot twists don't have to be foreshadowed or make dramatic sense!

velcrovan 3y ago

I think the “Murderbot Diaries” series by Martha Wells anticipates and depicts this very well. https://en.wikipedia.org/wiki/The_Murderbot_Diaries https://en.wikipedia.org/wiki/The_Murderbot_Diaries Anyway, read the first one, you’ll be hooked.

slowmovintarget 3y ago

Now the question is, how do you balance that with a simulation of patience, which ought to run out?

throwaway4aday 3y ago

Using prediction error as the definition of curiosity rings hollow for me. Curiosity in my mind is more about mapping out an unexplored thing and not about being surprised.

annowiki 3y ago
Actually it seems pretty accurate. Novelty-seeking is a well known phenomenon in curious individuals. https://en.wikipedia.org/wiki/Novelty_seeking https://en.wikipedia.org/wiki/Novelty_seeking Literally getting dopamine rewards for seeing something new is what keeps people glued to tik tok feeds and twitter. I tend to get bored halfway through a book if it is predictable.
samatman 3y ago
It's not a complete definition for a number of reasons, but it's a crucial component of curiousity. If you're not surprised at any point in mapping out an unexplored thing, in what sense is it unexplored? There are pretty high odds you've never been to this exact page before: https://oeis.org/A000079 https://oeis.org/A000079. But once you click on it, is there any remaining curiosity? It's an unexplored thing, in the sense that (I presume) you've never looked at this exact page before. But it doesn't provoke curiousity because there's nothing there to surprise you.
- kuchenbecker 3y ago
  When I go hiking I expect trees and hills. Despite the expectation, it's still novel / sublime finding a new vista or path
- throwaway4aday 3y ago
  True, I was thinking this over and I can see where surprise plays a part in curiosity but I don't think it's the main driver. I think your example shows this because when I click on the link out of curiosity it's because I don't have a good prediction for what will be on the other end of it. If I were a simple neural net then the outputs I would produce when seeing that link would be mostly similar across all the categories I could predict since I've never seen this url and there is little in it that allows me to predict what type of content it leads to. I am surprised to see the link since I couldn't predict it, so that's a point in favor of this approach, and I am also surprised by the content I see after clicking on it although only mildly because I didn't have a good prediction to start with. You're right that after I click on it I have very little curiosity to find out more about it but I think that is the key difference. The rest of that site remains an unexplored place, I've only seen one page of it which it seems would be a very small fraction of all of it. So why am I not curious about the rest of it and more crucially, what would make me curious to explore it? I could imagine someone with a particular love of mathematics being able to exercise their curiosity on that page but is that because they would be surprised by what they found after clicking on each link or entering a new sequence? What would drive them to explore it? I think that speaks to the problem, if you optimize for surprise then the ideal reward is paying attention to an infinite number of TV channels (or a page with a bunch of links you've never seen before leading to different pages you've never seen) but I wouldn't call that curiosity. I'm trying to imagine the simplest case, say a button you could press and every time you pressed it something entirely random would happen, always guaranteeing surprise. It would have a great deal of novelty at first but after a while it would cease to hold your attention even though your prediction of what would happen would never be accurate. I'd bet that after a while you might even never bother to push it again. The only way you would be convinced to push it consistently would be a) if you were assigned a reward for pushing it e.g. money in which case it is a slot machine or b) if by pushing it you could somehow reduce your uncertainty about what would happen which would as a by product reduce your surprise. Thinking about it this way, surprise is certainly a key element at first. It grabs your attention initially but it doesn't hold it. What keeps you focused on exploring the thing that surprised you initially is the learning process which involves reducing prediction error i.e. reducing surprise. So there is a tension between the two. The combination probably makes for a good exploration strategy. Initial surprise, look for a learnable pattern and follow it until another surprise, maybe backtrack and try other familiar patterns until those are exhausted and then investigate each sequence that led to a surprise by recursing through these steps. This would also explain the example where my curiosity was prompted by the unknown link but I was not motivated to explore further. The website wasn't interesting to me because it was too unfamiliar and I wasn't able to find any familiar routes to explore through it due to my lack of interest in that area of mathematics but our hypothetical mathematician with a fondness for integers would see lots of familiar patterns they could explore attached to which are likely some enticingly unknown and surprising links. Thanks for the prompt to think about this more!

bjnewman85 3y ago

as mentioned elsewhere this is from nearly six years ago and uses a very crude model for curiosity - and poses this as some sort of unsolveable problem instead of a decision made by the researchers in order to investigate particular behaviors of the systems they were working with. It is a fun thought experiment - how do our brains systems manage to reward seeking new information without getting trapped by simplistic pseudo-RNG patterns in nature

m3kw9 3y ago

This is similar to how human minds work, things are bland when it’s predictable like stories, work like factories. We would tend to avoid it

dguest 3y ago
Except that when things work like factorio we love it. There's kind of a sweet spot when humans discover something predicable but not downright repetitive.

photochemsyn 3y ago

Instead of TV, weather satellite feeds, and other streams of real-time constantly changing earth system-type data? Also, stock market prices, but I imagine a whole lot of effort is already quietly going into that at present.

kemiller 3y ago

Can confirm this works for humans too.

aantix 3y ago

This trait must serve us in some capacity. You see this with children and iPads. My kids know 3x what I did when I was their same age. It's as if something can finally fill their bandwidth, keep up with their thoughts, and answer those questions in real-time. No wonder they get angry when it's time to put away the iPad - they're engaged at a high level. Much like when I am programming on a sideproject and my focus is interrupted. The biggest travesty is that we don't get more control over Youtube's algorithm. That we can't ban all Youtube videos with the word "Minecraft" in it, so that we can override their curiosity and say "OK, learn something else today." Youtube's focus solely on engagement above all else is the biggest tragedy of the modern world. Such a massive opportunity for education - e.g. inserting more academic videos in between entertainment. But we don't get that feature, or any manual overrides for more control. If any Youtube engineers are reading, please give us (the parents) more control! Youtube, do better.

radium3d 3y ago
I would also like this for myself. Lol
- eimrine 3y ago
  LOL guys isn't that so hard not to watch dumb or short or news videos from your main account? My recommendations are never game videos, and rarely less than 1 hour videos because guess what? I seldomly watch videos which are less than 1 hour and never watch games.
  - retrac 3y ago
    On Youtube lately, I have lately mostly been watching videos on metal smelting and ancient Mesopotamia. There are conspiracies about these topics, apparently. I wouldn't have thought so. But the Youtube algorithm manages to dredge them up.
ok123456 3y ago
TikTok has this switch. There's a STEM mode in the user settings that will only show you educational content on your algorithmic stream.

blt 3y ago

The notion of curiosity by prediction error can probably be refined with some information theoretic quantities to eventually realize that new random samples of TV static are not surprising.

deleted 3y ago

[deleted]

Dxtros 3y ago

This reminds me of the murder bot book series by Martha Wells. The main character (an advanced ai robot) started really enjoying human media and even used media as a bargaining chip to work with other bots. One striking moment was when the main character was discussing a rogue and violent robot with a transport bot. “ART said, What does it want? To kill all the humans, I answered. I could feel ART metaphorically clutch its function. If there were no humans, there would be no crew to protect and no reason to do research and fill its databases. It said, That is irrational. I know, I said, if the humans were dead, who would make the media? It was so outrageous, it sounded like something a human would say.” -Martha Wells, Artificial Condition Although fiction, it’s very thought provoking in evaluating where a truly sentient AI might place its motives. On one hand the research transport bot (ART) is motivated to protect its humans because it would be functionless without them. While the main character (a security unit, who is typically treated badly by humans) sarcastically but partially truthfully places its motives to not kill humans in funding its curiosity of TV. Would implementing curiosity in a sentient AI act as a safeguard possibly? Would curiosity arise as a byproduct of sentience without being directly programmed?

aquova 3y ago
I've been reading through the series this week and this is the first thing that popped into my mind. A series about an AI who doesn't care about its job or its clients, and achieves a level of personal liberation by hacking itself, just so it could download TV shows and watch them when no one was looking.
gary_0 3y ago
Curiosity as a safeguard is an interesting thought. An AI might be disinclined to kill all humans for whatever reason if it considers the result boring. On the other hand, maybe it would also want to force humans to be more interesting for its own entertainment...
thfuran 3y ago
>Would implementing curiosity in a sentient AI act as a safeguard possibly? Unless it gets curious about the variety of sounds humans make when you vivisect them or something else you'd prefer not be rigorously investigated.

yamrzou 3y ago

Related: Reinforcement Learning with Prediction-Based Rewards — https://news.ycombinator.com/item?id=18346943 https://news.ycombinator.com/item?id=18346943 — Oct 2018 (38 comments)

t3estabc 3y ago

[dead]

blurbleblurble 3y ago

This sounds like one of the more potentially "dangerous" endowments to give a machine.

wantsanagent 3y ago

The important counterweight to this phenomenon is the brain's adaptability to noise. Some ML researchers like to think that noise is not predictable because this follows from the classic CS definition of noise. However in reality the brain quickly adapts to any noisy sensory input, it begins to predict higher level characteristics of the noisy input and no longer reacts with surprise or interest. This happens at all levels of sensory processing, from single cell firing (which is noisy) to the boredom you feel with a 100 channels of TV that are all technically novel to you but contain nothing remotely interesting. Basically if you've built an agent that can be perpetually distracted by noise or a "noisy" TV then you've forgotten an important piece of the puzzle.

sdl 3y ago
I like the Bayesian Surprise definition for this. It's not about predicting the exact next state of the world (or the next frame of the noisy TV) but about how much the next state changes your model of the world. https://papers.nips.cc/paper_files/paper/2005/hash/0172d289da48c48de8c5ebf3de9f7ee1-Abstract.html https://papers.nips.cc/paper_files/paper/2005/hash/0172d289d...

tehsauce 3y ago

I observed this phenomena not too long ago while training an RL agent to play pokemon using an intrinsic novelty reward based on the number of unique screens the agent had seen. The goal was to get it to explore the map, but instead it became fixated watching animated water, flowers, and npcs in the starting town. I made an extensive video analyzing the experiments, you can find it by searching "ai plays pokemon" :)

esafak 3y ago

> OpenAI researcher Harri Edwards tells Quartz that the idea for letting the AI agent flip through channels came from a thought experiment called the noisy-TV problem. The static on a TV is immensely random, so a curious AI agent could never truly predict what would happen next, and get drawn into watching the TV forever. In the real world, you could think of it as something completely random, like the way light shimmers off a waterfall... This is a great thought experiment but they're using the wrong metric. Animals are wired to seek information, not noise. We understand that there is nothing to be learned by absorbing noise.

massysett 3y ago

Was it curious but not learning? If the AI learns, it will not watch TV forever, because most TV is predictable. There is the cop show, the lawyer show, the doctor show, the news, the family sitcom, etc etc. Eventually it would learn all these and find the TV less interesting - which is exactly what happens to many people.

deleted 3y ago

[deleted]