6 ms·

The reason this problem keeps popping up is due to the mistaken belief that you can deduce intent via inferred behaviors, and worse that deduced intent is prefe

The reason this problem keeps popping up is due to the mistaken belief that you can deduce intent via inferred behaviors, and worse that deduced intent is preferable (or at least comparable) to explicit intent. You can't and it isn't. As an example of deduced intent let's envision a system where every action I commit over the course of a year can be observed and recorded for inference analysis to determine if I am evil or good. Assume that in all ways I meet the standard definition of good... I help people across the street, save puppies, etc. At the end of the year the BEST that you can conclude is that I behaved ethically (good actions), but that says nothing about MY intentions. I may in fact be evil and am only ingratiating myself within a community in order to later kill them all. There is a reason that within philosophy that morality and ethics are separate words. There is a reason that when viewing demographic data people say "I can tell you what 9 out of 10 people in the group prefer, but I can say nothing about any one individual within it." As to the preference between inferred vs explicit intent, you're placing your own judgement of the value of inference vs explicit above that of the user which will inevitably lead to frustrations on the part of the user. In the simple case of a catalog there are two distinct intent patterns that users engage in... one is the specific intent to find a known product and the other is what I refer to as "discovery." Discovery is, as the word suggests, finding things that the user did not know existed or knew very little about.

2sk21 9y ago
Very well articulated. I wonder if there is a difference between a machine trying to deduce intent vs a human doing the same task. As humans, we may be able to pick up subtle cues that machines simply lack the ability for.
- dragonwriter 9y ago
  > I wonder if there is a difference between a machine trying to deduce intent vs a human doing the same task. Fundamentally, no. There may be practical difference between specific machines and people. > As humans, we may be able to pick up subtle cues that machines simply lack the ability for. Depending on the machine, the reverse could be true.
- bilbo0s 9y ago
  "...As humans, we may be able to pick up subtle cues that machines simply lack the ability for..." But now we're back to the problem of deducing intent. Only now we have to deduce the intent of the human who deduces intent. (Back to the problem of "who watches the watchers?")
- fallous 9y ago
  The problem isn't the one doing the inferring, it's the inferring.
bsbechtel 9y ago
This is ultimately what will limit AI's impact/growth. Most people aren't comfortable with the idea of giving up their own judgement to anything else - other humans or machines.
- fallous 9y ago
  Yes, mostly because an external judgement that doesn't actually know their intent isn't as good for the user as their own judgement.
- quadrangle 9y ago
  Or AI will grow despite this, and the result will be powerful AI without any solution to the alignment-problem, thus being overall vastly damaging.
eksemplar 9y ago
It’s certainly interesting that in sciences like history, determinism is only plausible when you know very little. The more you know the harder it becomes to figure out why something happened. News feed prediction behaves the opposite of this. It tries to predict your intent based on past behavior, but that is, as you say, not possible, and could likely be why the news feed is dying.
- fallous 9y ago
  Behold the wisdom of the disclaimer "past performance is no guarantee of future results."
  - chiefalchemist 9y ago
    At the tree level, yes. At the forest level most people are predictable, often to a fault.
    - fallous 9y ago
      Of course, and yet there is a reason that demographics is useful only when dealing with aggregates... it tells you nothing about a specific individual. Trying to use inferred data from the behaviors of millions of people is quite useful in discovering certain aggregate info, but using it in place of a specific individual's chosen intent is an excellent way to lose a user.
- padobson 9y ago
  I wonder if another dimension is useful. YouTube's UI prominently features search. So if my interests change and I search for something I've never been interested before, YouTube can show me a pretty good selection of videos and tune its recommendations for future viewing. I will say, though, that all social networks seem to fail at weighting past behavior too strongly. Whenever I start a new account on a social network, I find it's recommendations are much more tuned to my interests, and those recommendations become less relevant overtime. Biologically, I'm a new person every seven years. As a collection of varied interests, it seems like I turnover much more often than that - and I suspect the rate of turnover is different for everyone.
Tyrek 9y ago
The challenge here is that there's equal and opposing problem - the social sciences (Econ, Psych, etc.) have established that what people say (expressed preferences) and how they act (implied preferences) are inconsistent. So you're adding an additional dimension here that further complicates the calculation.
- naravara 9y ago
  >the social sciences (Econ, Psych, etc.) have established that what people say (expressed preferences) and how they act (implied preferences) are inconsistent. This is obvious enough. But if I tell you I would like to lose weight and improve my diet, yet you, knowing I have a sweet-tooth, opt to keep offering tempting sweets and desserts anyway, that would make you kind of a jerk wouldn't it? I wouldn't want to spend much time around someone who actively tries to undermine my attempts at personal growth or improvement.
  - bo1024 9y ago
    That's a nice example.
  - AndrewKemendo 9y ago
    knowing I have a sweet-tooth, opt to keep offering tempting sweets and desserts anyway No, it means you aren't actually doing what you say you want to do. If you however gave the system input about wanting to lose weight, say by buying weight loss supplements, exercise clothes etc... then the "system" will adjust to give you what you want. Your example is basically akin to a scenario where you frequently order dessert at your favorite restaurant, then decide you want to lose weight, never tell anyone you want to lose weight, then get upset when the server brings you the dessert menu.
    - majormajor 9y ago
      > Your example is basically akin to a scenario where you frequently order dessert at your favorite restaurant, then decide you want to lose weight, never tell anyone you want to lose weight, then get upset when the server brings you the dessert menu. Except for the part where he said "if I tell you I would like to lose weight and improve my diet"? So not like the "never tell anyone" part at all? What he's describing is more like a system which learns "most people who research and buy diet stuff end up failing and going back to the ice cream and junk food, so let's start aggressively targeting them with unhealthy crap once they start looking up diets." That may be accurate regarding behavior and revenue maximization, but it's also pretty shitty.
tootie 9y ago
I was never all that into social media at all, but the thing that made me really quit was that the only things I ever clicked from my FB feed we're suggested posts. Their algorithm knew me better than my friends.
- gluelogic 9y ago
  Maybe, maybe not. A key difference is your friends are posting for themselves, but the suggestions are posts tailored for _you_.
AndrewKemendo 9y ago
the mistaken belief that you can deduce intent via inferred behaviors Maybe this is just a phrasing error, however the idea of inferred behavior doesn't make sense. If you view an action that is measurable (buy x, go to y) then it is an explicit behavior. Further, the concept you seem to be arguing against is the economic concept of revealed preferences. If I understand your argument, it is that it is impossible to derive a users preference from observing the users behavior. This is contrary to decades of empirical research in economics and cognitive science proving that you can rank preferences based on such behavioral profiling. Your example builds a strawman by introducing a value judgement into the example (evil vs good) that is unmeasurable. It conflates some kind of moral state with a series of actions without any way to empirically value them. The reality is that most people cannot accurately state what their true preferences actually are, and thus your actions reveal more about your future behavior than your "intentions" do. This is a classic debate in philosophy of the mind.
- fallous 9y ago
  Yeah, it's a phrasing error... although there are species of inferred behaviors (he went into a room for an hour and then came out and walked to a store. I don't know what he did in that room but I can infer that he breathed while in it, although as with all inferences that may be untrue.) I'm not arguing against revealed preferences as useful aggregate information regarding the probabilities of someone choosing X, I'm saying that you both can't know their intentions regarding those choices and more specifically those preferences are untrustworthy in predicting any single opportunity to choose X. The idiocy of many current products, be they physical or informational, is replacing the user's ability to make a choice with what the product designers believe... especially without explicit agreement. In most cases the user didn't opt-in to outsourcing or subordinating their decision-making to product X and every time their intent conflicts with the product's decision-making the user will become more discontented with the product.
- fallous 9y ago
  And no it's not a strawman to introduce my intentions into the example... it's the entire point. Measure everything you like with the exception of my brain and you cannot know my intentions which means that you lack the necessary information to substitute your own judgement or decision-making for mine.
Jedd 9y ago
You're describing Simpson's Paradox. https://en.m.wikipedia.org/wiki/Simpson%27s_paradox?wprov=sfla1 https://en.m.wikipedia.org/wiki/Simpson%27s_paradox?wprov=sf...
- fallous 9y ago
  No, Simpson's Paradox involves incorrect conclusions of statistical data/trend due to a mistake in viewpoint or assumption. I'm saying that even if you have the correct conclusion that it's not a valid replacement for the user's explicit intent or choice because a) the probability on which a prediction, even a ludicrously accurate one, depends makes no guarantee about the results of a single event... and b) that usurping the user's choice because your model, or you, think you know better is a stunningly bad idea in itself.
  - Jedd 9y ago
    > No, Simpson's Paradox involves incorrect conclusions of statistical data/trend due to a mistake in viewpoint or assumption. Not correct. There's no 'mistake in viewpoint or assumption' involved in Simpson's Paradox. Your example story was of someone committing 'a sequence of good deeds throughout the year' that may be part of a grander, longer plan to commit a profoundly not good deed. Analysis of month by month, grouped or in isolation, would reveal an apparent trend that would be opposite to the evident overall trend once you include the ultimate, negative act. From the wikipedia link I provided earlier: "Simpson's paradox ... is a phenomenon in probability and statistics, in which a trend appears in several different groups of data but disappears or reverses when these groups are combined."

2sk21 9y ago

Very well articulated. I wonder if there is a difference between a machine trying to deduce intent vs a human doing the same task. As humans, we may be able to pick up subtle cues that machines simply lack the ability for.

dragonwriter 9y ago
> I wonder if there is a difference between a machine trying to deduce intent vs a human doing the same task. Fundamentally, no. There may be practical difference between specific machines and people. > As humans, we may be able to pick up subtle cues that machines simply lack the ability for. Depending on the machine, the reverse could be true.
bilbo0s 9y ago
"...As humans, we may be able to pick up subtle cues that machines simply lack the ability for..." But now we're back to the problem of deducing intent. Only now we have to deduce the intent of the human who deduces intent. (Back to the problem of "who watches the watchers?")
fallous 9y ago
The problem isn't the one doing the inferring, it's the inferring.

bsbechtel 9y ago

This is ultimately what will limit AI's impact/growth. Most people aren't comfortable with the idea of giving up their own judgement to anything else - other humans or machines.

fallous 9y ago
Yes, mostly because an external judgement that doesn't actually know their intent isn't as good for the user as their own judgement.
quadrangle 9y ago
Or AI will grow despite this, and the result will be powerful AI without any solution to the alignment-problem, thus being overall vastly damaging.

eksemplar 9y ago

It’s certainly interesting that in sciences like history, determinism is only plausible when you know very little. The more you know the harder it becomes to figure out why something happened. News feed prediction behaves the opposite of this. It tries to predict your intent based on past behavior, but that is, as you say, not possible, and could likely be why the news feed is dying.

fallous 9y ago
Behold the wisdom of the disclaimer "past performance is no guarantee of future results."
- chiefalchemist 9y ago
  At the tree level, yes. At the forest level most people are predictable, often to a fault.
  - fallous 9y ago
    Of course, and yet there is a reason that demographics is useful only when dealing with aggregates... it tells you nothing about a specific individual. Trying to use inferred data from the behaviors of millions of people is quite useful in discovering certain aggregate info, but using it in place of a specific individual's chosen intent is an excellent way to lose a user.
padobson 9y ago
I wonder if another dimension is useful. YouTube's UI prominently features search. So if my interests change and I search for something I've never been interested before, YouTube can show me a pretty good selection of videos and tune its recommendations for future viewing. I will say, though, that all social networks seem to fail at weighting past behavior too strongly. Whenever I start a new account on a social network, I find it's recommendations are much more tuned to my interests, and those recommendations become less relevant overtime. Biologically, I'm a new person every seven years. As a collection of varied interests, it seems like I turnover much more often than that - and I suspect the rate of turnover is different for everyone.

Tyrek 9y ago

The challenge here is that there's equal and opposing problem - the social sciences (Econ, Psych, etc.) have established that what people say (expressed preferences) and how they act (implied preferences) are inconsistent. So you're adding an additional dimension here that further complicates the calculation.

naravara 9y ago
>the social sciences (Econ, Psych, etc.) have established that what people say (expressed preferences) and how they act (implied preferences) are inconsistent. This is obvious enough. But if I tell you I would like to lose weight and improve my diet, yet you, knowing I have a sweet-tooth, opt to keep offering tempting sweets and desserts anyway, that would make you kind of a jerk wouldn't it? I wouldn't want to spend much time around someone who actively tries to undermine my attempts at personal growth or improvement.
- bo1024 9y ago
  That's a nice example.
- AndrewKemendo 9y ago
  knowing I have a sweet-tooth, opt to keep offering tempting sweets and desserts anyway No, it means you aren't actually doing what you say you want to do. If you however gave the system input about wanting to lose weight, say by buying weight loss supplements, exercise clothes etc... then the "system" will adjust to give you what you want. Your example is basically akin to a scenario where you frequently order dessert at your favorite restaurant, then decide you want to lose weight, never tell anyone you want to lose weight, then get upset when the server brings you the dessert menu.
  - majormajor 9y ago
    > Your example is basically akin to a scenario where you frequently order dessert at your favorite restaurant, then decide you want to lose weight, never tell anyone you want to lose weight, then get upset when the server brings you the dessert menu. Except for the part where he said "if I tell you I would like to lose weight and improve my diet"? So not like the "never tell anyone" part at all? What he's describing is more like a system which learns "most people who research and buy diet stuff end up failing and going back to the ice cream and junk food, so let's start aggressively targeting them with unhealthy crap once they start looking up diets." That may be accurate regarding behavior and revenue maximization, but it's also pretty shitty.

tootie 9y ago

I was never all that into social media at all, but the thing that made me really quit was that the only things I ever clicked from my FB feed we're suggested posts. Their algorithm knew me better than my friends.

gluelogic 9y ago
Maybe, maybe not. A key difference is your friends are posting for themselves, but the suggestions are posts tailored for _you_.

AndrewKemendo 9y ago

the mistaken belief that you can deduce intent via inferred behaviors Maybe this is just a phrasing error, however the idea of inferred behavior doesn't make sense. If you view an action that is measurable (buy x, go to y) then it is an explicit behavior. Further, the concept you seem to be arguing against is the economic concept of revealed preferences. If I understand your argument, it is that it is impossible to derive a users preference from observing the users behavior. This is contrary to decades of empirical research in economics and cognitive science proving that you can rank preferences based on such behavioral profiling. Your example builds a strawman by introducing a value judgement into the example (evil vs good) that is unmeasurable. It conflates some kind of moral state with a series of actions without any way to empirically value them. The reality is that most people cannot accurately state what their true preferences actually are, and thus your actions reveal more about your future behavior than your "intentions" do. This is a classic debate in philosophy of the mind.

fallous 9y ago
Yeah, it's a phrasing error... although there are species of inferred behaviors (he went into a room for an hour and then came out and walked to a store. I don't know what he did in that room but I can infer that he breathed while in it, although as with all inferences that may be untrue.) I'm not arguing against revealed preferences as useful aggregate information regarding the probabilities of someone choosing X, I'm saying that you both can't know their intentions regarding those choices and more specifically those preferences are untrustworthy in predicting any single opportunity to choose X. The idiocy of many current products, be they physical or informational, is replacing the user's ability to make a choice with what the product designers believe... especially without explicit agreement. In most cases the user didn't opt-in to outsourcing or subordinating their decision-making to product X and every time their intent conflicts with the product's decision-making the user will become more discontented with the product.
fallous 9y ago
And no it's not a strawman to introduce my intentions into the example... it's the entire point. Measure everything you like with the exception of my brain and you cannot know my intentions which means that you lack the necessary information to substitute your own judgement or decision-making for mine.

Jedd 9y ago

You're describing Simpson's Paradox. https://en.m.wikipedia.org/wiki/Simpson%27s_paradox?wprov=sfla1 https://en.m.wikipedia.org/wiki/Simpson%27s_paradox?wprov=sf...

fallous 9y ago
No, Simpson's Paradox involves incorrect conclusions of statistical data/trend due to a mistake in viewpoint or assumption. I'm saying that even if you have the correct conclusion that it's not a valid replacement for the user's explicit intent or choice because a) the probability on which a prediction, even a ludicrously accurate one, depends makes no guarantee about the results of a single event... and b) that usurping the user's choice because your model, or you, think you know better is a stunningly bad idea in itself.
- Jedd 9y ago
  > No, Simpson's Paradox involves incorrect conclusions of statistical data/trend due to a mistake in viewpoint or assumption. Not correct. There's no 'mistake in viewpoint or assumption' involved in Simpson's Paradox. Your example story was of someone committing 'a sequence of good deeds throughout the year' that may be part of a grander, longer plan to commit a profoundly not good deed. Analysis of month by month, grouped or in isolation, would reveal an apparent trend that would be opposite to the evident overall trend once you include the ultimate, negative act. From the wikipedia link I provided earlier: "Simpson's paradox ... is a phenomenon in probability and statistics, in which a trend appears in several different groups of data but disappears or reverses when these groups are combined."