8 ms·

How Visa Predicts Divorce

96 points by subbu 17y ago

MichaelGG 17y ago
I wonder how expensive it is to purchase things that decrease your risk rating, while also using cash for things considered particularly risky.
- jfoutz 17y ago
  You have to be more careful than that. You need to write some script to follow random links on HN while you're away from your computer, so when you're off doing risky things you won't change your profile. You need to occasionally forget your cell phone, so when you're doing risky things, forgetting your cell phone won't change your profile. If you have a car, you need to pass through the intersections with red light cameras at the normal times, but no extra times, so you don't change your profile. Does visa buy dns lookup histories? or cellphone active tower histories? or car intersection histories? probably not. yet. better to build the habits now though.
  - boredguy8 17y ago
    Even better would be to have a probabilistic weight to these various items. Computers are way better at finding patterns and irregularities in patterns than we are at making irregularities. If I were to "accidentally" forget my phone based on my own decision, it would probably cluster in some way, say, forgetting it every 17 days +/- 1, or on Thursdays more than any other day, or something. So create a profile, then use RNG to match yourself to that profile.
  - 4ensic 17y ago
    Stepwise linear regression is how they do it. All statistical packages marketed since the late 1970's include the procedures. It's what the AI craze of the early 1990's used in their black box apps. Basically you associate a narrow set of outcomes (Filing for divorce, filing for bankruptcy, reporting a stolen credit card) and a mountain of possible predictors (store type, store name, store location, number of purchases from store in the billing cycle, ranges of purchases from a given store within the cycle). The stat program builds a regression equation out of the raw data by literally predicting trying to predict the past from data in the further past. The cool thing is that you can't put in too many variables (as in you don't have to know if something may be related or not) since the uncorrelated variables will simply vanish in the final equation.
og1 17y ago
According to the article UPS predicts when you are high risk of leaving their service and will offer a sales call. I assume the salesman is allowed to offer perk/discounts to get a customer to try to stay. I'm wondering if these models can be figured out so you can intentionally trigger something like this to get deals from companies that do data mining.
kurtosis 17y ago
I've heard rumors that visa uses its purchase records as a leading indicator to trade in stocks. Having all of these sales records gives them a leading indicator of how a particular retailer is performing, before the company releases any statements, or speaks with analysts. While the concept sounded simple, I'm not sure if I believe this because I had no way of knowing how large of an advantage it would translate into. Has anyone else here heard these kinds of stories about visa or banks? Is there are reason why this kind of thing would be illegal or ineffective?
- byrneseyeview 17y ago
  I know of one financial firm that did this. They didn't trade the specific stocks, but they used their credit card data to predict what the Fed would do. Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading. But it's accepted that people who bet on interest rates will gather lots of data--since every economic indicator you can imagine will have some effect on rates, it's both impossible to be a true "insider" and impossible not to have some degree of insider information.
  - anamax 17y ago
    > Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading. That's not the definition of "insider trading". Insider trading is you trading on your company's info - the company is free to do so. For example, Google employees can't trade Rackspace on knowledge that Google is about to make a big deal with Rackspace. Google, on the other hand, is free to do so. From http://en.wikipedia.org/wiki/Insider_trading http://en.wikipedia.org/wiki/Insider_trading "In the United States and many other jurisdictions, however, "insiders" are not just limited to corporate officials and major shareholders where illegal insider trading is concerned, but can include any individual who trades shares based on material non-public information in violation of some duty of trust. This duty may be imputed; for example, in many jurisdictions, in cases of where a corporate insider "tips" a friend about non-public information likely to have an effect on the company's share price, the duty the corporate insider owes the company is now imputed to the friend and the friend violates a duty to the company if he or she trades on the basis of this information."
shrikant 17y ago
Correction: The headline of this article originally referenced MasterCard, not Visa. As is evident from the post URL. But why the change?
- jules 17y ago
  On the left it says: > How Visa Predicts Divorce > by Nicholas Ciarelli
  - xtho 17y ago
    Captain Obvious at work? How does it answer the question? Assuming you know what the question was about?
- easyfrag 17y ago
  Another headline correction that is needed is that the article doesn't tell us how Visa predicts divorce, just that they can.
  - xtho 17y ago
    Since they don't seem to tell what their success rate & the accuracy of the predicted date are, I wouldn't say they "can" do it, which would imply that they are 100% successful at predicting the day of a divorce.
binarymax 17y ago
I highly recommend Super Crunchers to anyone who has not yet read it. It is written for the layman, but nonetheless it is highly fascinating and insightful for anyone who is remotely interested in data mining.
- jgrahamc 17y ago
  I found the book very disappointing because it glossed over any real detail.
  - deleted 17y ago
    [deleted]
  - binarymax 17y ago
    Like I said, its "written for the layman". If you want to get into high detail for one of the many examples in his book...you will eventually find yourself at places like this: http://www.retrosheet.org/game.htm http://www.retrosheet.org/game.htm :) Also if you want some more technical detail about the overall process there are plenty of books out there: http://news.ycombinator.com/item?id=1055042 http://news.ycombinator.com/item?id=1055042.
_debug_ 17y ago
The comments on that article are telling. The loss of privacy due to inappropriate data mining looks like it may cause at least some people to "go Amish" slowly (now, before I get jumped, I just mean "the avoidance of technology" by "go Amish"; please don't get religious over that). Personally, reading this article makes me glad I never opened a FaceBook account. I wish gmail was not so all-knowing either; I hate seeing very personally directed ads (anti-depressants when I am sad after reading a break-up mail, for example). Just too creepy.
- jacquesm 17y ago
  Anybody that breaks up using email in stead of a face-to-face is probably not worth being sad over. How people part ways says lots about them.
euroclydon 17y ago
The Underground History of American Education posits that the main factor at work in the school system is the homogenization of people for the benefit of big business. From a tech perspective, it's easy to understand this motivation; wouldn't it be easier to develop that killer mobile coupon app if everyone had a device with the same capabilities? Now I'm beginning to wonder if data mining is making the homogenization of people obsolete, if hundreds of profiles can be created, and it suffices business interests to simply be able to categorize you?
- pmichaud 17y ago
  They don't only want to homogenize the consumer, they want to also homogenize the workers. A fungible workforce is a boon.
goodside 17y ago
Slightly off-topic: if you post something with a misleading title that you can improve, please do so. This article has no information at all on how Visa predicts divorce, just the third-hand assertion that they do. An Amazon text search within Supercrunchers yields nothing of value, either.
- idm 17y ago
  Exactly. Why is the word "how" in the title? Here's a choice quote from the article regarding that question: "Exactly how the credit industry does it—through sophisticated data-mining techniques—is a closely guarded secret. " Well great! Now the title isn't simply misleading, it's factually inaccurate.
  - Alex3917 17y ago
    "Why is the word 'how' in the title?" Because TDB raise tens of millions of dollars so they could hire dozens of celebrities and go on Colbert the day they launched, and they're trying to make their money back.
Estragon 17y ago
The article is badly titled. It explicitly states after the lede that the author was unable to determine how Visa predicts divorce.
patio11 17y ago
And if you think your purchases say a lot about you, just wait until somebody gets a hold of your social network data. (I am willing to bet that "Percentage of friends who divorced" and "Percentage of friends who divorced recently" both whallop any naive demographic predictor of divorce risk you care to name.)
- jreposa 17y ago
  Rapleaf is already making progress with this. http://www.rapleaf.com/apidoc/person http://www.rapleaf.com/apidoc/person
ttol 17y ago
Factoid: The author, Nicholas Ciarelli, was also known as Nick de Plume -- founder of thinksecret.com.
jasonlotito 17y ago
Off-topic, but related to stuff mentioned in the article. How do we relate the idea that we own the data a company like Visa collects about us? For example, our buying habits should be "our data." While I understand the need for privacy, people go on to worry about how Visa will use this information, and it's impact on privacy. Does Visa have the right to use this data to target users? On one hand, we'd be highly upset if Visa didn't use the data it collects in certain cases, like anti-fraud and tracking odd behavior and warning us of it. But then you have an article like this asking for consumer protection. I guess the issue here is one of balance: who owns the data? If it's the user, does the company really have the right to use that data to provide a service with it? Even if it's anti-fraud or simple customer service? This isn't something new, either. It's only become easier and easier to do more accurately as technology grows. On one hand, privacy is important for obvious reasons. On the other hand, generated data is useful and can be used for improving our lives. Just as privacy protects the criminals as well as the innocent, generated data can be used for ill or for good. I don't think I'm asking anything new. I've always been a person that wants to track everything users do. On sites I create, I want to track everything, know everything. Yes, I want to do this so that I can increase profit. But it's only because I believe that a user who attempts to make a purchase and fails really wanted the product, and it's my job to do everything in my power to get that product to him. I have no desire to steal his data, sell his information, or anything else. But I can see how something like this could be misrepresented or misused. Just some thoughts I had on this. I'd like to hear other points of view on both sides of the spectrum. I admit I don't know enough, and while my motives are honest, they are in a way profit driven. Customer services is profit driven (Apple doesn't provide the CS it does because it loses money on it!)! And if anyone has any good links on the topic, please share.
- froo 17y ago
  >Does Visa have the right to use this data to target users? Its probably buried somewhere in legalese in one of the Product Diclosure Statements. I was surprised by some of the things my bank had wanted me to agree too several months ago when they "updated" this document, so much so that I switched banks.
- lotharbot 17y ago
asimjalis 17y ago
Given that the article does not actually talk about how Visa predicts divorce, maybe we can speculate. Here are some ideas: * Meals at two restaurants on same night. * Separate vacations. * Ages approach mid-life. * Card used on personals websites.
- crocowhile 17y ago
  I doubt there is a rationale trend. It's just statistics, really. My guess is that it has more to do with the amount of money spent by each spouse rather than what.
jsm386 17y ago
Political parties have been doing this for some time. Check out http://www.voterlistsonline.com/version2/site/page.asp?page_id=home http://www.voterlistsonline.com/version2/site/page.asp?page_... You can learn a lot about a potential voter (ie is he worth making a stop when canvasing?) Purchase Premium Fields, including: - Home purchase/mortgage/refinance information - Presence/ages of children per household - Magazine subscriptions - Hunting/fishing licenses - Charity contributions - Additional polling options - including Targeted Random Sampling and Normalized Sampling*
rythie 17y ago
Does Visa actually know what you've bought? "Cardholders who purchased carbon-monoxide detectors, premium birdseed, and felt pads for the bottoms of their chair legs rarely missed a payment." My bank statements only state where something was bought, not what was bought. I'm not aware of a store that only sells felt pads for bottoms of chairs and premium birdseed.
- The_Fox 17y ago
  That statement was referring just to Canadian Tire, a retail store that also offers credit cards. I assume the card company can only get that kind of detailed information if you make your Canadian Tire purchase with your Canadian Tire MasterCard. I once purchased a CO detector at Canadian Tire... maybe I'll bring that up when I apply for my next credit card?
rmorrison 17y ago
The mobile social network Loopt or its competitors could conceivably predict with 90 percent accuracy where an individual will be tomorrow. I would think that Visa would be better positioned to do this than Loopt, because people are more compelled to use their credit cards when they go somewhere. You can't go to a store and forget to pay, although you can forget to checkin.
pw0ncakes 17y ago
It's probably pretty easy, because in addition to what is being bought, Visa knows where the purchases occur. Most people have two clusters for their purchases: work and home. A person who develops a third cluster may be having an affair. If you know what's being bought, you can figure it out with even higher probability.

96 points by subbu 17y ago

I wonder how expensive it is to purchase things that decrease your risk rating, while also using cash for things considered particularly risky.

jfoutz 17y ago
You have to be more careful than that. You need to write some script to follow random links on HN while you're away from your computer, so when you're off doing risky things you won't change your profile. You need to occasionally forget your cell phone, so when you're doing risky things, forgetting your cell phone won't change your profile. If you have a car, you need to pass through the intersections with red light cameras at the normal times, but no extra times, so you don't change your profile. Does visa buy dns lookup histories? or cellphone active tower histories? or car intersection histories? probably not. yet. better to build the habits now though.
- boredguy8 17y ago
  Even better would be to have a probabilistic weight to these various items. Computers are way better at finding patterns and irregularities in patterns than we are at making irregularities. If I were to "accidentally" forget my phone based on my own decision, it would probably cluster in some way, say, forgetting it every 17 days +/- 1, or on Thursdays more than any other day, or something. So create a profile, then use RNG to match yourself to that profile.
- 4ensic 17y ago
  Stepwise linear regression is how they do it. All statistical packages marketed since the late 1970's include the procedures. It's what the AI craze of the early 1990's used in their black box apps. Basically you associate a narrow set of outcomes (Filing for divorce, filing for bankruptcy, reporting a stolen credit card) and a mountain of possible predictors (store type, store name, store location, number of purchases from store in the billing cycle, ranges of purchases from a given store within the cycle). The stat program builds a regression equation out of the raw data by literally predicting trying to predict the past from data in the further past. The cool thing is that you can't put in too many variables (as in you don't have to know if something may be related or not) since the uncorrelated variables will simply vanish in the final equation.

og1 17y ago

According to the article UPS predicts when you are high risk of leaving their service and will offer a sales call. I assume the salesman is allowed to offer perk/discounts to get a customer to try to stay. I'm wondering if these models can be figured out so you can intentionally trigger something like this to get deals from companies that do data mining.

kurtosis 17y ago

I've heard rumors that visa uses its purchase records as a leading indicator to trade in stocks. Having all of these sales records gives them a leading indicator of how a particular retailer is performing, before the company releases any statements, or speaks with analysts. While the concept sounded simple, I'm not sure if I believe this because I had no way of knowing how large of an advantage it would translate into. Has anyone else here heard these kinds of stories about visa or banks? Is there are reason why this kind of thing would be illegal or ineffective?

byrneseyeview 17y ago
I know of one financial firm that did this. They didn't trade the specific stocks, but they used their credit card data to predict what the Fed would do. Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading. But it's accepted that people who bet on interest rates will gather lots of data--since every economic indicator you can imagine will have some effect on rates, it's both impossible to be a true "insider" and impossible not to have some degree of insider information.
- anamax 17y ago
  > Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading. That's not the definition of "insider trading". Insider trading is you trading on your company's info - the company is free to do so. For example, Google employees can't trade Rackspace on knowledge that Google is about to make a big deal with Rackspace. Google, on the other hand, is free to do so. From http://en.wikipedia.org/wiki/Insider_trading http://en.wikipedia.org/wiki/Insider_trading "In the United States and many other jurisdictions, however, "insiders" are not just limited to corporate officials and major shareholders where illegal insider trading is concerned, but can include any individual who trades shares based on material non-public information in violation of some duty of trust. This duty may be imputed; for example, in many jurisdictions, in cases of where a corporate insider "tips" a friend about non-public information likely to have an effect on the company's share price, the duty the corporate insider owes the company is now imputed to the friend and the friend violates a duty to the company if he or she trades on the basis of this information."

shrikant 17y ago

Correction: The headline of this article originally referenced MasterCard, not Visa. As is evident from the post URL. But why the change?

jules 17y ago
On the left it says: > How Visa Predicts Divorce > by Nicholas Ciarelli
- xtho 17y ago
  Captain Obvious at work? How does it answer the question? Assuming you know what the question was about?
easyfrag 17y ago
Another headline correction that is needed is that the article doesn't tell us how Visa predicts divorce, just that they can.
- xtho 17y ago
  Since they don't seem to tell what their success rate & the accuracy of the predicted date are, I wouldn't say they "can" do it, which would imply that they are 100% successful at predicting the day of a divorce.

binarymax 17y ago

I highly recommend Super Crunchers to anyone who has not yet read it. It is written for the layman, but nonetheless it is highly fascinating and insightful for anyone who is remotely interested in data mining.

jgrahamc 17y ago
I found the book very disappointing because it glossed over any real detail.
- deleted 17y ago
  [deleted]
- binarymax 17y ago
  Like I said, its "written for the layman". If you want to get into high detail for one of the many examples in his book...you will eventually find yourself at places like this: http://www.retrosheet.org/game.htm http://www.retrosheet.org/game.htm :) Also if you want some more technical detail about the overall process there are plenty of books out there: http://news.ycombinator.com/item?id=1055042 http://news.ycombinator.com/item?id=1055042.

_debug_ 17y ago

The comments on that article are telling. The loss of privacy due to inappropriate data mining looks like it may cause at least some people to "go Amish" slowly (now, before I get jumped, I just mean "the avoidance of technology" by "go Amish"; please don't get religious over that). Personally, reading this article makes me glad I never opened a FaceBook account. I wish gmail was not so all-knowing either; I hate seeing very personally directed ads (anti-depressants when I am sad after reading a break-up mail, for example). Just too creepy.

jacquesm 17y ago
Anybody that breaks up using email in stead of a face-to-face is probably not worth being sad over. How people part ways says lots about them.

euroclydon 17y ago

The Underground History of American Education posits that the main factor at work in the school system is the homogenization of people for the benefit of big business. From a tech perspective, it's easy to understand this motivation; wouldn't it be easier to develop that killer mobile coupon app if everyone had a device with the same capabilities? Now I'm beginning to wonder if data mining is making the homogenization of people obsolete, if hundreds of profiles can be created, and it suffices business interests to simply be able to categorize you?

pmichaud 17y ago
They don't only want to homogenize the consumer, they want to also homogenize the workers. A fungible workforce is a boon.

goodside 17y ago

Slightly off-topic: if you post something with a misleading title that you can improve, please do so. This article has no information at all on how Visa predicts divorce, just the third-hand assertion that they do. An Amazon text search within Supercrunchers yields nothing of value, either.

idm 17y ago
Exactly. Why is the word "how" in the title? Here's a choice quote from the article regarding that question: "Exactly how the credit industry does it—through sophisticated data-mining techniques—is a closely guarded secret. " Well great! Now the title isn't simply misleading, it's factually inaccurate.
- Alex3917 17y ago
  "Why is the word 'how' in the title?" Because TDB raise tens of millions of dollars so they could hire dozens of celebrities and go on Colbert the day they launched, and they're trying to make their money back.

Estragon 17y ago

The article is badly titled. It explicitly states after the lede that the author was unable to determine how Visa predicts divorce.

patio11 17y ago

And if you think your purchases say a lot about you, just wait until somebody gets a hold of your social network data. (I am willing to bet that "Percentage of friends who divorced" and "Percentage of friends who divorced recently" both whallop any naive demographic predictor of divorce risk you care to name.)

jreposa 17y ago
Rapleaf is already making progress with this. http://www.rapleaf.com/apidoc/person http://www.rapleaf.com/apidoc/person

ttol 17y ago

Factoid: The author, Nicholas Ciarelli, was also known as Nick de Plume -- founder of thinksecret.com.

jasonlotito 17y ago

Off-topic, but related to stuff mentioned in the article. How do we relate the idea that we own the data a company like Visa collects about us? For example, our buying habits should be "our data." While I understand the need for privacy, people go on to worry about how Visa will use this information, and it's impact on privacy. Does Visa have the right to use this data to target users? On one hand, we'd be highly upset if Visa didn't use the data it collects in certain cases, like anti-fraud and tracking odd behavior and warning us of it. But then you have an article like this asking for consumer protection. I guess the issue here is one of balance: who owns the data? If it's the user, does the company really have the right to use that data to provide a service with it? Even if it's anti-fraud or simple customer service? This isn't something new, either. It's only become easier and easier to do more accurately as technology grows. On one hand, privacy is important for obvious reasons. On the other hand, generated data is useful and can be used for improving our lives. Just as privacy protects the criminals as well as the innocent, generated data can be used for ill or for good. I don't think I'm asking anything new. I've always been a person that wants to track everything users do. On sites I create, I want to track everything, know everything. Yes, I want to do this so that I can increase profit. But it's only because I believe that a user who attempts to make a purchase and fails really wanted the product, and it's my job to do everything in my power to get that product to him. I have no desire to steal his data, sell his information, or anything else. But I can see how something like this could be misrepresented or misused. Just some thoughts I had on this. I'd like to hear other points of view on both sides of the spectrum. I admit I don't know enough, and while my motives are honest, they are in a way profit driven. Customer services is profit driven (Apple doesn't provide the CS it does because it loses money on it!)! And if anyone has any good links on the topic, please share.

froo 17y ago
>Does Visa have the right to use this data to target users? Its probably buried somewhere in legalese in one of the Product Diclosure Statements. I was surprised by some of the things my bank had wanted me to agree too several months ago when they "updated" this document, so much so that I switched banks.
lotharbot 17y ago

asimjalis 17y ago

Given that the article does not actually talk about how Visa predicts divorce, maybe we can speculate. Here are some ideas: * Meals at two restaurants on same night. * Separate vacations. * Ages approach mid-life. * Card used on personals websites.

crocowhile 17y ago
I doubt there is a rationale trend. It's just statistics, really. My guess is that it has more to do with the amount of money spent by each spouse rather than what.

jsm386 17y ago

Political parties have been doing this for some time. Check out http://www.voterlistsonline.com/version2/site/page.asp?page_id=home http://www.voterlistsonline.com/version2/site/page.asp?page_... You can learn a lot about a potential voter (ie is he worth making a stop when canvasing?) Purchase Premium Fields, including: - Home purchase/mortgage/refinance information - Presence/ages of children per household - Magazine subscriptions - Hunting/fishing licenses - Charity contributions - Additional polling options - including Targeted Random Sampling and Normalized Sampling*

rythie 17y ago

Does Visa actually know what you've bought? "Cardholders who purchased carbon-monoxide detectors, premium birdseed, and felt pads for the bottoms of their chair legs rarely missed a payment." My bank statements only state where something was bought, not what was bought. I'm not aware of a store that only sells felt pads for bottoms of chairs and premium birdseed.

The_Fox 17y ago
That statement was referring just to Canadian Tire, a retail store that also offers credit cards. I assume the card company can only get that kind of detailed information if you make your Canadian Tire purchase with your Canadian Tire MasterCard. I once purchased a CO detector at Canadian Tire... maybe I'll bring that up when I apply for my next credit card?

rmorrison 17y ago

The mobile social network Loopt or its competitors could conceivably predict with 90 percent accuracy where an individual will be tomorrow. I would think that Visa would be better positioned to do this than Loopt, because people are more compelled to use their credit cards when they go somewhere. You can't go to a store and forget to pay, although you can forget to checkin.

pw0ncakes 17y ago

It's probably pretty easy, because in addition to what is being bought, Visa knows where the purchases occur. Most people have two clusters for their purchases: work and home. A person who develops a third cluster may be having an affair. If you know what's being bought, you can figure it out with even higher probability.