5 ms·

I remember once being in a branch of HSBC trying to sort out a loan. I saw on the screen for my account they had a score of something like "customer behaviour",

I remember once being in a branch of HSBC trying to sort out a loan. I saw on the screen for my account they had a score of something like "customer behaviour", which was like 54 something. I had drunkenly called HSBC and ended up ranting at the person on the phone a few times (lost cards etc) so I think this was a rating of how well behaved I was towards their staff. The manager I was speaking to changed his tone fairly quickly after that screen came up! So this is nothing new, but I guess the scale and opportunities for data points are new. Although, having said that, a lot of marketing based scoring data is woefully inaccurate. I was once responsible for distributing a data set to company clients which covered interests and personal info for all UK population. Not only was most of the data about my dad wrong, the things that were accurate were years out of date. My information didn't exist. A colleague's email was completely wrong. Indicated he liked going on holiday but had never been outside of the UK. So these scores might be useless anyway. edit: got rid of "even if they exist".

elliekelly 6y ago
It very well might be garbage data but if companies are acting on that data it’s still highly problematic. Maybe even worse than if the data were accurate.
- dijksterhuis 6y ago
  Yeah that's probably the bigger problem!
- air7 6y ago
  The possible silver lining is that companies will quickly realize the data is garbage (which is causing then losses) and stop using it.
  - vladvasiliu 6y ago
    Depends on how quick is "quickly". If anything, they seem to be doubling down on this. From my discussions on the subject with people "in marketing", they are really convinced that not only is the data "accurate enough", but that collecting and exploiting it is actually for the client's benefit. I wonder how one would go about quantifying the usefulness of all this tracking, because I doubt it's cheap. The best I could get was "we're able to see changes in sales which correlate to marketing campaigns". I can totally buy that and it seems somewhat more advanced of an answer than the more common one of "If didn't work they wouldn't be doing it". What I wonder though is how they would go about quantifying how much better a tracking-based campaign worked than a "traditional" one would have.
    - kiba 6y ago
      What if most of the marketing is useless in the first place? And most of the data are useless too. How would they know?
  - usrusr 6y ago
    Data that appears "good enough" under the purely economic lens can leave an awful lot of people unfairly out of service. In a low margin business a single bad customer can easily cost more than what is earned from ten good customers. In that situation, an oracle that rejects eight good customers per rejected bad customer would already be good enough.
  - mindslight 6y ago
    That's a pretty strong assumption. Data that is garbage isn't necessarily going to cause losses. In fact, making the market more opaque (less efficient) will add to profit margins. If all companies are using the same data/algorithm, then even incorrect decisions form a focal point. If all the companies think someone is a bad risk, then that person will end up paying more regardless. Even if one company defects and accurately prices their risk, that company will just lower their rate a little and that customer still won't be getting the rate they would have if full market competition were taking place.
- m463 6y ago
  Does "risk avoidance" mean they have to act on the data?
kgwxd 6y ago
Useless, but being used. A divining rod in the hands of people deciding the big steps in your life.
hrwl 6y ago
I've had a similar experience. Had access to a dataset about which many breathless articles were written when it was leaked/breached a couple years later. Was able to find very little data on family members in the set and what was there was quite stale.
- beervirus 6y ago
  The fact that it’s inaccurate doesn’t make it more acceptable.
hnhg 6y ago
Shouldn't you be able to demand an explanation of how that score was calculated under GDPR? I'm really curious as to what it means now!
- dijksterhuis 6y ago
  I'm betting it was just a rating out of 10 the person answering the phone did after each call.
- dane-pgp 6y ago
  https://gdpr-info.eu/art-14-gdpr/ https://gdpr-info.eu/art-14-gdpr/ "2. In addition to the information referred to in paragraph 1, the controller shall provide the data subject with the following information necessary to ensure fair and transparent processing in respect of the data subject: ... (g) the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject."
  - indziektor 6y ago
    Case in point, I was just recently getting nowhere on an obviously automated decision, so I sent an email quoting GDPR Article 15(1)(h) (instead of Article 14) and got it resolved amicably within an hour. Maybe just coincidence, but their follow-up was a complete 180. I suppose it could be abused to get around certain automatic safeguards, but I made it clear that my intentions where just to ensure that my data is correct and being processed fairly, as I was sure it wasn't the case. But that requires that the company even cares about following the GDPR correctly, or perhaps even the Data Protection Authority of a given EU country. I know of one small but popular enough data broker that's been operating for years in the EU and flaunting the GDPR without any punishment so far. Not sure if this is the place to advertise it. But I suppose somebody whose GDPR rights were violated has to lodge a complaint first.
- klyrs 6y ago
  In Europe, sure. But imho if a customer verbally abuses employees, it's perfectly fair to cease doing business with them.
Karunamon 6y ago
I think that people wind up worried about the wrong thing on these systems. The worst case scenario, where some all-knowing system negatively impacts you because it's rating your behavior on some opaque scale, isn't the one we should be worried about. That by itself is scary enough, but the much more likely case is the one where this system is rating you based on wrong information, and given Finagle's law.. Especially concerning when the justice system starts using this info; you really, really don't want to be the false positive.
- justanotherc 6y ago
  The justice system has built into it the requirement to justify allegations against you though. By design everything in the justice system is transparent or its not admissible. So an opaque "score" of any sort wouldn't ever be acceptable evidence.
  - sroussey 6y ago
    You are thinking in about evidence in court. Police have other data that can flag you for greater scrutiny. Then your chances of even being in court go up.
    - justanotherc 6y ago
      But they have to establish reasonable suspicion, wuch they also have to justify in court in order to investigate you further. They can't just say "he had a bad score, so I pulled him over and found weed". If they did that, the stop would be unjustified and the whole case would be thrown out.
  - patmorgan23 6y ago
    Predictive policing
  - inetknght 6y ago
    > By design everything in the justice system is transparent or its not admissible. https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
usrusr 6y ago
There's a very good reason why organizations should be allowed to keep "fool me once/twice/thrice" records: it's what enables them to be nice to customers who don't abuse that niceness. But there's no process to appeal and the data might be laughably wrong. Outlawing cross-organizational aggregation might be a reasonable middle ground. It gives a further advantage to giants like Amazon, but all the problems compound at interorganizational integration.
- PopeDotNinja 6y ago
  In 2006, I spent some time in the hospital after a bike accident. Someone who shared my recovery room was an guy with kidney failure & an extremely poor attitude. I learned he was running out of places to get dialysis because he kept treating the staff poorly & getting kicked out of the dialysis centers. Imagine having some poor newly hired receptionist trying to help this near death, abusive asshole & reading a note in the computer screen that said “do not treat this person under any circumstances”.
  - tomcooks 6y ago
    Precisely what the Hippocratic Oath states, take care of all who have the green smiley icon and 70 or more happy points
    - deleted 6y ago
      [deleted]

elliekelly 6y ago

It very well might be garbage data but if companies are acting on that data it’s still highly problematic. Maybe even worse than if the data were accurate.

dijksterhuis 6y ago
Yeah that's probably the bigger problem!
air7 6y ago
The possible silver lining is that companies will quickly realize the data is garbage (which is causing then losses) and stop using it.
- vladvasiliu 6y ago
  Depends on how quick is "quickly". If anything, they seem to be doubling down on this. From my discussions on the subject with people "in marketing", they are really convinced that not only is the data "accurate enough", but that collecting and exploiting it is actually for the client's benefit. I wonder how one would go about quantifying the usefulness of all this tracking, because I doubt it's cheap. The best I could get was "we're able to see changes in sales which correlate to marketing campaigns". I can totally buy that and it seems somewhat more advanced of an answer than the more common one of "If didn't work they wouldn't be doing it". What I wonder though is how they would go about quantifying how much better a tracking-based campaign worked than a "traditional" one would have.
  - kiba 6y ago
    What if most of the marketing is useless in the first place? And most of the data are useless too. How would they know?
- usrusr 6y ago
  Data that appears "good enough" under the purely economic lens can leave an awful lot of people unfairly out of service. In a low margin business a single bad customer can easily cost more than what is earned from ten good customers. In that situation, an oracle that rejects eight good customers per rejected bad customer would already be good enough.
- mindslight 6y ago
  That's a pretty strong assumption. Data that is garbage isn't necessarily going to cause losses. In fact, making the market more opaque (less efficient) will add to profit margins. If all companies are using the same data/algorithm, then even incorrect decisions form a focal point. If all the companies think someone is a bad risk, then that person will end up paying more regardless. Even if one company defects and accurately prices their risk, that company will just lower their rate a little and that customer still won't be getting the rate they would have if full market competition were taking place.
m463 6y ago
Does "risk avoidance" mean they have to act on the data?

kgwxd 6y ago

Useless, but being used. A divining rod in the hands of people deciding the big steps in your life.

hrwl 6y ago

I've had a similar experience. Had access to a dataset about which many breathless articles were written when it was leaked/breached a couple years later. Was able to find very little data on family members in the set and what was there was quite stale.

beervirus 6y ago
The fact that it’s inaccurate doesn’t make it more acceptable.

hnhg 6y ago

Shouldn't you be able to demand an explanation of how that score was calculated under GDPR? I'm really curious as to what it means now!

dijksterhuis 6y ago
I'm betting it was just a rating out of 10 the person answering the phone did after each call.
dane-pgp 6y ago
https://gdpr-info.eu/art-14-gdpr/ https://gdpr-info.eu/art-14-gdpr/ "2. In addition to the information referred to in paragraph 1, the controller shall provide the data subject with the following information necessary to ensure fair and transparent processing in respect of the data subject: ... (g) the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject."
- indziektor 6y ago
  Case in point, I was just recently getting nowhere on an obviously automated decision, so I sent an email quoting GDPR Article 15(1)(h) (instead of Article 14) and got it resolved amicably within an hour. Maybe just coincidence, but their follow-up was a complete 180. I suppose it could be abused to get around certain automatic safeguards, but I made it clear that my intentions where just to ensure that my data is correct and being processed fairly, as I was sure it wasn't the case. But that requires that the company even cares about following the GDPR correctly, or perhaps even the Data Protection Authority of a given EU country. I know of one small but popular enough data broker that's been operating for years in the EU and flaunting the GDPR without any punishment so far. Not sure if this is the place to advertise it. But I suppose somebody whose GDPR rights were violated has to lodge a complaint first.
klyrs 6y ago
In Europe, sure. But imho if a customer verbally abuses employees, it's perfectly fair to cease doing business with them.

Karunamon 6y ago

I think that people wind up worried about the wrong thing on these systems. The worst case scenario, where some all-knowing system negatively impacts you because it's rating your behavior on some opaque scale, isn't the one we should be worried about. That by itself is scary enough, but the much more likely case is the one where this system is rating you based on wrong information, and given Finagle's law.. Especially concerning when the justice system starts using this info; you really, really don't want to be the false positive.

justanotherc 6y ago
The justice system has built into it the requirement to justify allegations against you though. By design everything in the justice system is transparent or its not admissible. So an opaque "score" of any sort wouldn't ever be acceptable evidence.
- sroussey 6y ago
  You are thinking in about evidence in court. Police have other data that can flag you for greater scrutiny. Then your chances of even being in court go up.
  - justanotherc 6y ago
    But they have to establish reasonable suspicion, wuch they also have to justify in court in order to investigate you further. They can't just say "he had a bad score, so I pulled him over and found weed". If they did that, the stop would be unjustified and the whole case would be thrown out.
- patmorgan23 6y ago
  Predictive policing
- inetknght 6y ago
  > By design everything in the justice system is transparent or its not admissible. https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction

usrusr 6y ago

There's a very good reason why organizations should be allowed to keep "fool me once/twice/thrice" records: it's what enables them to be nice to customers who don't abuse that niceness. But there's no process to appeal and the data might be laughably wrong. Outlawing cross-organizational aggregation might be a reasonable middle ground. It gives a further advantage to giants like Amazon, but all the problems compound at interorganizational integration.

PopeDotNinja 6y ago
In 2006, I spent some time in the hospital after a bike accident. Someone who shared my recovery room was an guy with kidney failure & an extremely poor attitude. I learned he was running out of places to get dialysis because he kept treating the staff poorly & getting kicked out of the dialysis centers. Imagine having some poor newly hired receptionist trying to help this near death, abusive asshole & reading a note in the computer screen that said “do not treat this person under any circumstances”.
- tomcooks 6y ago
  Precisely what the Hippocratic Oath states, take care of all who have the green smiley icon and 70 or more happy points
  - deleted 6y ago
    [deleted]