3 ms·

Credit history, social media footprint, payment history, etc. Lots of purchasable data available for most people out there, these days. Someone could easily wri

Credit history, social media footprint, payment history, etc. Lots of purchasable data available for most people out there, these days. Someone could easily write a prompt today like: """Here is a person's power usage history: {{ power usage by month }} Here is their bill payment history: amount, date bill sent, date bill due, date payment received, if any: {{ history }} Here is their credit history for the last five years: {{ credit history }} Here is the location of their home and some overall information about the grid: {{ grid info }} Please give me a strategy for maximizing profit from this customer, options include "disconnection", "encourage them to use power at different times", "encourage them to buy solar", "move them to variable usage-based billing". """ Power companies in most places are probably too regulated to get too sneaky, but I'm sure there's shady stuff that could be done, especially if you can similarly tailor the marketing to each user e.g. "we want this person to get solar since the grid going towards them is near its max capacity and we don't want to invest in upgrading it, tailor the messaging towards them based on their credit history and social media profiles." If you imagine a less regulated industry there's even more room for price discrimination and such. The issues, I think, are at least three-fold: 1) Do we want that sort of individualized attention per-user based not just on observed behavioral metrics (ads clicked, sites visited) but every word they've typed online? too? 2) Who is responsible if this model then makes decisions that harm people? The people who trained it? The people who used it? The CEO? It's a wonderful tool for bureaucracy to avoid there having to be a "decision maker" and just have people follow what the tool says to do. 3) And just the practical: the model also is going to spit stuff out, but is it really going to be "optimal" for something closer to a maximization exercise vs just text generation? Possibly not, but I've seen people try stuff like this anyway.

cmilton 3y ago
Thank you for the reply. The power companies are going to buy access to this data. This government regulated utility is going to be allowed to charge its subscribers based on social metrics. This sounds very far fetched to me.
- majormajor 3y ago
  I think you'd be surprised by how many companies buy access to that data. Would it surprise you if your utility pulled your credit score? Cause that already happens. Why is it far-fetched that they wouldn't take another step or two along that path? And let's say it isn't the power company. Let's say it's your employer. Feel good about that?
  - cmilton 3y ago
    I'm not surprised about how data is shared. I know we freely share it to companies without concern. Much of this is within our own control. Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism.
    - majormajor 3y ago
      > Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism. So... you agree with the original post now that there's a specific risk with the current potential uses of these models? I'm not sure what you're questioning at this point. ("what if it was your employer" is also not "whatabouttism," it's another facet of the same concern. Another party who pulls publicly available data today, and could potentially pull even more, giving us as the public the need to decide if we want to allow that to happen. Should someone be able to be made unemployable if an LLM decides they're too much of an asshole on Twitter? Let's figure it out.)
      - cmilton 3y ago
        >So... you agree with the original post now that there's a specific risk with the current potential uses of these models No. I do not believe the US government would allow public utilities to set prices based on social metrics bought from 3rd parties. >"what if it was your employer" is also not "whatabouttism," I do not believe a public utility and my employer correlate. Maybe if I was a government employee. Even then, I don't believe the government would have the right to use that data against me either. I have a hard time believing these underhanded tactics are overlooked and allowed.
nradov 3y ago
I don't understand your concerns. Many utilities already perform credit checks on new accounts, and customers without good credit may be required to pre-pay or post a deposit rather than post-pay (which is effectively a loan). They also already have programs to encourage customers to shift load to off-peak hours through rate differentials and rebates. And they aren't legally allowed to disconnect customers unless the bill is months overdue; they have zero financial incentive to ever disconnect a customer who pays on time. None of this requires AI, or would even benefit much from it.
andrewmutz 3y ago
Are your concerns about using AI for this? Or doing this in general? Planning coercive disconnections in order to maximize profit from a customer based on credit history seems like the problematic thing in this example. Asking an LLM for recommendations on doing seems unrelated. If they asked a human consultant for it instead, it would be just as bad. And just because the human's recommendations would be more explainable, it doesn't make the human consultant's recommendations any less problematic.
- majormajor 3y ago
  That's what I mean by the first of the three ways I'd be concerned here. Asking a human consultant to take a human-intelligence-using look at every single customer (or every single visitor to a website, or whatever) isn't realistic at scale. A company will pay a bit for a background check for a prospective employee, they won't hire a PI to read every single piece of social media correspondence that person has ever written. So if LLMs get cheaper and cheaper, there's a major scale difference due to the existence of the tool. Technological developments leading to re-thinking laws and regulations is nothing new, of course, and "we should outlaw employers hiring PIs to read every piece of text a human has ever posted online" is something that never would've been a realistic thing to pitch to a politician 25 years ago. Now, the technology has changed, so we need to talk about it. You seem to be agreeing that it would be bad, so like the original poster here proposed, let's focus on figuring out what "bad" use cases this tech makes far cheaper and more realistic today and then trying to convince others that we should regulate those use cases. (The second thing I brought up is a difference not in scale but in deniability and ass-covering: it's illegal already to take a retaliatory action against an employee who complains about harassment, for instance. Figure out a sneaky-but-very-plausible way to make that employee just-happen-to-come-up-bad in some LLM-mediated part of a performance review process? Now you'd have a much harder time getting caught.)
  - visarga 3y ago
    Gotta fight fire with fire. For every public facing process we should perform bias analysis with specialised tools. These tools should elicit edge cases of all sorts, do automated "red teaming".

cmilton 3y ago

Thank you for the reply. The power companies are going to buy access to this data. This government regulated utility is going to be allowed to charge its subscribers based on social metrics. This sounds very far fetched to me.

majormajor 3y ago
I think you'd be surprised by how many companies buy access to that data. Would it surprise you if your utility pulled your credit score? Cause that already happens. Why is it far-fetched that they wouldn't take another step or two along that path? And let's say it isn't the power company. Let's say it's your employer. Feel good about that?
- cmilton 3y ago
  I'm not surprised about how data is shared. I know we freely share it to companies without concern. Much of this is within our own control. Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism.
  - majormajor 3y ago
    > Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism. So... you agree with the original post now that there's a specific risk with the current potential uses of these models? I'm not sure what you're questioning at this point. ("what if it was your employer" is also not "whatabouttism," it's another facet of the same concern. Another party who pulls publicly available data today, and could potentially pull even more, giving us as the public the need to decide if we want to allow that to happen. Should someone be able to be made unemployable if an LLM decides they're too much of an asshole on Twitter? Let's figure it out.)
    - cmilton 3y ago
      >So... you agree with the original post now that there's a specific risk with the current potential uses of these models No. I do not believe the US government would allow public utilities to set prices based on social metrics bought from 3rd parties. >"what if it was your employer" is also not "whatabouttism," I do not believe a public utility and my employer correlate. Maybe if I was a government employee. Even then, I don't believe the government would have the right to use that data against me either. I have a hard time believing these underhanded tactics are overlooked and allowed.

nradov 3y ago

I don't understand your concerns. Many utilities already perform credit checks on new accounts, and customers without good credit may be required to pre-pay or post a deposit rather than post-pay (which is effectively a loan). They also already have programs to encourage customers to shift load to off-peak hours through rate differentials and rebates. And they aren't legally allowed to disconnect customers unless the bill is months overdue; they have zero financial incentive to ever disconnect a customer who pays on time. None of this requires AI, or would even benefit much from it.

andrewmutz 3y ago

Are your concerns about using AI for this? Or doing this in general? Planning coercive disconnections in order to maximize profit from a customer based on credit history seems like the problematic thing in this example. Asking an LLM for recommendations on doing seems unrelated. If they asked a human consultant for it instead, it would be just as bad. And just because the human's recommendations would be more explainable, it doesn't make the human consultant's recommendations any less problematic.

majormajor 3y ago
That's what I mean by the first of the three ways I'd be concerned here. Asking a human consultant to take a human-intelligence-using look at every single customer (or every single visitor to a website, or whatever) isn't realistic at scale. A company will pay a bit for a background check for a prospective employee, they won't hire a PI to read every single piece of social media correspondence that person has ever written. So if LLMs get cheaper and cheaper, there's a major scale difference due to the existence of the tool. Technological developments leading to re-thinking laws and regulations is nothing new, of course, and "we should outlaw employers hiring PIs to read every piece of text a human has ever posted online" is something that never would've been a realistic thing to pitch to a politician 25 years ago. Now, the technology has changed, so we need to talk about it. You seem to be agreeing that it would be bad, so like the original poster here proposed, let's focus on figuring out what "bad" use cases this tech makes far cheaper and more realistic today and then trying to convince others that we should regulate those use cases. (The second thing I brought up is a difference not in scale but in deniability and ass-covering: it's illegal already to take a retaliatory action against an employee who complains about harassment, for instance. Figure out a sneaky-but-very-plausible way to make that employee just-happen-to-come-up-bad in some LLM-mediated part of a performance review process? Now you'd have a much harder time getting caught.)
- visarga 3y ago
  Gotta fight fire with fire. For every public facing process we should perform bias analysis with specialised tools. These tools should elicit edge cases of all sorts, do automated "red teaming".