5 ms·

But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. If someone m

But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. If someone modifies their Tesla to accept input from Uber, and Tesla does nothing to stop them, and it causes a crash or other problems, then there'll be headlines ("Tesla self-driving car stops in middle of highway!"), technology-ignorant bureaucrats will clamp down on self-driving cars rather than the modder who caused the problem, and Tesla will have to spend their time and resources dealing with the problems of unapproved inputs, rather than their core tasks. Easier to say, "just don't do it."

csallen 10y ago
We're not talking about modifying the self-driving algorithms. I assume those are a totally closed black box. We're merely talking about inputting destination addresses. What Tesla is saying with this no-Uber rule is, "This is where you can and cannot go." What happens when they, say, strike a deal with McDonald's and ban drivers from going to Burger King locations? It sounds far-fetched, but I don't see any difference. Where we take our cars and why should not be Tesla's business.
- snrplfth 10y ago
  But even just inputting destination addresses has its problems. What happens when an Uber-like app freaks out and spams the self-driving algos with a hundred destinations a second? Or screws up the location and sends a passenger a dozen miles from their intended destination? Or where the Tesla algos find a route it's been given to be undriveable and just turn back? Sure, you and I would rightly hold the modder accountable, but that's because we understand how this stuff works. Everybody else will see "Tesla cars" not working right, and that's a huge headache and liability for Tesla. We don't know exactly how "black box" their system is, and how easily it can be made immune to spurious (or even malicious!) inputs, while also leaving it open to third-party apps. I would guess it's just a risk that they don't want to take. And I don't think they'll ever ban people from going to places - as long as you use approved Tesla tools to request that destination.
  - deleted 10y ago
    [deleted]
  - csallen 10y ago
    > But even just inputting destination addresses has its problems. What happens when an Uber-like app freaks out and spams the self-driving algos with a hundred destinations a second? If Tesla's APIs can't handle input spam, that's on them. That's a very predictable and easily-solved problem. The entire point of an API is to expose a limited surface area so that you can easily interoperate while maintaining control and responsibility. > Or screws up the location and sends a passenger a dozen miles from their intended destination? Or where the Tesla algos find a route it's been given to be undriveable and just turn back? Sure, you and I would rightly hold the modder accountable, but that's because we understand how this stuff works. Everybody else will see "Tesla cars" not working right, and that's a huge headache and liability for Tesla. If Uber sends your car to the wrong location, then that's that. No big deal. People go to the wrong locations themselves all the time. I don't see how this is an Uber-specific problem, and I don't buy the argument that people are smart enough to hook their Tesla up to Uber but too stupid to understand who's at fault when the car correctly navigates itself to an incorrect location. > And I don't think they'll ever ban people from going to places - as long as you use approved Tesla tools to request that destination. There is no "approved Tesla tool" to connect to Uber. What happens if they modify their "approved Tesla tools" so they won't auto-drive you to a Burger King?
    - snrplfth 10y ago
      > If Tesla's APIs can't handle input spam, that's on them. That's a very predictable and easily-solved problem. The entire point of an API is to expose a limited surface area so that you can easily interoperate while maintaining control and responsibility. Sure, it's on them, and it appears they've solved that problem by cutting out other apps entirely. Certainly I'd prefer it if they would let people experiment with their product, but I can see why they wouldn't want them to, considering that it's a rapidly evolving, high-stakes product. > If Uber sends your car to the wrong location, then that's that. No big deal. People go to the wrong locations themselves all the time. I don't see how this is an Uber-specific problem, and I don't buy the argument that people are smart enough to hook their Tesla up to Uber but too stupid to understand who's at fault when the car correctly navigates itself to an incorrect location. It's not "Uber-specific", it's just an example of the sort of thing that could happen if a third-party app makes errors. Reasonable people will understand that it's the app at fault, but most people will just see the headline: "Panicked partiers trapped as Tesla self-driving car goes on a robot joyride!" > There is no "approved Tesla tool" to connect to Uber. What happens if they modify their "approved Tesla tools" so they won't auto-drive you to a Burger King? Tesla disallowing third-party apps and self-driving-for-revenue outside Tesla Network doesn't make a difference to whether they could do such a thing. But you can see that they have sensible reasons to not want these things (software concerns, commercial insurance, public/regulator perception) and there's no real reason that they would block you from going to particular destinations. Internet Explorer could block you from using anything but Bing, but they don't. That would be ridiculous.
- nsgi 10y ago
  What happens when Google strike a deal with McDonalds and prevent Google Maps from giving people directions to Burger King locations?
snowwrestler 10y ago
> But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. Why not? It flies just fine now with physical goods. If someone hand-builds their own bicycle wheel, and installs it on their bicycle, and then the wheel breaks and they crash... no one is blaming the bike manufacturer. No one is saying "that bike company should have welded the safer wheels on and issued a license prohibiting wheel alterations." Everyone gets that the customer took the liability in their own hands once they started using their own hands to modify the product. It's the classic trade-off of freedom vs. security. This ability to allocate liability to the customer is part of what we DO need for software, IMO. Right now there is a popular perception that software cannot be modified or interacted with by anyone but the company who wrote it. I don't think that has to be true forever though.
- snrplfth 10y ago
  See my answer to AnthonyMouse below for my thoughts on this. I'm not saying that it's ideal for Tesla to lock down the system like this. I'm saying that, given the risks they face, in terms of legislation, regulation and public opinion, their approach is very understandable.
  - snowwrestler 10y ago
    I agree that it's understandable in the current environment. I'm interested in thinking about how the environment would need to change to give us the company behavior we want.
    - snrplfth 10y ago
      You mean for society to quickly turn into technophile contractarians? I'm not holding my breath, personally.
      - snowwrestler 10y ago
        I'd be happy to start with a framework that permits the technophiles who already exist to work on the products they already own.
AnthonyMouse 10y ago
> But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. Why not? It's how everything else works. If you buy a house and then install your own electrical wiring which causes your house to burn down, you have no claim against the construction company and nobody blames them for it because it was your fault, not theirs. This recent attitude of "people won't understand, therefore corporations have to be paternalistic" is patronizing and factually incorrect. People do actually understand. The worst outcome is that technology-ignorant bureaucrats require companies to clamp down on modding. But having companies do that to begin with is no improvement. The basis of this seems to be the conceit that manufacturers can actually control their products after they've been sold. It can be true for a time, in the sense that it takes people that long to figure out how to break back into the things they own, but it happens. If you can jailbreak an iPhone then you can jailbreak your car. Which means that we have a choice. The first option is to accept the inevitable and support it. Expect people to make changes and take responsibility for their own changes, and facilitate that. The second is to try to lock everything down, so that the people making modifications have no access to documentation and have to be running out of date software because the old version with the jailbreak vulnerability is the same old version with the anti-lock brake glitch, which makes it much more likely that people will die. What does that do for your brand?
- snrplfth 10y ago
  > Why not? It's how everything else works. If you buy a house and then install your own electrical wiring which causes your house to burn down, you have no claim against the construction company and nobody blames them for it because it was your fault, not theirs. This recent attitude of "people won't understand, therefore corporations have to be paternalistic" is patronizing and factually incorrect. People do actually understand. I think you're really overestimating both the level of technical comprehension generally, and people's attitude towards "due diligence". People understand what a house is and what wires are. They are not, as of yet, that clear on how neural nets, sensor systems, and dynamically updated software come together to make a self-driving vehicle. If modders cause problems, especially with something as mobile and potentially destructive as a car, the general reaction could easily be "this is mostly the fault of the self-driving Tesla, and the fact that they didn't do due diligence in preventing misuse." It is not a very reasonable reaction, but it is a likely one - and one that could have serious consequences for Tesla as a company. >The worst outcome is that technology-ignorant bureaucrats require companies to clamp down on modding. But having companies do that to begin with is no improvement. It's not the same. A bureaucrat's rules are mandatory for everybody. Tesla has competitors who may choose a different route. > The basis of this seems to be the conceit that manufacturers can actually control their products after they've been sold. It can be true for a time, in the sense that it takes people that long to figure out how to break back into the things they own, but it happens. If you can jailbreak an iPhone then you can jailbreak your car. A self-driving Tesla is not an iPhone, because an iPhone is unlikely to run somebody over or block a freeway. Risks are relevant. Self-driving systems are also dynamically updated and dependent on networked information. It already is a service, not a fixed product. You don't own a self-driving system (at least Tesla's idea of it) any more than you own an Amazon Web Services server rack by having something hosted on it. > Which means that we have a choice. The first option [....] which makes it much more likely that people will die. What does that do for your brand? Maybe in the future, when such things are more familiar, Tesla will feel confident enough to open their products up to modders. But right now? When even the idea of self-driving cars is a challenging sell, and they're being criticized for it not being secure and consistent enough? No, not now.

csallen 10y ago

We're not talking about modifying the self-driving algorithms. I assume those are a totally closed black box. We're merely talking about inputting destination addresses. What Tesla is saying with this no-Uber rule is, "This is where you can and cannot go." What happens when they, say, strike a deal with McDonald's and ban drivers from going to Burger King locations? It sounds far-fetched, but I don't see any difference. Where we take our cars and why should not be Tesla's business.

snrplfth 10y ago
But even just inputting destination addresses has its problems. What happens when an Uber-like app freaks out and spams the self-driving algos with a hundred destinations a second? Or screws up the location and sends a passenger a dozen miles from their intended destination? Or where the Tesla algos find a route it's been given to be undriveable and just turn back? Sure, you and I would rightly hold the modder accountable, but that's because we understand how this stuff works. Everybody else will see "Tesla cars" not working right, and that's a huge headache and liability for Tesla. We don't know exactly how "black box" their system is, and how easily it can be made immune to spurious (or even malicious!) inputs, while also leaving it open to third-party apps. I would guess it's just a risk that they don't want to take. And I don't think they'll ever ban people from going to places - as long as you use approved Tesla tools to request that destination.
- deleted 10y ago
  [deleted]
- csallen 10y ago
  > But even just inputting destination addresses has its problems. What happens when an Uber-like app freaks out and spams the self-driving algos with a hundred destinations a second? If Tesla's APIs can't handle input spam, that's on them. That's a very predictable and easily-solved problem. The entire point of an API is to expose a limited surface area so that you can easily interoperate while maintaining control and responsibility. > Or screws up the location and sends a passenger a dozen miles from their intended destination? Or where the Tesla algos find a route it's been given to be undriveable and just turn back? Sure, you and I would rightly hold the modder accountable, but that's because we understand how this stuff works. Everybody else will see "Tesla cars" not working right, and that's a huge headache and liability for Tesla. If Uber sends your car to the wrong location, then that's that. No big deal. People go to the wrong locations themselves all the time. I don't see how this is an Uber-specific problem, and I don't buy the argument that people are smart enough to hook their Tesla up to Uber but too stupid to understand who's at fault when the car correctly navigates itself to an incorrect location. > And I don't think they'll ever ban people from going to places - as long as you use approved Tesla tools to request that destination. There is no "approved Tesla tool" to connect to Uber. What happens if they modify their "approved Tesla tools" so they won't auto-drive you to a Burger King?
  - snrplfth 10y ago
    > If Tesla's APIs can't handle input spam, that's on them. That's a very predictable and easily-solved problem. The entire point of an API is to expose a limited surface area so that you can easily interoperate while maintaining control and responsibility. Sure, it's on them, and it appears they've solved that problem by cutting out other apps entirely. Certainly I'd prefer it if they would let people experiment with their product, but I can see why they wouldn't want them to, considering that it's a rapidly evolving, high-stakes product. > If Uber sends your car to the wrong location, then that's that. No big deal. People go to the wrong locations themselves all the time. I don't see how this is an Uber-specific problem, and I don't buy the argument that people are smart enough to hook their Tesla up to Uber but too stupid to understand who's at fault when the car correctly navigates itself to an incorrect location. It's not "Uber-specific", it's just an example of the sort of thing that could happen if a third-party app makes errors. Reasonable people will understand that it's the app at fault, but most people will just see the headline: "Panicked partiers trapped as Tesla self-driving car goes on a robot joyride!" > There is no "approved Tesla tool" to connect to Uber. What happens if they modify their "approved Tesla tools" so they won't auto-drive you to a Burger King? Tesla disallowing third-party apps and self-driving-for-revenue outside Tesla Network doesn't make a difference to whether they could do such a thing. But you can see that they have sensible reasons to not want these things (software concerns, commercial insurance, public/regulator perception) and there's no real reason that they would block you from going to particular destinations. Internet Explorer could block you from using anything but Bing, but they don't. That would be ridiculous.
nsgi 10y ago
What happens when Google strike a deal with McDonalds and prevent Google Maps from giving people directions to Burger King locations?

snowwrestler 10y ago

> But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. Why not? It flies just fine now with physical goods. If someone hand-builds their own bicycle wheel, and installs it on their bicycle, and then the wheel breaks and they crash... no one is blaming the bike manufacturer. No one is saying "that bike company should have welded the safer wheels on and issued a license prohibiting wheel alterations." Everyone gets that the customer took the liability in their own hands once they started using their own hands to modify the product. It's the classic trade-off of freedom vs. security. This ability to allocate liability to the customer is part of what we DO need for software, IMO. Right now there is a popular perception that software cannot be modified or interacted with by anyone but the company who wrote it. I don't think that has to be true forever though.

snrplfth 10y ago
See my answer to AnthonyMouse below for my thoughts on this. I'm not saying that it's ideal for Tesla to lock down the system like this. I'm saying that, given the risks they face, in terms of legislation, regulation and public opinion, their approach is very understandable.
- snowwrestler 10y ago
  I agree that it's understandable in the current environment. I'm interested in thinking about how the environment would need to change to give us the company behavior we want.
  - snrplfth 10y ago
    You mean for society to quickly turn into technophile contractarians? I'm not holding my breath, personally.
    - snowwrestler 10y ago
      I'd be happy to start with a framework that permits the technophiles who already exist to work on the products they already own.

AnthonyMouse 10y ago

> But saying "Okay, but we're not responsible for damages" will not fly with regulators, legislators, public opinion or the integrity of their brand. Why not? It's how everything else works. If you buy a house and then install your own electrical wiring which causes your house to burn down, you have no claim against the construction company and nobody blames them for it because it was your fault, not theirs. This recent attitude of "people won't understand, therefore corporations have to be paternalistic" is patronizing and factually incorrect. People do actually understand. The worst outcome is that technology-ignorant bureaucrats require companies to clamp down on modding. But having companies do that to begin with is no improvement. The basis of this seems to be the conceit that manufacturers can actually control their products after they've been sold. It can be true for a time, in the sense that it takes people that long to figure out how to break back into the things they own, but it happens. If you can jailbreak an iPhone then you can jailbreak your car. Which means that we have a choice. The first option is to accept the inevitable and support it. Expect people to make changes and take responsibility for their own changes, and facilitate that. The second is to try to lock everything down, so that the people making modifications have no access to documentation and have to be running out of date software because the old version with the jailbreak vulnerability is the same old version with the anti-lock brake glitch, which makes it much more likely that people will die. What does that do for your brand?

snrplfth 10y ago
> Why not? It's how everything else works. If you buy a house and then install your own electrical wiring which causes your house to burn down, you have no claim against the construction company and nobody blames them for it because it was your fault, not theirs. This recent attitude of "people won't understand, therefore corporations have to be paternalistic" is patronizing and factually incorrect. People do actually understand. I think you're really overestimating both the level of technical comprehension generally, and people's attitude towards "due diligence". People understand what a house is and what wires are. They are not, as of yet, that clear on how neural nets, sensor systems, and dynamically updated software come together to make a self-driving vehicle. If modders cause problems, especially with something as mobile and potentially destructive as a car, the general reaction could easily be "this is mostly the fault of the self-driving Tesla, and the fact that they didn't do due diligence in preventing misuse." It is not a very reasonable reaction, but it is a likely one - and one that could have serious consequences for Tesla as a company. >The worst outcome is that technology-ignorant bureaucrats require companies to clamp down on modding. But having companies do that to begin with is no improvement. It's not the same. A bureaucrat's rules are mandatory for everybody. Tesla has competitors who may choose a different route. > The basis of this seems to be the conceit that manufacturers can actually control their products after they've been sold. It can be true for a time, in the sense that it takes people that long to figure out how to break back into the things they own, but it happens. If you can jailbreak an iPhone then you can jailbreak your car. A self-driving Tesla is not an iPhone, because an iPhone is unlikely to run somebody over or block a freeway. Risks are relevant. Self-driving systems are also dynamically updated and dependent on networked information. It already is a service, not a fixed product. You don't own a self-driving system (at least Tesla's idea of it) any more than you own an Amazon Web Services server rack by having something hosted on it. > Which means that we have a choice. The first option [....] which makes it much more likely that people will die. What does that do for your brand? Maybe in the future, when such things are more familiar, Tesla will feel confident enough to open their products up to modders. But right now? When even the idea of self-driving cars is a challenging sell, and they're being criticized for it not being secure and consistent enough? No, not now.