7 ms·

> All evidence from their own AI/AP team and presentations is that there is no real design and system validation going on over there. They're flying by the seat

> All evidence from their own AI/AP team and presentations is that there is no real design and system validation going on over there. They're flying by the seat of their pants, introducing potentially lethal regressions in every update. What is this evidence? I've seen a few talks from Andrej Karpathy that indicate to me a more deliberate approach.[0] "Software 2.0" itself seems like an approach meant to systematize the development, validation & testing of AI systems, hardly a seat-of-your-pants approach to releases. I have my own criticisms of their approach, but it seems there is pretty deliberate care taken when developing models. [0] https://youtu.be/hx7BXih7zx8 https://youtu.be/hx7BXih7zx8

refulgentis 4y ago
I see your point, to OP's point, I know a couple people who were horrified at what they saw and it did not match this public talk. Both started at least 6 months after this video, and both left Tesla within 8 months, of their own volition. Unfortunately, off the record. Not to disparage Andrej, sometimes (frequently, even) what executive leadership thinks is going is not the day-to-day reality of the team.
- plankers 4y ago
  can confirm, a former coworker had just come from Tesla 5 years ago and he had serious ethical problems with his work over there. Tesla is killing people through negligence and greed, it's pretty disgusting, but par for the course
justapassenger 4y ago
I’ve been working few years ago at a very big tech company, focusing on validation of the AI systems. It’s all smoke and mirrors. You cannot perform proper validation of AI systems. Rollbacks of new versions of ML models are very common in production, and even after very extensive validation you can see that real life results are nothing like what tests have shown.
- urthor 4y ago
  Can't you do outlier detection, and disable the AI if the input wasn't in the training set?
  - justapassenger 4y ago
    How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that? You basically put another ML on top of ML, to correct it. I’ve seen that in use in production systems, and it helps with some problems and generates new ones. And if you thought that reasoning about correctness was hard before… And what do you mean by disabling AI, if input wasn’t in the training set? That’s the whole point of ML, to reason about new data based on data seen in past.
    - urthor 4y ago
      > That’s the whole point of ML, to reason about new data based on data seen in past. I think we like to think this is true. In reality, I have seen a lot of real world ML models. I wouldn't trust ANY of them to do extrapolation. There are just tons of real world problems, and extrapolation is HARD. I have to put extremely tight boundaries on ML models for deployment scenarios, and ALWAYS have a backup rule engine in case the ML model comes up with an answer that has a low confidence score. > How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that? > You need to write some rules that could look at it. Pretty much. Any time ML is involved, you will need TONS of lines of code. In short, tightly define the target classes your ML model deals with. Any variable that falls outside your tightly bound list of target classes, you have to deal with using a rules engine. THEN you need to spend a lot of time doing work to minimize false positive classification in your target classes. And make sure that "false positive, high confidence" classifications don't do racist things/lose the business a lot of money things. ML projects are just a ton of work. You essentially make the entire ML workflow, and you NEED a backup "not-ML" workflow. In my experience, 50-80% of normal software engineering projects fail. 90% of ML projects fail. Square the fraction of normal software projects. ML is complex AND it's a ton of work. Really, really hard.
stefan_ 4y ago
This is the Karpathy that gave a big talk about how vision was superior to radar when Tesla dropped all radar units at the height of the chip crisis. Now they are bringing radar back. Give it a few years and they will probably introduce LIDAR.
- pbronez 4y ago
  Tesla is bringing Radar back? First I've heard about it, and good news if true.
  - aaaaaaaaaaab 4y ago
    https://electrek.co/2022/06/08/tesla-files-use-new-radar-confused/ https://electrek.co/2022/06/08/tesla-files-use-new-radar-con...
    - epgui 4y ago
      Wasn’t this approval based on an application from 2018?
phkahler 4y ago
>> They're flying by the seat of their pants, introducing potentially lethal regressions in every update. >> What is this evidence? Without a documented development and testing program, every development is essentially this.
kelnos 4y ago
> What is this evidence? I think the onus should be on Tesla to prove that their testing and validation methodology is sufficient. Until and unless they have done so, Autopilot should be completely disabled. I really don't get why the regulatory environment is so behind here. None of these driver assistance technologies (from any manufacturer, not just Tesla) should be by default legal to put in a car.

refulgentis 4y ago

I see your point, to OP's point, I know a couple people who were horrified at what they saw and it did not match this public talk. Both started at least 6 months after this video, and both left Tesla within 8 months, of their own volition. Unfortunately, off the record. Not to disparage Andrej, sometimes (frequently, even) what executive leadership thinks is going is not the day-to-day reality of the team.

plankers 4y ago
can confirm, a former coworker had just come from Tesla 5 years ago and he had serious ethical problems with his work over there. Tesla is killing people through negligence and greed, it's pretty disgusting, but par for the course

justapassenger 4y ago

I’ve been working few years ago at a very big tech company, focusing on validation of the AI systems. It’s all smoke and mirrors. You cannot perform proper validation of AI systems. Rollbacks of new versions of ML models are very common in production, and even after very extensive validation you can see that real life results are nothing like what tests have shown.

urthor 4y ago
Can't you do outlier detection, and disable the AI if the input wasn't in the training set?
- justapassenger 4y ago
  How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that? You basically put another ML on top of ML, to correct it. I’ve seen that in use in production systems, and it helps with some problems and generates new ones. And if you thought that reasoning about correctness was hard before… And what do you mean by disabling AI, if input wasn’t in the training set? That’s the whole point of ML, to reason about new data based on data seen in past.
  - urthor 4y ago
    > That’s the whole point of ML, to reason about new data based on data seen in past. I think we like to think this is true. In reality, I have seen a lot of real world ML models. I wouldn't trust ANY of them to do extrapolation. There are just tons of real world problems, and extrapolation is HARD. I have to put extremely tight boundaries on ML models for deployment scenarios, and ALWAYS have a backup rule engine in case the ML model comes up with an answer that has a low confidence score. > How do you identify the outlier? You need to write some rules that could look at it. But that’s a lot of rules. What if you could use computers to do that? > You need to write some rules that could look at it. Pretty much. Any time ML is involved, you will need TONS of lines of code. In short, tightly define the target classes your ML model deals with. Any variable that falls outside your tightly bound list of target classes, you have to deal with using a rules engine. THEN you need to spend a lot of time doing work to minimize false positive classification in your target classes. And make sure that "false positive, high confidence" classifications don't do racist things/lose the business a lot of money things. ML projects are just a ton of work. You essentially make the entire ML workflow, and you NEED a backup "not-ML" workflow. In my experience, 50-80% of normal software engineering projects fail. 90% of ML projects fail. Square the fraction of normal software projects. ML is complex AND it's a ton of work. Really, really hard.

stefan_ 4y ago

This is the Karpathy that gave a big talk about how vision was superior to radar when Tesla dropped all radar units at the height of the chip crisis. Now they are bringing radar back. Give it a few years and they will probably introduce LIDAR.

pbronez 4y ago
Tesla is bringing Radar back? First I've heard about it, and good news if true.
- aaaaaaaaaaab 4y ago
  https://electrek.co/2022/06/08/tesla-files-use-new-radar-confused/ https://electrek.co/2022/06/08/tesla-files-use-new-radar-con...
  - epgui 4y ago
    Wasn’t this approval based on an application from 2018?

phkahler 4y ago

>> They're flying by the seat of their pants, introducing potentially lethal regressions in every update. >> What is this evidence? Without a documented development and testing program, every development is essentially this.

kelnos 4y ago

> What is this evidence? I think the onus should be on Tesla to prove that their testing and validation methodology is sufficient. Until and unless they have done so, Autopilot should be completely disabled. I really don't get why the regulatory environment is so behind here. None of these driver assistance technologies (from any manufacturer, not just Tesla) should be by default legal to put in a car.