5 ms·

Ok would anyone like to do a long term bet with me? I bet that Carmack/Atwood's bet will be too ambiguous to resolve, and, neither wanting to look bad, they wil

Ok would anyone like to do a long term bet with me? I bet that Carmack/Atwood's bet will be too ambiguous to resolve, and, neither wanting to look bad, they will both pay out to charity.

ajross 5y ago
If it comes down to ambiguity, it will be on the minutiae of what "Level 5" (full autonomy / no human in the loop) actually means in practice. Because in the world we live in, all these vehicles will be networked regardless. The marginal cost of having a human available for whatever fallbacks are needed approaches zero, which means that even if L5 were technically achievable there's little value in actually deploying anything like that. But for sure count me with Carmack. My car is already driving me around my western suburban environment without trouble; zero-intervention drives are the rule now, not the exception. And it's been a long, long time since I've had to intervene for anything reasonably interpretable as a "safety" concern (mostly it's about stuff like navigation decisions, or the car is being a jerk at a merge and I don't want to be honked at, or it's being too slow pulling into an intersection and there's a lot of traffic, etc...). Don't be fooled by the memery regarding Tesla. There are 60k+ FSD beta cars on the roads now, all of them with eager drivers wanting to post interesting failures to youtube. If these things were genuinely having trouble, we'd know. It's not "done" for sure (in particular a lot of tuning needs to be done still), but it's at an effective level 3 already at least in my areas. And some work with remote recovery interfaces (something that Tesla doesn't seem to be focusing on) would get it to L4 in the kind of constrained environments Waymo and Cruise are operating in reasonably quickly, IMHO.
- danuker 5y ago
  "FSD" is a misnomer bordering fraud. The fine print: > The currently enabled features require a fully attentive driver, who has their hands on the wheel and is prepared to take over at any moment. https://www.tesla.com/support/full-self-driving-subscriptions https://www.tesla.com/support/full-self-driving-subscription...
- smcl 5y ago
  > If these things were genuinely having trouble, we'd know. Hm, I see videos of pretty terrible FSD fails all the time - stuff like the car veering off into the other lane, trying to drive the pillars of bridges, failing to spot pedestrians and other fun things. It's quite impressive that the system is nearly there, but that I suspect this is the famous final 20% which is gonna be a monumental challenge to overcome
  - gitfan86 5y ago
    It will never be 100% correct. But that has never been the goal. The goal is to be 10x safer than humans. I don't know exactly when it hits that level, but at this point there doesn't appear to be any reason it cannot get there eventually.
    - smcl 5y ago
      Oh eventually, yeah maybe? But reaching that in the next eight years seems a bit unsurmountable
  - a9h74j 5y ago
    Title of new policy paper: Driving in the uncanny valley
  - JacobThreeThree 5y ago
    >It's quite impressive that the system is nearly there, but that I suspect this is the famous final 20% which is gonna be a monumental challenge to overcome The self-driving AI marketers will tell you it's 99% of the way there! https://www.bloomberg.com/news/articles/2021-08-17/waymo-s-self-driving-cars-are-99-of-the-way-there-the-last-1-is-the-hardest https://www.bloomberg.com/news/articles/2021-08-17/waymo-s-s... I'm definitely in the Atwood boat.
- itsoktocry 5y ago
  >Don't be fooled by the memery regarding Tesla. There are 60k+ FSD beta cars on the roads now, all of them with eager drivers wanting to post interesting failures to youtube. If these things were genuinely having trouble, we'd know. You are completely out of touch with reality if you think the people driving "FSD" beta are eagerly trying to show fault. It's precisely the opposite. You think those who get beta are randomly chosen? >It's at an effective level 3 already at least in my areas. That's a loooonnng way from an unattended drive across the country. You own a car and, I bet, a financial stake in Tesla. Am I right?
  - ajross 5y ago
    > You think those who get beta are randomly chosen? Certainly not. We qualify with a safety score and wait for a new batch of cars to be added. I genuinely don't understand the level of vitriol about this product. There are a lot of these cars out there. Given the demographics of this site, surely one of your friends has one already. Call them up and get a ride. See it for yourself. I mean, everything looks terrible if you only watch greatest hits on youtube. There are literally whole subreddits dedicated to bad human driving, even. Yet... things work out. This is working far more than it's not, is all I'm saying. And the echo chamber effect has IMHO gotten a little out of control. This isn't a hard technology to find for yourself. Go try it.
    - itsoktocry 5y ago
      >This isn't a hard technology to find for yourself. Go try it I don't rely on anecdata when it comes to public safety and law. Where is the data from Tesla?
    - clouddrover 5y ago
      > I genuinely don't understand the level of vitriol about this product. The product doesn't exist. It has not been delivered. Tesla has consistently lied about full-self driving. Here are some lies about full-self driving hardware: https://www.tesla.com/blog/all-tesla-cars-being-produced-now-have-full-self-driving-hardware https://www.tesla.com/blog/all-tesla-cars-being-produced-now... Here are some further lies about full-self driving delivery: https://jalopnik.com/elon-musk-promises-full-self-driving-next-year-for-th-1848432496 https://jalopnik.com/elon-musk-promises-full-self-driving-ne...
- kajecounterhack 5y ago
  > And some work with remote recovery interfaces (something that Tesla doesn't seem to be focusing on) would get it to L4 in the kind of constrained environments Waymo and Cruise are operating in reasonably quickly, IMHO. It takes more than that. Just think, Uber's driver had lidar, safety driver, and still killed a woman. There's _a lot_ going on under the hood that separates the L4 from L2/L3 drivers. One thing I think Tesla's missing is a repeatable evaluation framework for how their software performs (a.k.a what you need to convince yourself of the safety). As I understand it, they mostly test autonomous software by deploying it in ghost mode (current/future miles) instead of simulating on past miles. A lot of long tail problems (which is what's hard about autonomous driving) are only visible on the order of millions of miles. Some cases you may never encounter at a high enough rate, and you may need to build synthetic data or use other tricks to teach your driver how to act. Consider construction zones: where are the lanes formed by the cones? Where should I enter, and where should cars come shooting out? Every construction zone is quite nuanced and contextual -- can you prove the autonomous driver can handle situations reasonably? I think Tesla's software isn't close to being able to crack those driving situations. I mean they make basic mistakes with traffic lights still (which are very challenging in and of themselves; how about those lights where there's an arrow? light that is out?) If you lack infra to save difficult cases and no ability to generate synthetic situations and resimulate, it gets real dicey when you try to draw conclusions about the real-world performance. Their simulation capabilities are also limited by the fact that they only use cameras; depth estimation / object contours are not accurate enough to do the kinds of sims the Cruise and Waymo do. Without good ground truth it's really hard to do proper safety evaluation. I suspect Tesla would either need a huge breakthrough in vision technology (maybe they can use something like this https://waymo.com/research/block-nerf https://waymo.com/research/block-nerf) or at least map places their cars are driving so they can properly assess rates to give themselves, regulators, and customers confidence that they are safe for L4. Otherwise, their "good performance" will just be a product of survivorship bias, and bad performance will really hurt people (this is kind of happening now).
csours 5y ago
I would bet 10k that the definition of Level 5 will change enough to make this a moot point, or that another new classification system will be introduced. There are so many assumptions in this space that it's a fool's errand to try to list them. I think the next paradigm shift will be in the cultural and regulatory perception, not in technology. I have NO IDEA what that might mean, and I could be completely wrong! I accept that. I think the technology has to demonstrate what it can do, and society has to come to terms with what the tech can do, and by doing those things we will painfully update our priors. Our existing prior beliefs will not be our new prior beliefs.
- type_enthusiast 5y ago
  Changing classifications aren't going to affect the bet, because both of these gents know what they're betting on. If I could make a $10,000 bet with John Carmack, I'd go the way Atwood has. No way there will be fully self-driving cars by 2030. That's only 8 years away. Fully self-driving cars need one of two things: A) All cars are self-driving, and are networked, and centrally controlled. No exceptions. B) The AI is smart enough that we should be worried about the ethical implications. If an AI can react to terrible human drivers as well as a good human driver can, then I'd invite that AI over for dinner.
  - Germanika 5y ago
    A) also doesn't work unless all other road users (bikes, scooters, skateboards, pedestrians, etc.) are also banned from the roads. The other option is limiting the autonomous cars to tunnels or dedicated tracks, but then we're just reinventing trains again.
    - kajecounterhack 5y ago
      L4 is basically reinventing trains, with invisible tracks in the form of high-def maps. Even that is just so hard ;_;
    - foobar2021 5y ago
      Given how many people you see driving down the road staring at their phones, reinvention of trains is probably still a good thing for society.
rvz 5y ago
Well, I remember the last time someone did such a bet with me. [0] It seems that, whenever the ‘predictor’ turned out to be wrong, they seem to have gone into ‘hiding’. I wonder why that is in my case… So as for your offer, its a no thanks and no deal from me. [0] https://news.ycombinator.com/item?id=27046521 https://news.ycombinator.com/item?id=27046521
- robbrown451 5y ago
  Heh yeah well I would probably never actually do a long term bet with anyone over the internet, unless I was well known like these guys are, so they can't really hide. That said I wish there was a good site that allowed doing bets like this online, where you put the money in up front, and deputize a list of people to be judges to decide the winner. (yes it would typically be shorter term than 8 years, and obviously there would have to be rules for dealing with what happens if the judges aren't all available, incentives to reduce the chance of that happening, and so on) I imagine anti-gambling laws make it difficult to do such things. But I can wish. It would reduce the number of people making poorly thought out assertions as to the future.
  - sgerenser 5y ago
    https://longbets.org/ https://longbets.org/
    - robbrown451 5y ago
      Interesting. I wish it didn't require it to be to charity, and wish it had better ways of judging outcome, and wish it allowed any bet (too many requirements as to bet type). But still interesting.
tim333 5y ago
I read the definition and it wouldn't be too hard to narrow it down. It's basically on whether you'll be able to get a robotaxi in a place like NYC, tell it to go anywhere you want and take a nap while it drives. Which seems fairly clear cut. (They have more than one top 10 US city rather than NYC).

Ok would anyone like to do a long term bet with me? I bet that Carmack/Atwood's bet will be too ambiguous to resolve, and, neither wanting to look bad, they will both pay out to charity.

ajross 5y ago

If it comes down to ambiguity, it will be on the minutiae of what "Level 5" (full autonomy / no human in the loop) actually means in practice. Because in the world we live in, all these vehicles will be networked regardless. The marginal cost of having a human available for whatever fallbacks are needed approaches zero, which means that even if L5 were technically achievable there's little value in actually deploying anything like that. But for sure count me with Carmack. My car is already driving me around my western suburban environment without trouble; zero-intervention drives are the rule now, not the exception. And it's been a long, long time since I've had to intervene for anything reasonably interpretable as a "safety" concern (mostly it's about stuff like navigation decisions, or the car is being a jerk at a merge and I don't want to be honked at, or it's being too slow pulling into an intersection and there's a lot of traffic, etc...). Don't be fooled by the memery regarding Tesla. There are 60k+ FSD beta cars on the roads now, all of them with eager drivers wanting to post interesting failures to youtube. If these things were genuinely having trouble, we'd know. It's not "done" for sure (in particular a lot of tuning needs to be done still), but it's at an effective level 3 already at least in my areas. And some work with remote recovery interfaces (something that Tesla doesn't seem to be focusing on) would get it to L4 in the kind of constrained environments Waymo and Cruise are operating in reasonably quickly, IMHO.

danuker 5y ago
"FSD" is a misnomer bordering fraud. The fine print: > The currently enabled features require a fully attentive driver, who has their hands on the wheel and is prepared to take over at any moment. https://www.tesla.com/support/full-self-driving-subscriptions https://www.tesla.com/support/full-self-driving-subscription...
smcl 5y ago
> If these things were genuinely having trouble, we'd know. Hm, I see videos of pretty terrible FSD fails all the time - stuff like the car veering off into the other lane, trying to drive the pillars of bridges, failing to spot pedestrians and other fun things. It's quite impressive that the system is nearly there, but that I suspect this is the famous final 20% which is gonna be a monumental challenge to overcome
- gitfan86 5y ago
  It will never be 100% correct. But that has never been the goal. The goal is to be 10x safer than humans. I don't know exactly when it hits that level, but at this point there doesn't appear to be any reason it cannot get there eventually.
  - smcl 5y ago
    Oh eventually, yeah maybe? But reaching that in the next eight years seems a bit unsurmountable
- a9h74j 5y ago
  Title of new policy paper: Driving in the uncanny valley
- JacobThreeThree 5y ago
  >It's quite impressive that the system is nearly there, but that I suspect this is the famous final 20% which is gonna be a monumental challenge to overcome The self-driving AI marketers will tell you it's 99% of the way there! https://www.bloomberg.com/news/articles/2021-08-17/waymo-s-self-driving-cars-are-99-of-the-way-there-the-last-1-is-the-hardest https://www.bloomberg.com/news/articles/2021-08-17/waymo-s-s... I'm definitely in the Atwood boat.
itsoktocry 5y ago
>Don't be fooled by the memery regarding Tesla. There are 60k+ FSD beta cars on the roads now, all of them with eager drivers wanting to post interesting failures to youtube. If these things were genuinely having trouble, we'd know. You are completely out of touch with reality if you think the people driving "FSD" beta are eagerly trying to show fault. It's precisely the opposite. You think those who get beta are randomly chosen? >It's at an effective level 3 already at least in my areas. That's a loooonnng way from an unattended drive across the country. You own a car and, I bet, a financial stake in Tesla. Am I right?
- ajross 5y ago
  > You think those who get beta are randomly chosen? Certainly not. We qualify with a safety score and wait for a new batch of cars to be added. I genuinely don't understand the level of vitriol about this product. There are a lot of these cars out there. Given the demographics of this site, surely one of your friends has one already. Call them up and get a ride. See it for yourself. I mean, everything looks terrible if you only watch greatest hits on youtube. There are literally whole subreddits dedicated to bad human driving, even. Yet... things work out. This is working far more than it's not, is all I'm saying. And the echo chamber effect has IMHO gotten a little out of control. This isn't a hard technology to find for yourself. Go try it.
  - itsoktocry 5y ago
    >This isn't a hard technology to find for yourself. Go try it I don't rely on anecdata when it comes to public safety and law. Where is the data from Tesla?
  - clouddrover 5y ago
    > I genuinely don't understand the level of vitriol about this product. The product doesn't exist. It has not been delivered. Tesla has consistently lied about full-self driving. Here are some lies about full-self driving hardware: https://www.tesla.com/blog/all-tesla-cars-being-produced-now-have-full-self-driving-hardware https://www.tesla.com/blog/all-tesla-cars-being-produced-now... Here are some further lies about full-self driving delivery: https://jalopnik.com/elon-musk-promises-full-self-driving-next-year-for-th-1848432496 https://jalopnik.com/elon-musk-promises-full-self-driving-ne...
kajecounterhack 5y ago
> And some work with remote recovery interfaces (something that Tesla doesn't seem to be focusing on) would get it to L4 in the kind of constrained environments Waymo and Cruise are operating in reasonably quickly, IMHO. It takes more than that. Just think, Uber's driver had lidar, safety driver, and still killed a woman. There's _a lot_ going on under the hood that separates the L4 from L2/L3 drivers. One thing I think Tesla's missing is a repeatable evaluation framework for how their software performs (a.k.a what you need to convince yourself of the safety). As I understand it, they mostly test autonomous software by deploying it in ghost mode (current/future miles) instead of simulating on past miles. A lot of long tail problems (which is what's hard about autonomous driving) are only visible on the order of millions of miles. Some cases you may never encounter at a high enough rate, and you may need to build synthetic data or use other tricks to teach your driver how to act. Consider construction zones: where are the lanes formed by the cones? Where should I enter, and where should cars come shooting out? Every construction zone is quite nuanced and contextual -- can you prove the autonomous driver can handle situations reasonably? I think Tesla's software isn't close to being able to crack those driving situations. I mean they make basic mistakes with traffic lights still (which are very challenging in and of themselves; how about those lights where there's an arrow? light that is out?) If you lack infra to save difficult cases and no ability to generate synthetic situations and resimulate, it gets real dicey when you try to draw conclusions about the real-world performance. Their simulation capabilities are also limited by the fact that they only use cameras; depth estimation / object contours are not accurate enough to do the kinds of sims the Cruise and Waymo do. Without good ground truth it's really hard to do proper safety evaluation. I suspect Tesla would either need a huge breakthrough in vision technology (maybe they can use something like this https://waymo.com/research/block-nerf https://waymo.com/research/block-nerf) or at least map places their cars are driving so they can properly assess rates to give themselves, regulators, and customers confidence that they are safe for L4. Otherwise, their "good performance" will just be a product of survivorship bias, and bad performance will really hurt people (this is kind of happening now).

csours 5y ago

I would bet 10k that the definition of Level 5 will change enough to make this a moot point, or that another new classification system will be introduced. There are so many assumptions in this space that it's a fool's errand to try to list them. I think the next paradigm shift will be in the cultural and regulatory perception, not in technology. I have NO IDEA what that might mean, and I could be completely wrong! I accept that. I think the technology has to demonstrate what it can do, and society has to come to terms with what the tech can do, and by doing those things we will painfully update our priors. Our existing prior beliefs will not be our new prior beliefs.

type_enthusiast 5y ago
Changing classifications aren't going to affect the bet, because both of these gents know what they're betting on. If I could make a $10,000 bet with John Carmack, I'd go the way Atwood has. No way there will be fully self-driving cars by 2030. That's only 8 years away. Fully self-driving cars need one of two things: A) All cars are self-driving, and are networked, and centrally controlled. No exceptions. B) The AI is smart enough that we should be worried about the ethical implications. If an AI can react to terrible human drivers as well as a good human driver can, then I'd invite that AI over for dinner.
- Germanika 5y ago
  A) also doesn't work unless all other road users (bikes, scooters, skateboards, pedestrians, etc.) are also banned from the roads. The other option is limiting the autonomous cars to tunnels or dedicated tracks, but then we're just reinventing trains again.
  - kajecounterhack 5y ago
    L4 is basically reinventing trains, with invisible tracks in the form of high-def maps. Even that is just so hard ;_;
  - foobar2021 5y ago
    Given how many people you see driving down the road staring at their phones, reinvention of trains is probably still a good thing for society.

rvz 5y ago

Well, I remember the last time someone did such a bet with me. [0] It seems that, whenever the ‘predictor’ turned out to be wrong, they seem to have gone into ‘hiding’. I wonder why that is in my case… So as for your offer, its a no thanks and no deal from me. [0] https://news.ycombinator.com/item?id=27046521 https://news.ycombinator.com/item?id=27046521

robbrown451 5y ago
Heh yeah well I would probably never actually do a long term bet with anyone over the internet, unless I was well known like these guys are, so they can't really hide. That said I wish there was a good site that allowed doing bets like this online, where you put the money in up front, and deputize a list of people to be judges to decide the winner. (yes it would typically be shorter term than 8 years, and obviously there would have to be rules for dealing with what happens if the judges aren't all available, incentives to reduce the chance of that happening, and so on) I imagine anti-gambling laws make it difficult to do such things. But I can wish. It would reduce the number of people making poorly thought out assertions as to the future.
- sgerenser 5y ago
  https://longbets.org/ https://longbets.org/
  - robbrown451 5y ago
    Interesting. I wish it didn't require it to be to charity, and wish it had better ways of judging outcome, and wish it allowed any bet (too many requirements as to bet type). But still interesting.

tim333 5y ago

I read the definition and it wouldn't be too hard to narrow it down. It's basically on whether you'll be able to get a robotaxi in a place like NYC, tell it to go anywhere you want and take a nap while it drives. Which seems fairly clear cut. (They have more than one top 10 US city rather than NYC).