8 ms·

I don't know where you got these papers but you clearly got an awful sampling and I don't think you're giving computer vision a fair evaluation. Granted, we're

I don't know where you got these papers but you clearly got an awful sampling and I don't think you're giving computer vision a fair evaluation. Granted, we're not quite ready for self driving, but there's no question that the neural network subfield of ML has absolutely exploded in the last 5-10 years and is bursting with productionizable, application ready architectures that are already solving real world problems.

fxtentacle 7y ago
You sound like an NVIDIA salesperson trying to sell me on a $3000 Titan ;) There is no doubt in my mind that an AI with billions of parameters will be excellent at memorizing stuff. I also have no doubt that research activity has exploded, which might be related to the generous hardware grants being handed out... But all that research has produced surprisingly little progress over algorithms from 2004 in the field of optical flow. The papers I looked at were the top-ranked optical flow algorithms on Sintel and KITTY. So those were the AIs that work best in practice and better than 99% of the other AI solutions.
- skybrian 7y ago
  While it's not my area of expertise, I am a bit wary of contest results. It seems like an exercise in overfitting via multiple comparisons? Maybe some algorithms with a slightly lower rank are actually more robust? If it's as bad as you say, it seems like a critical evaluation would be pretty interesting and advance the field.
- scirocco 7y ago
  I wonder how many solutions within the AI field could just be categorized as "Automation"
  - DeedsMoraine 7y ago
    All of them. That's how AI works. Not by making smarter machines, but by destroying intelligence by smashing it into machine-digestible bits.
  - michaelwilson 7y ago
    That's what caused the first "AI Winter": Rules-based "AI" engines became what we call "Business Rules". AI didn't go away - it just stopped being an over-valued set of if-then logic and slots (value-driven triggers) with a cool set of GUI tools to build rule networks. Source: Used to work for an 80s-era "AI Company"
- allovernow 7y ago
  >There is no doubt in my mind that an AI with billions of parameters will be excellent at memorizing stuff We're way past memorization. We're into interpolation and extrapolation in high D spaces with Bayesian parameters. Sentiment analysis and contextual searches - search by idea, not keyword. Heuristic decision making. Massively accelerated 3D modeling with 99% accuracy. Generative nets for text, music, scientific models... Sorry, but you're behind the times, and that's ok - one of us will be proven right in the next 1-5 years. Based solely on the work we're doing at the startup I'm working for, we're on the cusp of an ML revolution. Time will tell, but personally I'm pretty excited. And don't worry, I'm not working in adtech or anything useless. That said, the driving problem seems to be quite far from being solved, I agree though it is outside my expertise; but I think the primary issue is that this is an application where error must be unrealistically low, a constraint which does not apply to many other domains. You can get away with a couple percent of uncertainty when people's lives aren't on the line!
  - SolaceQuantum 7y ago
    Would you be willing to link to some papers and cite some specific algos to play with? The above cited specific algos. What are the new versions of these named algorithms?
stephenr 7y ago
> Granted, we're not quite ready for self driving And yet it's literally in cars on the road. I'm not saying you're wrong because of that. I just wonder how far from "ready" we are, and how much of a gamble manufacturers are taking, and how much risk that presents for not just their customers, but everyone else their customers may drive near.
- ascar 7y ago
  Based on tesla's safety report [1] it's already less dangerous than letting humans drive (alone). The error rate of human drivers tends to be downplayed, while the perceived risks of automated driving is exaggerated, distorting the picture. Yes, it's a hard problem, yes we are not nearly there and there is a lot of development/research to do. Yes, accidents will happen during the process. But humans suck at driving and kill themselves and other people daily. It's the least safe form of transportation we have. [1] https://www.tesla.com/VehicleSafetyReport?redirect=no https://www.tesla.com/VehicleSafetyReport?redirect=no
  - notahacker 7y ago
    The gross human fatal accident rate is ~7 accidents per billion miles in the US, including fatalities caused by incompetent or irresponsible drivers, and substantially lower in Europe. But humans drive a lot of miles Based on Tesla's safety report, 'more than 1 billion' miles have been driven using autopilot. Given the small data sample and the fatalities already attributed to autopilot, I think we're some way from proving it's safer than letting drivers drive alone, never mind close to being a driver substitute.
    - thaumasiotes 7y ago
      Marginal Revolution just highlighted an interesting detailed look at US car (driver) fatalities. https://marginalrevolution.com/marginalrevolution/2020/02/what-causes-car-deaths-and-how-to-limit-them.html https://marginalrevolution.com/marginalrevolution/2020/02/wh... >> After accounting for freeways (18%) and intersections and junctions (20%), we’re still left with more than 60% of drivers killed in automotive accidents left accounted for. >> It turns out that drivers killed on rural roads with 2 lanes (i.e., one lane in each direction divided by a double yellow line) accounts for a staggering 38% of total mortality. This number would actually be higher, except to keep the three categories we have mutually exclusive, we backed out any intersection-related driver deaths on these roads and any killed on 2-lane rural roads that were classified as “freeway.” >> In drivers killed on 2-lane rural roads, 50% involved a driver not wearing a seat belt. Close to 40% have alcohol in their system and nearly 90% of these drivers were over the legal limit of 0.08 g/dL. I don't think people give enough attention to whether broad statistics actually apply to cases of interest. That's about 40% of all driver fatalities occurring on rural non-freeway roads, of which 35% (~14% overall) were legally driving drunk. People compare various fatality rates associated with riding an airplane vs driving a car all the time, but I've never seen anyone point out that an incredibly simple mitigation you're probably already doing -- not driving on non-freeway rural roads -- lowers your risk of dying in a car accident by more than a third. And it gets even better if you're not driving drunk! If you measure driving quality in terms of fatality rate, it is actually the case that almost everyone is better than average. A lot better than average. But public discussion completely misses this, because we prefer to aggregate unlike with unlike.
fxtentacle 7y ago
BTW, I would very much like to see progress in optical flow because I could really use it for one of my projects. If you know any good paper that tries a novel approach and doesn't just recycle the old SSIM+pyramid loss, please post the title or DOI :)
innagadadavida 7y ago
There is 40-50% drop in precision with state of the art results if test images are ill-formed. The imagenet dataset used is far from ready for real world use cases. A bunch of IBM and MIT researchers are trying to fix this - https://objectnet.dev/ https://objectnet.dev/
- Piskvorrr 7y ago
  As in, "only works in great visibility on a perfectly spherical road"? That does seem an appropriate summary.
nestorD 7y ago
> I don't know where you got these papers but you clearly got an awful sampling and I don't think you're giving computer vision a fair evaluation. I disagree, I saw such an awful number of bugs in ML codes going with papers that I now take for granted that there is a bug somewhere and hope that it does not impact the concept being evaluated. (here having everyone use python, a language that will try its best to run what you throw at him, feels like a very bad idea)
tsukurimashou 7y ago
> Granted, we're not quite ready for self driving if it was the case self driving cars wouldn't be on the road, I don't think we should aim for perfection, perfection will come. We should be looking for cars that make less errors on average than humans, once you have that you can start putting cars on the road and use data from the fleet to correct remaining errors.
- dontreact 7y ago
  Humans have an average of 1 fatality per 100 million miles driven. No one is anywhere close to 100 million miles without intervention.
- brailsafe 7y ago
  Are there any fully autonomous cars on public roads with no driver that can intervene? Seems like only maybe in tightly constrained situations are we ready.
  - tsukurimashou 7y ago
    I don't think I mentioned FULLY autonomous cars, my point was: something doesn't have to be perfect before we have to use it, but I probably didn't express myself correctly
    - brailsafe 7y ago
      I think that the necessity of intervening drivers atm indicates that we aren't at that point yet, even if that point is far from perfection, and also that the reason any self-driving cars are on the road is because of the fairly loose but significant requirements from regulation. We might be at that point in otherwise very dangerous situations, like if I was very tired or drunk, but otherwise I don't know that I'd have so much faith in software engineers to completely control my car.

fxtentacle 7y ago

You sound like an NVIDIA salesperson trying to sell me on a $3000 Titan ;) There is no doubt in my mind that an AI with billions of parameters will be excellent at memorizing stuff. I also have no doubt that research activity has exploded, which might be related to the generous hardware grants being handed out... But all that research has produced surprisingly little progress over algorithms from 2004 in the field of optical flow. The papers I looked at were the top-ranked optical flow algorithms on Sintel and KITTY. So those were the AIs that work best in practice and better than 99% of the other AI solutions.

skybrian 7y ago
While it's not my area of expertise, I am a bit wary of contest results. It seems like an exercise in overfitting via multiple comparisons? Maybe some algorithms with a slightly lower rank are actually more robust? If it's as bad as you say, it seems like a critical evaluation would be pretty interesting and advance the field.
scirocco 7y ago
I wonder how many solutions within the AI field could just be categorized as "Automation"
- DeedsMoraine 7y ago
  All of them. That's how AI works. Not by making smarter machines, but by destroying intelligence by smashing it into machine-digestible bits.
- michaelwilson 7y ago
  That's what caused the first "AI Winter": Rules-based "AI" engines became what we call "Business Rules". AI didn't go away - it just stopped being an over-valued set of if-then logic and slots (value-driven triggers) with a cool set of GUI tools to build rule networks. Source: Used to work for an 80s-era "AI Company"
allovernow 7y ago
>There is no doubt in my mind that an AI with billions of parameters will be excellent at memorizing stuff We're way past memorization. We're into interpolation and extrapolation in high D spaces with Bayesian parameters. Sentiment analysis and contextual searches - search by idea, not keyword. Heuristic decision making. Massively accelerated 3D modeling with 99% accuracy. Generative nets for text, music, scientific models... Sorry, but you're behind the times, and that's ok - one of us will be proven right in the next 1-5 years. Based solely on the work we're doing at the startup I'm working for, we're on the cusp of an ML revolution. Time will tell, but personally I'm pretty excited. And don't worry, I'm not working in adtech or anything useless. That said, the driving problem seems to be quite far from being solved, I agree though it is outside my expertise; but I think the primary issue is that this is an application where error must be unrealistically low, a constraint which does not apply to many other domains. You can get away with a couple percent of uncertainty when people's lives aren't on the line!
- SolaceQuantum 7y ago
  Would you be willing to link to some papers and cite some specific algos to play with? The above cited specific algos. What are the new versions of these named algorithms?

stephenr 7y ago

> Granted, we're not quite ready for self driving And yet it's literally in cars on the road. I'm not saying you're wrong because of that. I just wonder how far from "ready" we are, and how much of a gamble manufacturers are taking, and how much risk that presents for not just their customers, but everyone else their customers may drive near.

ascar 7y ago
Based on tesla's safety report [1] it's already less dangerous than letting humans drive (alone). The error rate of human drivers tends to be downplayed, while the perceived risks of automated driving is exaggerated, distorting the picture. Yes, it's a hard problem, yes we are not nearly there and there is a lot of development/research to do. Yes, accidents will happen during the process. But humans suck at driving and kill themselves and other people daily. It's the least safe form of transportation we have. [1] https://www.tesla.com/VehicleSafetyReport?redirect=no https://www.tesla.com/VehicleSafetyReport?redirect=no
- notahacker 7y ago
  The gross human fatal accident rate is ~7 accidents per billion miles in the US, including fatalities caused by incompetent or irresponsible drivers, and substantially lower in Europe. But humans drive a lot of miles Based on Tesla's safety report, 'more than 1 billion' miles have been driven using autopilot. Given the small data sample and the fatalities already attributed to autopilot, I think we're some way from proving it's safer than letting drivers drive alone, never mind close to being a driver substitute.
  - thaumasiotes 7y ago
    Marginal Revolution just highlighted an interesting detailed look at US car (driver) fatalities. https://marginalrevolution.com/marginalrevolution/2020/02/what-causes-car-deaths-and-how-to-limit-them.html https://marginalrevolution.com/marginalrevolution/2020/02/wh... >> After accounting for freeways (18%) and intersections and junctions (20%), we’re still left with more than 60% of drivers killed in automotive accidents left accounted for. >> It turns out that drivers killed on rural roads with 2 lanes (i.e., one lane in each direction divided by a double yellow line) accounts for a staggering 38% of total mortality. This number would actually be higher, except to keep the three categories we have mutually exclusive, we backed out any intersection-related driver deaths on these roads and any killed on 2-lane rural roads that were classified as “freeway.” >> In drivers killed on 2-lane rural roads, 50% involved a driver not wearing a seat belt. Close to 40% have alcohol in their system and nearly 90% of these drivers were over the legal limit of 0.08 g/dL. I don't think people give enough attention to whether broad statistics actually apply to cases of interest. That's about 40% of all driver fatalities occurring on rural non-freeway roads, of which 35% (~14% overall) were legally driving drunk. People compare various fatality rates associated with riding an airplane vs driving a car all the time, but I've never seen anyone point out that an incredibly simple mitigation you're probably already doing -- not driving on non-freeway rural roads -- lowers your risk of dying in a car accident by more than a third. And it gets even better if you're not driving drunk! If you measure driving quality in terms of fatality rate, it is actually the case that almost everyone is better than average. A lot better than average. But public discussion completely misses this, because we prefer to aggregate unlike with unlike.

fxtentacle 7y ago

BTW, I would very much like to see progress in optical flow because I could really use it for one of my projects. If you know any good paper that tries a novel approach and doesn't just recycle the old SSIM+pyramid loss, please post the title or DOI :)

innagadadavida 7y ago

There is 40-50% drop in precision with state of the art results if test images are ill-formed. The imagenet dataset used is far from ready for real world use cases. A bunch of IBM and MIT researchers are trying to fix this - https://objectnet.dev/ https://objectnet.dev/

Piskvorrr 7y ago
As in, "only works in great visibility on a perfectly spherical road"? That does seem an appropriate summary.

nestorD 7y ago

> I don't know where you got these papers but you clearly got an awful sampling and I don't think you're giving computer vision a fair evaluation. I disagree, I saw such an awful number of bugs in ML codes going with papers that I now take for granted that there is a bug somewhere and hope that it does not impact the concept being evaluated. (here having everyone use python, a language that will try its best to run what you throw at him, feels like a very bad idea)

tsukurimashou 7y ago

> Granted, we're not quite ready for self driving if it was the case self driving cars wouldn't be on the road, I don't think we should aim for perfection, perfection will come. We should be looking for cars that make less errors on average than humans, once you have that you can start putting cars on the road and use data from the fleet to correct remaining errors.

dontreact 7y ago
Humans have an average of 1 fatality per 100 million miles driven. No one is anywhere close to 100 million miles without intervention.
brailsafe 7y ago
Are there any fully autonomous cars on public roads with no driver that can intervene? Seems like only maybe in tightly constrained situations are we ready.
- tsukurimashou 7y ago
  I don't think I mentioned FULLY autonomous cars, my point was: something doesn't have to be perfect before we have to use it, but I probably didn't express myself correctly
  - brailsafe 7y ago
    I think that the necessity of intervening drivers atm indicates that we aren't at that point yet, even if that point is far from perfection, and also that the reason any self-driving cars are on the road is because of the fairly loose but significant requirements from regulation. We might be at that point in otherwise very dangerous situations, like if I was very tired or drunk, but otherwise I don't know that I'd have so much faith in software engineers to completely control my car.