4 ms·

All you coders out there say it with me: "cost function". Really, is that so hard? I see a lot of comments about what is going to happen when robotic cars do

All you coders out there say it with me: "cost function". Really, is that so hard? I see a lot of comments about what is going to happen when robotic cars do this. Well, every roboticist I know, and I know a big bunch of them, is well acquainted with the concept of having a cost-map overlay on the world map. And it is not only OK, but common, to have infinite cost assigned to many nodes -- "Don't go there robot, you will not be a happy robot." So why does the GPS database have poorly assigned cost values on certain road segments? Perhaps because QA de-prioritizes little-traveled roads? But more to the point -- I'll get out of roboticist mode and into seasoned backwoods traveler mode for a moment -- don't go where you are not prepared to go. Look at a paper map, ask a local at the gas station about conditions, and if the road ends up bruising your kidneys, turn back.

jakevoytko 10y ago
This is a problem that we can fix just by throwing data at it. Now that it's possible for any app to know what routes people actually take, we can do even better. It's possible to generate cost functions based on what people actually do. This would fix a lot of different types of problems. First: issues like the ones raised in the article. The killer "roads" have never had drivers successfully complete them, so they should have infinite cost. It would handle construction. If the traffic in a route suddenly falls to zero, or suddenly goes from zero to lots, the cost functions can be updated to reroute. It would handle dangerous neighborhoods. Route 31 was the most direct route between my first apartment and Pennsylvania. I once followed MapQuest directions on this route, and it turns out that it takes you through the part of Trenton that makes it one of the top-20 most dangerous cities in the nation (at the time). Nobody really drives this way, so it could route people via safer roads. The problem here is that the companies that already have good databases are probably going to keep them secret, in case they figure out a way to turn that data into more ad revenue
- tremon 10y ago
  I don't think it's as simple a picture as you paint. The world is not a static place, and there will always be cases where human judgement is required. Consider for example ice roads or mountain passes: thousands of people may ride them on a single day, and the next day the road may be closed off because of frost, thaw or avalanche danger. There is no way you want your car to only consider a different route after ten drivees have lost their life.
  - jakevoytko 10y ago
    Who limited it to one data signal? I sure didn't.
x0x0 10y ago
Or you start going up towards a pass in winter in a vehicle not equipped to do so (awd/4wd, probably backup chains if you're by yourself, water, blankets or winter clothes enough to get you through 24 hours). Particularly with zero other traffic on the road so there's no one to flag down. That the gps took them there sucks, but you need to be responsible for looking at the road and deciding if your vehicle can do it. Particularly in the mountains in winter. And be extra careful when you're approaching the half-way mark on gas, ie you can't get back to where you last saw a human. Going past that point means you're all in. I drive to Tahoe all the time and the number of accidents you see because morons decide to zoom over mountain passes in their crappy civic with bald tires passing people in awd suvs with ski stickers on them (ie people who regularly drive in snow)...
- sandworm101 10y ago
  And I've seen plenty of Tahoes in the ditch after people think that having one will somehow make a difference. 4x4 is great for getting moving, but a soon as you want to stop it's all about the tires and traction control systems.
nkozyra 10y ago
The solutions are not in question. Cost functions, reinforcement learning, etc. are decades old and well understood. There's a big distinction between a roboticist and someone programming GPS route determination. When you build a robot and throw some sensors at it and ask "what was your last position, your last turn, your last accelerometer and your last shift sensor reading," well, you get it. That kind of data is readily available. If you ask a lot of people for that information today, they uncheck the "share with ..." button before continuing. Or even more commonly, there's only one data point. If there's not enough data to do anything intelligent with it, then for practical purposes there's no data. I'm not saying we should share more, but I'm trying to highlight a key difference and one that can make solving this problem more complicated than just throwing a relatively simple and common algorithm at it.
- gugagore 10y ago
  I don't think I understand the rest of your comment, but I don't agree that "RL is well understood".
  - nkozyra 10y ago
    Most of the ML approaches boil down to some well-defined algorithms that are at least 10 years and often 30 years old. Understanding it isn't really the issue, it's testing it for usage and tweaking it for better performance. What exactly is to "understand" about reinforcement learning (application, not theory)? You could say it's developing/emergent, but not that it's "not well understood." The rest of my comment is centered on what the real problem is: a lack of data. That will hurt any and every algorithm. If I have to make a decision/prediction and you give me a single or two pieces of input data, whatever process I employ is likely to return something poor.
    - gugagore 10y ago
      I'm not sure how close you are to working with RL/ML, possibly more than me! However, I don't think you it is useful to talk about the state of the field so broadly. If you say "supervised learning is well understood", I think you have a better chance of being right. The challenges that arise in taking sequential decisions, and learning from sequential observations is a lot harder, because all your data is temporally correlated. Yeah we have algorithms for RL, but actually there are still many theoretical things we cannot prove and in practice do not work well. I am not intimately familiar, but one example is value iteration with a non-linear value function approximation. I still maintain that RL is not well-understood, and I think evidence for that is that we don't see many examples of RL in our phones, cameras, search engines, etc. (which is not the case for supervised learning).
rcthompson 10y ago
I think people are willing to trust the GPS precisely because because they don't understand that the routing is based on a cost function that was ultimately determined by humans with imperfect knowledge about the conditions on every road in the country (not to mention variable conditions that can't be known precisely in advance, like weather and traffic). When you understand that the route is simply the output of an algorithm whose input may or may not be accurate, you're a lot more willing to disregard the route if other information contradicts it than if you believe that the GPS is somehow telling you the "best" route.

jakevoytko 10y ago

This is a problem that we can fix just by throwing data at it. Now that it's possible for any app to know what routes people actually take, we can do even better. It's possible to generate cost functions based on what people actually do. This would fix a lot of different types of problems. First: issues like the ones raised in the article. The killer "roads" have never had drivers successfully complete them, so they should have infinite cost. It would handle construction. If the traffic in a route suddenly falls to zero, or suddenly goes from zero to lots, the cost functions can be updated to reroute. It would handle dangerous neighborhoods. Route 31 was the most direct route between my first apartment and Pennsylvania. I once followed MapQuest directions on this route, and it turns out that it takes you through the part of Trenton that makes it one of the top-20 most dangerous cities in the nation (at the time). Nobody really drives this way, so it could route people via safer roads. The problem here is that the companies that already have good databases are probably going to keep them secret, in case they figure out a way to turn that data into more ad revenue

tremon 10y ago
I don't think it's as simple a picture as you paint. The world is not a static place, and there will always be cases where human judgement is required. Consider for example ice roads or mountain passes: thousands of people may ride them on a single day, and the next day the road may be closed off because of frost, thaw or avalanche danger. There is no way you want your car to only consider a different route after ten drivees have lost their life.
- jakevoytko 10y ago
  Who limited it to one data signal? I sure didn't.

x0x0 10y ago

Or you start going up towards a pass in winter in a vehicle not equipped to do so (awd/4wd, probably backup chains if you're by yourself, water, blankets or winter clothes enough to get you through 24 hours). Particularly with zero other traffic on the road so there's no one to flag down. That the gps took them there sucks, but you need to be responsible for looking at the road and deciding if your vehicle can do it. Particularly in the mountains in winter. And be extra careful when you're approaching the half-way mark on gas, ie you can't get back to where you last saw a human. Going past that point means you're all in. I drive to Tahoe all the time and the number of accidents you see because morons decide to zoom over mountain passes in their crappy civic with bald tires passing people in awd suvs with ski stickers on them (ie people who regularly drive in snow)...

sandworm101 10y ago
And I've seen plenty of Tahoes in the ditch after people think that having one will somehow make a difference. 4x4 is great for getting moving, but a soon as you want to stop it's all about the tires and traction control systems.

nkozyra 10y ago

The solutions are not in question. Cost functions, reinforcement learning, etc. are decades old and well understood. There's a big distinction between a roboticist and someone programming GPS route determination. When you build a robot and throw some sensors at it and ask "what was your last position, your last turn, your last accelerometer and your last shift sensor reading," well, you get it. That kind of data is readily available. If you ask a lot of people for that information today, they uncheck the "share with ..." button before continuing. Or even more commonly, there's only one data point. If there's not enough data to do anything intelligent with it, then for practical purposes there's no data. I'm not saying we should share more, but I'm trying to highlight a key difference and one that can make solving this problem more complicated than just throwing a relatively simple and common algorithm at it.

gugagore 10y ago
I don't think I understand the rest of your comment, but I don't agree that "RL is well understood".
- nkozyra 10y ago
  Most of the ML approaches boil down to some well-defined algorithms that are at least 10 years and often 30 years old. Understanding it isn't really the issue, it's testing it for usage and tweaking it for better performance. What exactly is to "understand" about reinforcement learning (application, not theory)? You could say it's developing/emergent, but not that it's "not well understood." The rest of my comment is centered on what the real problem is: a lack of data. That will hurt any and every algorithm. If I have to make a decision/prediction and you give me a single or two pieces of input data, whatever process I employ is likely to return something poor.
  - gugagore 10y ago
    I'm not sure how close you are to working with RL/ML, possibly more than me! However, I don't think you it is useful to talk about the state of the field so broadly. If you say "supervised learning is well understood", I think you have a better chance of being right. The challenges that arise in taking sequential decisions, and learning from sequential observations is a lot harder, because all your data is temporally correlated. Yeah we have algorithms for RL, but actually there are still many theoretical things we cannot prove and in practice do not work well. I am not intimately familiar, but one example is value iteration with a non-linear value function approximation. I still maintain that RL is not well-understood, and I think evidence for that is that we don't see many examples of RL in our phones, cameras, search engines, etc. (which is not the case for supervised learning).

rcthompson 10y ago

I think people are willing to trust the GPS precisely because because they don't understand that the routing is based on a cost function that was ultimately determined by humans with imperfect knowledge about the conditions on every road in the country (not to mention variable conditions that can't be known precisely in advance, like weather and traffic). When you understand that the route is simply the output of an algorithm whose input may or may not be accurate, you're a lot more willing to disregard the route if other information contradicts it than if you believe that the GPS is somehow telling you the "best" route.