11 ms·

The End of Starsky Robotics

357 points by stefan8r 7y ago

notlukesky 7y ago
I always thought that autonomous driving was more than 10 years off (if not 20) for cars in city traffic. But I thought trucking in the US had a chance for intersate traffic till the proverbial last mile, because there are less edge cases that you need to train the model. Were you off by just 3 years for interstate “driving” till the last mile? Will investors lose their loss memory and a whole new set will invest in the space in say 3 years.
- stefan8r 7y ago
  System worked. Problem was investors have mostly bet on full autonomy, and when that failed to materialized they got scared out of the space. Full autonomy isn't necessary. And I don't know if it's even profitable for trucks.
  - ozborn 7y ago
    Does the current coronavirus pandemic assist you in promoting a teleop model? I'm not sure how much human to human contact there is in the business.
    - cosmodisk 7y ago
      There are some. Initial pickup, all the pit stops for fuel,food,sleep. The end of the journey,where the recipient may check the delivery and etc.
    - stefan8r 7y ago
      Hard to say. Truck drivers will be hit hard by this - they can't really stop interacting with people, almost never get paid time off, are older and in poor health, and are paycheck to paycheck. My bet is that nearly all of them get COVID, with a higher hospitalization / mortality rate. Could knock 30% of capacity off the road in the next 2 quarters, which would mean food shortages.
  - redis_mlc 7y ago
    > investors have mostly bet on full autonomy lol. I used to live on one of the side streets in MV that Google's AV cars trained on. Saw three in a row once. They were usually the only traffic, so that annual report of millions of miles travelled was meaningless - they might as well have driven arounf the Safeway parking lot at 5 am. I guess passenger-carrying quadcopters is next. Oh wait ...
    - stefan8r 7y ago
      I live next to the weird intersection at Duboce Park. I see a Cruise car go by it every 30m. A better solution for robotaxis - just avoid the weird intersections.
cyanoacry 7y ago
Thank you for the write-up! It's a little depressing (but not entirely surprising) to hear that you couldn't get folks excited about safety -- I work in the aerospace field, and our day-to-day is all about risk management: how, why, when. It's frustrating that high-reliability systems aren't seen as exciting when really they're what makes the magic run. Best of luck, and I'm looking forward to your next venture!
- smabie 7y ago
  Ironically, in my experience, finance is the field that probably cares most about risk management. That and maybe healthcare (though less than you would think, surprisingly).
- ajeet_dhaliwal 7y ago
  It’s the same thing when it comes to quality. A lot of lip service paid but no one sees it as anything exciting.
rsp1984 7y ago
There are too many problems with the AV industry to detail here: the professorial pace at which most teams work, the lack of tangible deployment milestones, the open secret that there isn’t a robotaxi business model, etc. Just curious, why would there not be a business model for robotaxis?
- Animats 7y ago
  Uber: the drivers own the cars, the drivers maintain the cars, the drivers clean the cars, the drivers store the cars when not in use. Driverless: the company needs an operation, real estate, and staff comparable to a big auto rental company to do all that. Plus the engineering and technical staff required for autonomy. Even with startups doing autonomous shuttle buses, which works at low speed, nobody is making money in that space. It's all demos.
  - edshiro 7y ago
    Add to that the fact that you are never going to get 100% fleet utilisation, will need to pay for qualified tele-operators, maintain/repair vehicle along with sensors... I recently listened to a podcast episode of the Autonocast[1], where they interviewed a Harvard Researcher who claimed the economics of Robotaxis just don't work. Very interesting listen. [1] http://www.autonocast.com/blog/2020/3/11/177-ashley-nunes-on-robotaxi-economics http://www.autonocast.com/blog/2020/3/11/177-ashley-nunes-on...
    - stefan8r 7y ago
      All transportation businesses are utilization businesses. Empty hours/miles are never recovered. Early robotaxis will have tight geofences. Makes them particularly bad competitors to poorly paid people who are willing to drive wherever.
  - cosmodisk 7y ago
    I think it will probably work at some point in the future( 20-40 years). Today the reality is that the jobs we reward the least are the ones that turn out to be the most difficult to automate. Someone with 10 min training can do better job with a thread and a needle than the most advanced robot painfully trying to stich two pieces of garmet together.If these can be overcome,then it may work.
m0zg 7y ago
I wonder what Elon is going to do when _his_ "full self driving" fails to materialize. Which it will. Not only you can't do this with just cameras and radar, I doubt you can do it _period_ without modifying the roads specifically for such cars, and segregating them from human drivers. And even then it will be difficult psychologically and legally to convince the public that this is "better" than a (possibly inebriated) human, for reasons that have been discussed to death already.
- MegaButts 7y ago
  Tesla makes their money selling cars, and Autopilot sales are non-refundable. They get paid even if they don't deliver, so I'm not sure how much they care other than keeping the hype alive.
  - m0zg 7y ago
    That's all well and good until they get sued by people who paid $7.5K each for FSD and never got it.
    - marcinzm 7y ago
      Then worst case they'll get a class action lawsuit, appeal the verdict three times and then end up giving everyone $100 worth of credits towards a future Tesla. Lawyers will net a couple hundred million.
      - csours 7y ago
        I hesitate to comment, as I work for a competitor, but I imagine that the reputational harm may be even worse than the monetary.
        marcinzm 7y ago
        I would have thought so before but we've had multiple people killed in auto-pilot related accidents already and people still buy. Tesla will spin the news and results and probably get a settlement statement that makes them look not so bad. Especially if everyone is in the same boat and failing to get the technology working well enough. Then Tesla will simply boast at how much better they are than competitors.
DanFeldman 7y ago
Ah well, it was great while it lasted. Most of the engineering team has spread throughout the AV industry, with most folks going to our neighbors and fellow YC company Cruise Automation. Some are at Waymo, Zoox, AutoX, and some purposefully exited the AV space entirely. I joined Applied Intuition to help build out Simulation and Infra for other companies producing AVs/robotics. There are a few folks I know of who are still looking for their next roles in the BizOps/PplOps side, which has been especially hard during COVID-19 season if anyone wants to do some linkedin stalking.
- xiaolingxiao 7y ago
  Hey Dan, could I ask you a few questions about what you learned at Starsky? I have a robotics background and what they tried to do is very very interesting.
  - krak12 7y ago
    Please Dan, ignore this idiot trying to steal anything from your hard work to raise funds in China
  - stefan8r 7y ago
    Also happy to answer questions (my username might sound similar to the name of the author ;) )
    - kondrich 7y ago
      If you don't mind me hijacking this, you mentioned this in the article: Nevertheless, we found an incredible amount of industry and investor resistance to our teleop-dependent approach. Why was this the case? I can't see the harm in using teleop.
      - stefan8r 7y ago
        TBH - In the early days the "cool AV people" seemed to act as if teleop was an admission of bad engineering (your team isn't good enough to perfect L5). If everyone else is right on the cusp, and you're not trying, you must be some sort of loser. There are still a number of folks who think that teleop can never be safe - surprisingly sophisticated people hold onto that dogma. We wrote a whitepaper about it for investors, and I'll share that at some point as I open up more and more of the Starsky vault.
ssivark 7y ago
> There are too many problems with the AV industry to detail here [...] The biggest, however, is that supervised machine learning doesn’t live up to the hype [...] It’s widely understood that the hardest part of building AI is how it deals with situations that happen uncommonly, i.e. edge cases. In fact, the better your model, the harder it is to find robust data sets of novel edge cases. Additionally, the better your model, the more accurate the data you need to improve it. Rather than seeing exponential improvements in the quality of AI performance (a la Moore’s Law), we’re instead seeing exponential increases in the cost to improve AI systems This is exactly the problem with data hungry machine learning approaches, specifically deep learning (and that’s without even mentioning the compute resources necessary to learn). The only way to circumvent that is plausibly apply better inductive biases, and fundamentally rethink what the field considers important.
- Barrin92 7y ago
  I think the obvious problem is that induction (which is what learning from data is), is simply only one tool in the huge space that is intelligence, and it will never be enough to emulate the skill of a human driver, which is more or less what is necessary for autonomy in an open environment.
  - ibeckermayer 7y ago
    The "induction" that machine learning algorithms do also isn't the same as the induction that humans preform. We induce new concepts from experience (data) -- The description itself assumes consciousness in both "concepts" and "experience". Thinking of computers as getting more "intelligent" like humans is a category error -- computers are dumb matter configured in an intelligent way by actual intelligence (humans) to preform certain tasks for us. We get better at telling them how to preform certain tasks (software), but there's no reason to think we're moving along some continuum of intelligence towards us.
    - visarga 7y ago
      > We induce new concepts from experience (data) That's the premise of deep learning - inducing high level concepts from experience, without manual feature engineering. DL models are good at induction, what they can't handle is generalisation (being accurate outside of distribution). And self driving has a long tail, that's why it's so hard.
alricb 7y ago
It took me a while to figure out that "AV" means autonomous vehicle... I was kind of wondering what AV as in AV club had to do with trucking.
Adams472 7y ago
Thank you for sharing this. The insights and details you share here will help many future founders. I'm sorry things didn't end up in the exact way your team may have hoped. I hope you can take pride in everything you accomplished. I wish you all the best!
mrfusion 7y ago
Why not an aquihire? Surely lots of companies would want to augment their av teams?
- savrajsingh 7y ago
  I’m sure it was explored but it didn’t close. Stuff like this is annoying for a founder to hear because he’s like “aw yeah, why didn’t I think of that earlier?!” Of course he thought of it — but it just didn’t work out that way.
mrfusion 7y ago
Welcome to the trough of disillusionment. (sp) I think the plateau of productivity will be really awesome computerized copilots and safety assists/warnings. The human computer driving team could Be quite a combo of designed right.
- mrfusion 7y ago
  Perhaps for true self driving we’ll need one more breakthrough in AI. Perhaps a system that can identify its own edge cases and request additional training or seek out the training itself.
  - seibelj 7y ago
    We need AGI or we need to ban humans from driving, pedestrians, bikers, children, deer, and anything that can disrupt a perfect computer model.
- stefan8r 7y ago
  The problem is that most of those businesses don't make sense with the current cost of talent. The L2 systems OEMs are rolling out are primarily made by engineers in low-cost areas at $60-90k/yr, not $130-250k/yr in SV. Realistically, most of the AV talent will go to industries that can afford them. Salary isn't super elastic.
  - mrfusion 7y ago
    The engineers in the low cost of living areas do the real work. We don’t need rockstars.
- mark-r 7y ago
  It's well known that if the computer is handling a significant part of the task, the human will get distracted or zone out completely. The "human computer driving team" will end up being less than the sum of its parts. Airline pilots already have well established procedures to prevent this from happening when they engage the auto-pilot.
xiaolingxiao 7y ago
This is a very good and honest retrospective. He shows clear thinking, and surprising technical understanding for someone who is not technical. And most importantly he shows humility. A+
cjv 7y ago
This is sad news. I worked at Starsky Robotics as a perception team intern after graduating high school. I will always be grateful for the team for the opportunity, it was a fantastic first job and everyone who worked there was very kind (especially Stefan). Unfortunately, Starsky had effectively had no machine learning in 2017 (when I worked there), using solely classical computer vision techniques. This didn't match the company's ambitions of not using LIDAR and there was a strong stigma against switching to a deep learning approach. At the time, very few object detection models had public implementations and I spent a lot of time trying to get a YOLO9000 and RetinaNet implementations running at real-time speeds. Frustrating, as a small startup the labeling services kept screwing us over by returning poorly annotated images. I think what I took away from the experience is that deep learning in domains with long tails requires a enormous investment in a labeling pipeline - dwarfing the computational aspect - to get decent results. I don't think any solutions are on the horizon that will allow us to bypass this reality. You don't see improvements between Comma.ai and Tesla because it's about the improvements far out on the tail.
- lioeters 7y ago
  For others like me who were as amused by the name, YOLO9000, it's a real thing, "a state-of-the-art, real-time object detection system that can detect over 9000 object categories". YOLO9000: Better, Faster, Stronger - https://arxiv.org/abs/1612.08242 https://arxiv.org/abs/1612.08242 (2016)
  - nstj 7y ago
    YOLO = “You Only Look Once”
- jiayq84 7y ago
  Just to clarify a little bit... "At the time, very few object detection models had public implementations" - this is wrong. Almost all object detection models had public implementations starting from 2014, most notably Detectron (Caffe), GoogleNet/SSD (Tensorflow and matlab). Post 2015 when TensorFlow was released, one can find even more implementations. Data is the problem. Everyone has the algorithm but not enough people have data (especially labeled ones)
csours 7y ago
Disclaimer up front: I work for General Motors. I don't work on AV. Any opinions are my own. I have no special knowledge of GM's AV strategy. > It didn’t matter that that jump from “sometimes working” to statistically reliable was 10–1000x more work. There's 2 states of functionality: 1) It doesn't work 2) It sometimes works The inverse, for disk drives: Failing and Failed. Think about apps/services. You could say that your app is working, but over a long enough time period, it is only sometimes working. It's working while you have disk space, free memory, and a working network connection. It's working while your business assumptions hold true. It's working while your datacenter has power. We've developed strategies for managing all of these things; for load balancing and Active/Active hosting. But even with that, it only sometimes works. --- With all that, I think it may be useful to think of self driving in terms of tasks. If you can put a box around what you expect a computer to be able to do, you can define tasks that will always return a reasonable output. The more tasks a computer handles, the easier it is for the human in control (think driver assist, like lane keeping and automatic cruise control). If you add enough tasks, and perform them well enough, maybe you can take the human in control out of the vehicle. I think I'm in agreement with the authors that I don't see the day when there isn't a human in control. Or the other way to say that is that if there isn't a human in control, sometimes your AV will just stop.
- stefan8r 7y ago
  Just stopping is ok is ok in some use cases, but not others. It will be a long time before robotaxis "just stop" less frequently than uberdrivers, but AV trucks can already be more reliable than contemporary drivers.
d_burfoot 7y ago
> The biggest, however, is that supervised machine learning doesn’t live up to the hype. This is the key point. The new DNN approaches can outperform the classical techniques, but only when they can exploit vast amounts of training data. The dramatic successes of Deep Learning all depend on either unsupervised learning against enormous raw datasets (BERT, GPT-2, word2vec, etc) or games, where you can generate unlimited quantities of labelled data by playing the game against your own agent (AlphaGo, AlphaStar, OpenAI Five, etc).
- cosmodisk 7y ago
  As someone already mentioned here,I think it's probably not a goood approach. The way we store info in dstabases is very limited compated to what we can do in our heads. For instance we know a concept of a table.It can be made of almost any material,can have whatever shape,size, and colour we want and yet we can instantly recognise it without having some concrete data points on what it should look like. I can make a glass cube,put it in a middle of the room and people would know it's a table.How the hell we operate this way,I have no idea.
  - wolco 7y ago
    On a basic level we all: - Know what a room looks like. - Expect certain objects to exist at certain places and not expect others - The brain indexes these - Seeing one or two datapoints allows us to guess. So seeing a glass cube in that context gives us few logical chpices (it must be a table) similar to someone typing a letter in a textbox and having it filter a pre-defined list.
edshiro 7y ago
Thank you for sharing this very candid article on the Starsky Robotics and generally the autonomous vehicle space. It's a real eye opener. I've been following your progress for the last few years (and I also read about your company through Reilly Brennan's "Trucks - FOT" newsletter). I am sorry you could not get investors to believe more in what you and your team, especially as you required a lot less funds than many other companies in this arena. I also thought you had a clear business case (I worked in ride-hailing and also logistics so understand some of the problems in this space). I wanted to ask you a question: I am building a startup in the dash cam video analysis space. We are building a large and geographically diverse dataset of road videos, where our users can annotate/label the data. We then are going to look at detecting specific events like accidents and edge cases on videos. Do you feel this type of business, the data we collect, and insights we generate would have value for a AV startup? All the best in your next move. Stay strong - you can be proud of what you and your team did.
- stefan8r 7y ago
  Hard to say. We had really strict rules on what we wanted our data to look like, and were very specific about subject matter. We probably wouldn't have been able to be a customer. Our intention to deal with accidents/edge cases was that if anything looked outside of ODD (as in, not perfect driving conditions) we'd execute a MRC (pull to side of the road, or stop at next exit). Relatively easy way to solve most of the really hard edge cases.
  - edshiro 7y ago
    Understood and makes total sense given the ODD at Starsky.
petermcneeley 7y ago
All this L1/L2/L3 when most of the public failures in AV are at the level of vision and interpretation of the world.
- krisoft 7y ago
  So? That graph is about the capability of the system and how it improves with effort. This has nothing to do with which part of the system fails the most. Where do you see the contradiction?
  - karlding 7y ago
    The choice of naming here is somewhat unfortunate, and probably contributes to some confusion. Level 0-5 is terminology used by SAE to describe the range of fully manual (Level 0) to fully autonomous (Level 5). One would expect that in an article about Autonomous Vehicles, L{0,1,2,3,4,5} would describe the level of autonomy that the vehicle provides, but the article instead uses these to describe arbitrary thresholds of a graph demonstrating the capability of an AI.
    - stefan8r 7y ago
      Fair point. I'm shitty at drawing mathematical graphs. I'll update at some point with different notation.
      - ngcc_hk 7y ago
        I think it makes my day as it is a rare but simple insight into the s curve and the peak line. It will be a useful mental tool for life. Not sure why l meant anything in some other model would affect its reading. Good to me.
jedberg 7y ago
> Instead, the consensus has become that we’re at least 10 years away from self-driving cars. I'm going to assume the founder of a self-driving truck company knows what he's talking about. But at the same time, I have a hard time reconciling that with the fact that I sat in a car that drove itself all around San Francisco, dealing with a ton of edge cases. Maybe we won't get to a 100% drive-anywhere-you-want-car in 10 years, but to be fair, a lot of humans aren't capable of driving a car anywhere either. There are a lot of LA drivers who can't drive in snow, for example. I was one of them, until I got practice, and even then, I'm not that safe at it. I think as long as we set the bar at "drive anywhere a car can go with 100% safety" we will never reach that bar. But if the bar is at "drive as well as a human in most of the places humans drive", well, I've already seen a car that can do that.
- stefan8r 7y ago
  What's hard about this comparison is knowing exactly what you saw. If the safety driver disengaged 1-3x, it might have felt like a robotaxi drove you all over; but the effort to get that system safe for unmanned regular service might take another decade. If it was super mapped and following a fixed route with object avoidance; the difference between a car that can go on an HD-map track and one that can go point-to-point in a city is also maybe a decade. We humans have limited empathy for what is really really really hard for computers. Many know that's true for an app, but somehow ignore that knowledge when it comes to autonomy.
  - greglindahl 7y ago
    Even if jedberg observed 0 disengagements, it's still not the millions of miles of data you'd want to prove something.
pj_mukh 7y ago
Great read! And thanks to the author for all the candor. The business case always seemed clear to me and now reading this, I wonder if there is a case to be made for an engineer and a trucking operations veteran to build a business that requires minimal capital (or maybe even is bootstrapped?!), to take it across the finish line?
- stefan8r 7y ago
  There are a couple of people working on that, notably CloudTrucks. It's not for the faint of heart - truck drivers and engineers follow a very different set of social norms. It's hard to be the hardass boss drivers don't want to fuck with while being the cuddly CEO engineers like the culture of. That might be a future blogpost.
  - ngcc_hk 7y ago
    Waiting for it. Sad but good read.
chubot 7y ago
Thank you for the great writeup. It sounds like you took a responsible, valuable, and even economically viable approach, but the market doesn't want to hear it. I wonder when the market will start listening to people who actually WORK on the problem. > which means that no one should be betting a business on safe AI decision makers. The current companies who are will continue to drain momentum over the next two years Sounds about right. ----- Here's me over 2 years ago simply quoting people who worked close to the problem: https://news.ycombinator.com/item?id=16353541 https://news.ycombinator.com/item?id=16353541 Here's my negative scenario: self-driving is "AI-complete"; you can't really hit all the edge cases without solving AI in general, which is more than 30 years away (Kurzweil is the wild optimist and predicts 2045). You CAN use self-driving in limited circumstances, but those limitations are precisely the ones that make driving yourself around more attractive. The expense doesn't go down as quickly as anticipated because of this. They are a niche technology for DECADES. This looks about right, and I'm not claiming to be prescient, just basically saying what Chris Urmson and Bill Gurley already said years ago. It's weird to me that there's still so much money chasing this pipe dream. It was a sign when Google spun out Waymo. If they really believed in the product, it would be called something like "Google Self-Driving Cars", not "Waymo". ---- edit: it's easy to be negative, so on the positive (and contrarian) side, I believe rideshare is undervalued, and has a long way to go. I bought both Uber and Lyft stock at $16 yesterday (they may go down again, but ping me in 2 years). This comment in this thread is a nice take and makes me even more optimistic: https://news.ycombinator.com/item?id=22631151 https://news.ycombinator.com/item?id=22631151
- chrisco255 7y ago
  I still think that AI shuttles and buses driving predetermined routes and schedules would be tremendously useful. Especially if they could operate continuously 24/7. And that should be possible, with maybe tele-operators dialing in to override the edge cases or sending dispatch crews to takeover for any issues that come up.
  - stefan8r 7y ago
    This is true, but essentially every transit system in the world (minus Hong Kong and Singapore's) operates at a loss. Maybe you can save money vis a vis unionized drivers, but I'm not wildly optimistic.
ChuckMcM 7y ago
Such an excellent and cogent discussion of the challenges of making an autonomous vehicle startup work. I really resonate with the challenge they had making safety "sexy" from a coverage point of view. It seems it shares that attribute with "security" which, when done well, means nothing happens. I had a lot of hope that we would get four lane (two each way) "freight ways"[1]. These would be paved roads designed specifically for autonomous trucks to move freight from one point to another. With sufficient freight ways between key cities on the coast and further inland, and efficient way to move goods with somewhat worse costs than rail and quite a bit better in terms of reaction time. [1] Like a free way but designed specifically to exclusively service autonomous trucks carrying standard containers.
- Arbalest 7y ago
  This begs the question, how much does it cost to lay road than to lay rail? If it's going to be exclusively autonomous, the more narrow confines of the rail should be well suited.
  - stefan8r 7y ago
    Another common fallacy. The US actually has a better rail network than Europe - most people just think it's worse because passenger rail here sucks. That's the case because we prioritize freight trains over passenger (in America Business is #1). To paraphrase Churchill, trucking is the worst form of transportation, besides all the others. If you can put freight on a train, you do. It's slower, point-to-point, and less good a less-than-trainload freight than trucks are.
- stefan8r 7y ago
  Many many smart people share this thought. I don't. Freeway costs roughly $4m/lane/mile. Building a duplicate system for trucking would make UBI look cheap. Truck taxes also pay for much of current hway infrastructure, so you'd need to replace that $
sitkack 7y ago
The VSSA (Voluntary Safety Self-Assessment) referenced in the post [1] https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1be1791a7332594e0ff28b_Voluntary%20Safety%20Self-Assessment_Starsky%20Robotics.pdf https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1...
malandrew 7y ago
> In 2019, our truck became the first fully-unmanned truck to drive on a live highway. Uber drove an autonomous truck on a live highway to deliver beer for Anheuser-Busch back in October 2016 https://fortune.com/2016/10/25/uber-anheuser-busch-first-autonomous-truck-beer-delivery/ https://fortune.com/2016/10/25/uber-anheuser-busch-first-aut...
- stefan8r 7y ago
  There was a person in the vehicle. Amusingly - many VCs who "should know better" fell for the same error. I'd regularly get in arguments with investors about whether others had done unmanned or not.
  - malandrew 7y ago
    In the sleeper cabin monitoring the vehicle. The main cabin was unoccupied and there wasn't a single intervention for the entire trip. I knew people involved with that trip at the time.
    - stefan8r 7y ago
      The difference between "in the sleeper cab" and "outside the truck" are huge. We first started doing in the sleeper cab in Jul'17, took us 2 years of dedication to get confidence to remove from vehicle. To take the person out of the vehicle, the system itself needs to be able to self-monitor and pull itself over if it stops working. That's the big lift.
      - malandrew 7y ago
        That's it. The team at Uber did that like a year or so later but with no public fanfare because it wasn't considered noteworthy. That still puts it at ~2 years prior to this accomplishment.
Havoc 7y ago
This to me very much assumes no leap forward. See the start of deep learning - sudden rapid progress. I don’t see why there can’t be another jump like that. Much like the chip manufacturers mentioned in the post. No idea what that advance may be but it’s out there I think
- larrydag 7y ago
  Perhaps there is no leap forward in long haul trucking. Yet there are many supervised machine learning products in the market and doing well (Tesla autopilot, iRobot vacuums, etc). I bet the founder can find a niche that could be marketable and can be mass produced or scaled.
Quarrelsome 7y ago
If we'd sold AV as "grandma driving" instead of "autopilot" do you think we could have avoided some of this mess? The public are only ever going to accept perfection which prevents us from getting our hoverboards here.
keenmaster 7y ago
Sad news. Apart from improving upon ML, can more sensors and physical modeling get us past the AV plateau? An example of an additional sensor would be an infared camera. I don't know how well infared cameras work during the day or when they are occluded, but they seem like they can provide an additional datapoint to classify an object as "person" for example. That way, it doesn't matter if they're wearing a paper bag Halloween costume, the car still knows what's underneath. Physical modeling seems like it would help with prediction. If a deer is idle on the side of road ahead, a car with a physical "deer" model would account for the probability that the deer will suddenly dart across the road and the speed at which it can possibly do so. It will slow down and watch for certain behaviors, such as the deer standing up from a prone position, and modify its probabilities accordingly. That is, after all, what we humans do. The ML wouldn't need to perfectly model all animals. Some animals would be put into a "mammal" class or more vague "animal" class, paying special attention to size and stature. Additionally, depending on how long it might take to develop truly autonomous self-driving cars, we might want to develop self-driving car friendly infrastructure. I'm talking about AV-only lanes with safeguards in place to minimize variability, vehicle-to-vehicle communication, infrastructure-to-vehicle communication, and better than average maintenance. It seems like it might be worth it, given the gains to be had. Here's an underappreciated angle to the AV-friendly infrastructure: it's a hedge against incorrect predictions. What if truly autonomous vehicles are actually 15 years away? In that case, if an "AV heaven" is developed in a corridor somewhere in Los Angeles 5 years from now, that'll be the only place where the AV dream is realized for the ensuing 10 years. The profit from "AV heaven" can be reinvested into AV development. It can also be used to expand into other areas. Moreover, AV heaven might galvanize the government, public, academia, and investors to eliminate any bottlenecks in AV development. There's no immutable stone tablet, only viewable by God, that says "AV development must take X years." We can modify the timeline to some extent. Lastly, AV heaven would allow us to observe the true value of self driving cars. What if they're better than we ever imagined? What if they radically change human behavior? What if they increase development in surrounding areas by a lot? What if they change the nature of work and where people live by more than we even expected? What if self-driving trucks + last mile bot delivery radically push down prices on everything and enable previously impractical business models or behavior? We actually know many of the changes that AVs will bring, but, critically, we don't know the magnitude of those changes.
ackbar03 7y ago
Not sure about him blaming the "professorly pace" of research though. That's the rate good research is done at. If you were expecting anything faster than you don't really understand what it was going to take in the first place
machinelearning 7y ago
For any engineers reading this who want to solve the computer vision/machine learning problems explained in the blog post - reach out at info@nuronlabs.com.
smabie 7y ago
I'm not sure it's intellectually honest to put all the blame on stupid investors, as a founder it's your job to deliver what the market and the investors want, or convince them otherwise. And if in fact investors and the market is just stupid, that's actually a great sign: you were just early and have a shot at doing it again in the future.
johntiger1 7y ago
Just this hiring season, I still saw positions open...and now this
punnerud 7y ago
Why does every self-driving car company focus on driving like we do today? Seems more viable to make a system where the trucks dock at speed on the highway. This is where the power of AI come in to play by coordinating the two trucks next to each other, switching load and continue driving without ever have to stop. The end goal would be to also move people, but we have to start with something safer that makes economical sense.
DrNuke 7y ago
It is pretty cynical and hindsight is always 20/20, but... is self-driving automotive just trains and metros? There is a fundamental business case flaw in these startups, because regulators worldwide will never approve them go together with human traffic, which is messy, chaotic, smart in its own sense at local level (you need to know the uses of local people, not the general rules... eg. driving a car in Naples is very different from driving a car in Milan). Only dedicated lanes or tunnels then, trains!
blackrock 7y ago
Maybe it’s time they got bought out by a larger company, and go dark. Continue their research in private, until they build the perfect robot.
dwnvoted2hell 7y ago
It all seems rather tragic, especially when the solution to preventing the spread of disease seems to be largely in robotics.
baylessj 7y ago
Thank you for the excellent write-up here. I'm sad to hear that Starsky is dissolving, but I have hope that the lessons learned here will be applicable elsewhere. The point about safety being a low priority with investors makes sense but is unfortunate. I hoped that Google's investment in Waymo would push past this hurdle, but they're latest funding round makes it clear that they also have to deal with investor wants over the success of the technology. Really hoping to see someone get the funding to make this tech work reliably in the future and apply the same safety-first design that Starsky used.
putroce 7y ago
I’ve been developing software for self driving (both road and offroad) the past two years and what I’ve noticed is that there isn’t much innovation in the field. It seems like everyone follows the exact same design paradigm which is using something like Autoware. It’s pretty disheartening to see how much money has been poured into this field and what do we have out of it now? GM Supercruise came out in like 2013 [0] with similar performance to the level of autonomy we’re at in 2020. To clarify, this was in a production cadillac in 2013, so anyone who knows about automotive products knows how long this must have been in development before they were allowed to put it in production. To the author, it’s interesting to read how quick you rose and fell, but it’s good you got out now instead of holding on for years and years. What do you think is next? I’m honestly tired of the lack of progress in self driving, and was wondering what anyone’s thoughts were on home robotics. I feel like that’s the next big thing that VC’s will pour billions into. [0]https://media.gm.com/media/us/en/gm/news.detail.html/content/Pages/news/us/en/2013/Apr/0429-cadillac-super-cruise.html https://media.gm.com/media/us/en/gm/news.detail.html/content...
tnorthcutt 7y ago
> Around November 12 of 2019, our $20m Series B fell apart. We furloughed most of the team on the 15th (probably the worst day of my life), and then started work on selling the company and making sure the team didn’t go without shelter (or visa status, or healthcare for the new and expectant parents). We were able to land many of the vulnerable jobs by the end of January and I’m in the process of selling the assets of the company (which includes a number of patents essential to operating unmanned vehicles). Like the captain of a sinking ship, I’ve gotten most of the crew on lifeboats and am now noticing the icy waters at my ankles while I start to think about what I do next. Well done.
mleonhard 7y ago
Stefan & team: I'm sorry your company failed. I hope you can and will take time now to rest and recover. Things that help recovery from burnout: long walks, therapy, exercise, sleep, meditation, and spending time with people you love.
anandkulkarni 7y ago
Thanks for sharing, Stefan. Lots to be proud of in what Starsky achieved, and factors outside your control at work at the end. Excited to see what you do next.

357 points by stefan8r 7y ago

I always thought that autonomous driving was more than 10 years off (if not 20) for cars in city traffic. But I thought trucking in the US had a chance for intersate traffic till the proverbial last mile, because there are less edge cases that you need to train the model. Were you off by just 3 years for interstate “driving” till the last mile? Will investors lose their loss memory and a whole new set will invest in the space in say 3 years.

stefan8r 7y ago
System worked. Problem was investors have mostly bet on full autonomy, and when that failed to materialized they got scared out of the space. Full autonomy isn't necessary. And I don't know if it's even profitable for trucks.
- ozborn 7y ago
  Does the current coronavirus pandemic assist you in promoting a teleop model? I'm not sure how much human to human contact there is in the business.
  - cosmodisk 7y ago
    There are some. Initial pickup, all the pit stops for fuel,food,sleep. The end of the journey,where the recipient may check the delivery and etc.
  - stefan8r 7y ago
    Hard to say. Truck drivers will be hit hard by this - they can't really stop interacting with people, almost never get paid time off, are older and in poor health, and are paycheck to paycheck. My bet is that nearly all of them get COVID, with a higher hospitalization / mortality rate. Could knock 30% of capacity off the road in the next 2 quarters, which would mean food shortages.
- redis_mlc 7y ago
  > investors have mostly bet on full autonomy lol. I used to live on one of the side streets in MV that Google's AV cars trained on. Saw three in a row once. They were usually the only traffic, so that annual report of millions of miles travelled was meaningless - they might as well have driven arounf the Safeway parking lot at 5 am. I guess passenger-carrying quadcopters is next. Oh wait ...
  - stefan8r 7y ago
    I live next to the weird intersection at Duboce Park. I see a Cruise car go by it every 30m. A better solution for robotaxis - just avoid the weird intersections.

cyanoacry 7y ago

Thank you for the write-up! It's a little depressing (but not entirely surprising) to hear that you couldn't get folks excited about safety -- I work in the aerospace field, and our day-to-day is all about risk management: how, why, when. It's frustrating that high-reliability systems aren't seen as exciting when really they're what makes the magic run. Best of luck, and I'm looking forward to your next venture!

smabie 7y ago
Ironically, in my experience, finance is the field that probably cares most about risk management. That and maybe healthcare (though less than you would think, surprisingly).
ajeet_dhaliwal 7y ago
It’s the same thing when it comes to quality. A lot of lip service paid but no one sees it as anything exciting.

rsp1984 7y ago

There are too many problems with the AV industry to detail here: the professorial pace at which most teams work, the lack of tangible deployment milestones, the open secret that there isn’t a robotaxi business model, etc. Just curious, why would there not be a business model for robotaxis?

Animats 7y ago
Uber: the drivers own the cars, the drivers maintain the cars, the drivers clean the cars, the drivers store the cars when not in use. Driverless: the company needs an operation, real estate, and staff comparable to a big auto rental company to do all that. Plus the engineering and technical staff required for autonomy. Even with startups doing autonomous shuttle buses, which works at low speed, nobody is making money in that space. It's all demos.
- edshiro 7y ago
  Add to that the fact that you are never going to get 100% fleet utilisation, will need to pay for qualified tele-operators, maintain/repair vehicle along with sensors... I recently listened to a podcast episode of the Autonocast[1], where they interviewed a Harvard Researcher who claimed the economics of Robotaxis just don't work. Very interesting listen. [1] http://www.autonocast.com/blog/2020/3/11/177-ashley-nunes-on-robotaxi-economics http://www.autonocast.com/blog/2020/3/11/177-ashley-nunes-on...
  - stefan8r 7y ago
    All transportation businesses are utilization businesses. Empty hours/miles are never recovered. Early robotaxis will have tight geofences. Makes them particularly bad competitors to poorly paid people who are willing to drive wherever.
- cosmodisk 7y ago
  I think it will probably work at some point in the future( 20-40 years). Today the reality is that the jobs we reward the least are the ones that turn out to be the most difficult to automate. Someone with 10 min training can do better job with a thread and a needle than the most advanced robot painfully trying to stich two pieces of garmet together.If these can be overcome,then it may work.

m0zg 7y ago

I wonder what Elon is going to do when _his_ "full self driving" fails to materialize. Which it will. Not only you can't do this with just cameras and radar, I doubt you can do it _period_ without modifying the roads specifically for such cars, and segregating them from human drivers. And even then it will be difficult psychologically and legally to convince the public that this is "better" than a (possibly inebriated) human, for reasons that have been discussed to death already.

MegaButts 7y ago
Tesla makes their money selling cars, and Autopilot sales are non-refundable. They get paid even if they don't deliver, so I'm not sure how much they care other than keeping the hype alive.
- m0zg 7y ago
  That's all well and good until they get sued by people who paid $7.5K each for FSD and never got it.
  - marcinzm 7y ago
    Then worst case they'll get a class action lawsuit, appeal the verdict three times and then end up giving everyone $100 worth of credits towards a future Tesla. Lawyers will net a couple hundred million.
    - csours 7y ago
      I hesitate to comment, as I work for a competitor, but I imagine that the reputational harm may be even worse than the monetary.
      - marcinzm 7y ago
        I would have thought so before but we've had multiple people killed in auto-pilot related accidents already and people still buy. Tesla will spin the news and results and probably get a settlement statement that makes them look not so bad. Especially if everyone is in the same boat and failing to get the technology working well enough. Then Tesla will simply boast at how much better they are than competitors.

DanFeldman 7y ago

Ah well, it was great while it lasted. Most of the engineering team has spread throughout the AV industry, with most folks going to our neighbors and fellow YC company Cruise Automation. Some are at Waymo, Zoox, AutoX, and some purposefully exited the AV space entirely. I joined Applied Intuition to help build out Simulation and Infra for other companies producing AVs/robotics. There are a few folks I know of who are still looking for their next roles in the BizOps/PplOps side, which has been especially hard during COVID-19 season if anyone wants to do some linkedin stalking.

xiaolingxiao 7y ago
Hey Dan, could I ask you a few questions about what you learned at Starsky? I have a robotics background and what they tried to do is very very interesting.
- krak12 7y ago
  Please Dan, ignore this idiot trying to steal anything from your hard work to raise funds in China
- stefan8r 7y ago
  Also happy to answer questions (my username might sound similar to the name of the author ;) )
  - kondrich 7y ago
    If you don't mind me hijacking this, you mentioned this in the article: Nevertheless, we found an incredible amount of industry and investor resistance to our teleop-dependent approach. Why was this the case? I can't see the harm in using teleop.
    - stefan8r 7y ago
      TBH - In the early days the "cool AV people" seemed to act as if teleop was an admission of bad engineering (your team isn't good enough to perfect L5). If everyone else is right on the cusp, and you're not trying, you must be some sort of loser. There are still a number of folks who think that teleop can never be safe - surprisingly sophisticated people hold onto that dogma. We wrote a whitepaper about it for investors, and I'll share that at some point as I open up more and more of the Starsky vault.

ssivark 7y ago

> There are too many problems with the AV industry to detail here [...] The biggest, however, is that supervised machine learning doesn’t live up to the hype [...] It’s widely understood that the hardest part of building AI is how it deals with situations that happen uncommonly, i.e. edge cases. In fact, the better your model, the harder it is to find robust data sets of novel edge cases. Additionally, the better your model, the more accurate the data you need to improve it. Rather than seeing exponential improvements in the quality of AI performance (a la Moore’s Law), we’re instead seeing exponential increases in the cost to improve AI systems This is exactly the problem with data hungry machine learning approaches, specifically deep learning (and that’s without even mentioning the compute resources necessary to learn). The only way to circumvent that is plausibly apply better inductive biases, and fundamentally rethink what the field considers important.

Barrin92 7y ago
I think the obvious problem is that induction (which is what learning from data is), is simply only one tool in the huge space that is intelligence, and it will never be enough to emulate the skill of a human driver, which is more or less what is necessary for autonomy in an open environment.
- ibeckermayer 7y ago
  The "induction" that machine learning algorithms do also isn't the same as the induction that humans preform. We induce new concepts from experience (data) -- The description itself assumes consciousness in both "concepts" and "experience". Thinking of computers as getting more "intelligent" like humans is a category error -- computers are dumb matter configured in an intelligent way by actual intelligence (humans) to preform certain tasks for us. We get better at telling them how to preform certain tasks (software), but there's no reason to think we're moving along some continuum of intelligence towards us.
  - visarga 7y ago
    > We induce new concepts from experience (data) That's the premise of deep learning - inducing high level concepts from experience, without manual feature engineering. DL models are good at induction, what they can't handle is generalisation (being accurate outside of distribution). And self driving has a long tail, that's why it's so hard.

alricb 7y ago

It took me a while to figure out that "AV" means autonomous vehicle... I was kind of wondering what AV as in AV club had to do with trucking.

Adams472 7y ago

Thank you for sharing this. The insights and details you share here will help many future founders. I'm sorry things didn't end up in the exact way your team may have hoped. I hope you can take pride in everything you accomplished. I wish you all the best!

mrfusion 7y ago

Why not an aquihire? Surely lots of companies would want to augment their av teams?

savrajsingh 7y ago
I’m sure it was explored but it didn’t close. Stuff like this is annoying for a founder to hear because he’s like “aw yeah, why didn’t I think of that earlier?!” Of course he thought of it — but it just didn’t work out that way.

mrfusion 7y ago

Welcome to the trough of disillusionment. (sp) I think the plateau of productivity will be really awesome computerized copilots and safety assists/warnings. The human computer driving team could Be quite a combo of designed right.

mrfusion 7y ago
Perhaps for true self driving we’ll need one more breakthrough in AI. Perhaps a system that can identify its own edge cases and request additional training or seek out the training itself.
- seibelj 7y ago
  We need AGI or we need to ban humans from driving, pedestrians, bikers, children, deer, and anything that can disrupt a perfect computer model.
stefan8r 7y ago
The problem is that most of those businesses don't make sense with the current cost of talent. The L2 systems OEMs are rolling out are primarily made by engineers in low-cost areas at $60-90k/yr, not $130-250k/yr in SV. Realistically, most of the AV talent will go to industries that can afford them. Salary isn't super elastic.
- mrfusion 7y ago
  The engineers in the low cost of living areas do the real work. We don’t need rockstars.
mark-r 7y ago
It's well known that if the computer is handling a significant part of the task, the human will get distracted or zone out completely. The "human computer driving team" will end up being less than the sum of its parts. Airline pilots already have well established procedures to prevent this from happening when they engage the auto-pilot.

xiaolingxiao 7y ago

This is a very good and honest retrospective. He shows clear thinking, and surprising technical understanding for someone who is not technical. And most importantly he shows humility. A+

cjv 7y ago

This is sad news. I worked at Starsky Robotics as a perception team intern after graduating high school. I will always be grateful for the team for the opportunity, it was a fantastic first job and everyone who worked there was very kind (especially Stefan). Unfortunately, Starsky had effectively had no machine learning in 2017 (when I worked there), using solely classical computer vision techniques. This didn't match the company's ambitions of not using LIDAR and there was a strong stigma against switching to a deep learning approach. At the time, very few object detection models had public implementations and I spent a lot of time trying to get a YOLO9000 and RetinaNet implementations running at real-time speeds. Frustrating, as a small startup the labeling services kept screwing us over by returning poorly annotated images. I think what I took away from the experience is that deep learning in domains with long tails requires a enormous investment in a labeling pipeline - dwarfing the computational aspect - to get decent results. I don't think any solutions are on the horizon that will allow us to bypass this reality. You don't see improvements between Comma.ai and Tesla because it's about the improvements far out on the tail.

lioeters 7y ago
For others like me who were as amused by the name, YOLO9000, it's a real thing, "a state-of-the-art, real-time object detection system that can detect over 9000 object categories". YOLO9000: Better, Faster, Stronger - https://arxiv.org/abs/1612.08242 https://arxiv.org/abs/1612.08242 (2016)
- nstj 7y ago
  YOLO = “You Only Look Once”
jiayq84 7y ago
Just to clarify a little bit... "At the time, very few object detection models had public implementations" - this is wrong. Almost all object detection models had public implementations starting from 2014, most notably Detectron (Caffe), GoogleNet/SSD (Tensorflow and matlab). Post 2015 when TensorFlow was released, one can find even more implementations. Data is the problem. Everyone has the algorithm but not enough people have data (especially labeled ones)

csours 7y ago

Disclaimer up front: I work for General Motors. I don't work on AV. Any opinions are my own. I have no special knowledge of GM's AV strategy. > It didn’t matter that that jump from “sometimes working” to statistically reliable was 10–1000x more work. There's 2 states of functionality: 1) It doesn't work 2) It sometimes works The inverse, for disk drives: Failing and Failed. Think about apps/services. You could say that your app is working, but over a long enough time period, it is only sometimes working. It's working while you have disk space, free memory, and a working network connection. It's working while your business assumptions hold true. It's working while your datacenter has power. We've developed strategies for managing all of these things; for load balancing and Active/Active hosting. But even with that, it only sometimes works. --- With all that, I think it may be useful to think of self driving in terms of tasks. If you can put a box around what you expect a computer to be able to do, you can define tasks that will always return a reasonable output. The more tasks a computer handles, the easier it is for the human in control (think driver assist, like lane keeping and automatic cruise control). If you add enough tasks, and perform them well enough, maybe you can take the human in control out of the vehicle. I think I'm in agreement with the authors that I don't see the day when there isn't a human in control. Or the other way to say that is that if there isn't a human in control, sometimes your AV will just stop.

stefan8r 7y ago
Just stopping is ok is ok in some use cases, but not others. It will be a long time before robotaxis "just stop" less frequently than uberdrivers, but AV trucks can already be more reliable than contemporary drivers.

d_burfoot 7y ago

> The biggest, however, is that supervised machine learning doesn’t live up to the hype. This is the key point. The new DNN approaches can outperform the classical techniques, but only when they can exploit vast amounts of training data. The dramatic successes of Deep Learning all depend on either unsupervised learning against enormous raw datasets (BERT, GPT-2, word2vec, etc) or games, where you can generate unlimited quantities of labelled data by playing the game against your own agent (AlphaGo, AlphaStar, OpenAI Five, etc).

cosmodisk 7y ago
As someone already mentioned here,I think it's probably not a goood approach. The way we store info in dstabases is very limited compated to what we can do in our heads. For instance we know a concept of a table.It can be made of almost any material,can have whatever shape,size, and colour we want and yet we can instantly recognise it without having some concrete data points on what it should look like. I can make a glass cube,put it in a middle of the room and people would know it's a table.How the hell we operate this way,I have no idea.
- wolco 7y ago
  On a basic level we all: - Know what a room looks like. - Expect certain objects to exist at certain places and not expect others - The brain indexes these - Seeing one or two datapoints allows us to guess. So seeing a glass cube in that context gives us few logical chpices (it must be a table) similar to someone typing a letter in a textbox and having it filter a pre-defined list.

edshiro 7y ago

Thank you for sharing this very candid article on the Starsky Robotics and generally the autonomous vehicle space. It's a real eye opener. I've been following your progress for the last few years (and I also read about your company through Reilly Brennan's "Trucks - FOT" newsletter). I am sorry you could not get investors to believe more in what you and your team, especially as you required a lot less funds than many other companies in this arena. I also thought you had a clear business case (I worked in ride-hailing and also logistics so understand some of the problems in this space). I wanted to ask you a question: I am building a startup in the dash cam video analysis space. We are building a large and geographically diverse dataset of road videos, where our users can annotate/label the data. We then are going to look at detecting specific events like accidents and edge cases on videos. Do you feel this type of business, the data we collect, and insights we generate would have value for a AV startup? All the best in your next move. Stay strong - you can be proud of what you and your team did.

stefan8r 7y ago
Hard to say. We had really strict rules on what we wanted our data to look like, and were very specific about subject matter. We probably wouldn't have been able to be a customer. Our intention to deal with accidents/edge cases was that if anything looked outside of ODD (as in, not perfect driving conditions) we'd execute a MRC (pull to side of the road, or stop at next exit). Relatively easy way to solve most of the really hard edge cases.
- edshiro 7y ago
  Understood and makes total sense given the ODD at Starsky.

petermcneeley 7y ago

All this L1/L2/L3 when most of the public failures in AV are at the level of vision and interpretation of the world.

krisoft 7y ago
So? That graph is about the capability of the system and how it improves with effort. This has nothing to do with which part of the system fails the most. Where do you see the contradiction?
- karlding 7y ago
  The choice of naming here is somewhat unfortunate, and probably contributes to some confusion. Level 0-5 is terminology used by SAE to describe the range of fully manual (Level 0) to fully autonomous (Level 5). One would expect that in an article about Autonomous Vehicles, L{0,1,2,3,4,5} would describe the level of autonomy that the vehicle provides, but the article instead uses these to describe arbitrary thresholds of a graph demonstrating the capability of an AI.
  - stefan8r 7y ago
    Fair point. I'm shitty at drawing mathematical graphs. I'll update at some point with different notation.
    - ngcc_hk 7y ago
      I think it makes my day as it is a rare but simple insight into the s curve and the peak line. It will be a useful mental tool for life. Not sure why l meant anything in some other model would affect its reading. Good to me.

jedberg 7y ago

> Instead, the consensus has become that we’re at least 10 years away from self-driving cars. I'm going to assume the founder of a self-driving truck company knows what he's talking about. But at the same time, I have a hard time reconciling that with the fact that I sat in a car that drove itself all around San Francisco, dealing with a ton of edge cases. Maybe we won't get to a 100% drive-anywhere-you-want-car in 10 years, but to be fair, a lot of humans aren't capable of driving a car anywhere either. There are a lot of LA drivers who can't drive in snow, for example. I was one of them, until I got practice, and even then, I'm not that safe at it. I think as long as we set the bar at "drive anywhere a car can go with 100% safety" we will never reach that bar. But if the bar is at "drive as well as a human in most of the places humans drive", well, I've already seen a car that can do that.

stefan8r 7y ago
What's hard about this comparison is knowing exactly what you saw. If the safety driver disengaged 1-3x, it might have felt like a robotaxi drove you all over; but the effort to get that system safe for unmanned regular service might take another decade. If it was super mapped and following a fixed route with object avoidance; the difference between a car that can go on an HD-map track and one that can go point-to-point in a city is also maybe a decade. We humans have limited empathy for what is really really really hard for computers. Many know that's true for an app, but somehow ignore that knowledge when it comes to autonomy.
- greglindahl 7y ago
  Even if jedberg observed 0 disengagements, it's still not the millions of miles of data you'd want to prove something.

pj_mukh 7y ago

Great read! And thanks to the author for all the candor. The business case always seemed clear to me and now reading this, I wonder if there is a case to be made for an engineer and a trucking operations veteran to build a business that requires minimal capital (or maybe even is bootstrapped?!), to take it across the finish line?

stefan8r 7y ago
There are a couple of people working on that, notably CloudTrucks. It's not for the faint of heart - truck drivers and engineers follow a very different set of social norms. It's hard to be the hardass boss drivers don't want to fuck with while being the cuddly CEO engineers like the culture of. That might be a future blogpost.
- ngcc_hk 7y ago
  Waiting for it. Sad but good read.

chubot 7y ago

Thank you for the great writeup. It sounds like you took a responsible, valuable, and even economically viable approach, but the market doesn't want to hear it. I wonder when the market will start listening to people who actually WORK on the problem. > which means that no one should be betting a business on safe AI decision makers. The current companies who are will continue to drain momentum over the next two years Sounds about right. ----- Here's me over 2 years ago simply quoting people who worked close to the problem: https://news.ycombinator.com/item?id=16353541 https://news.ycombinator.com/item?id=16353541 Here's my negative scenario: self-driving is "AI-complete"; you can't really hit all the edge cases without solving AI in general, which is more than 30 years away (Kurzweil is the wild optimist and predicts 2045). You CAN use self-driving in limited circumstances, but those limitations are precisely the ones that make driving yourself around more attractive. The expense doesn't go down as quickly as anticipated because of this. They are a niche technology for DECADES. This looks about right, and I'm not claiming to be prescient, just basically saying what Chris Urmson and Bill Gurley already said years ago. It's weird to me that there's still so much money chasing this pipe dream. It was a sign when Google spun out Waymo. If they really believed in the product, it would be called something like "Google Self-Driving Cars", not "Waymo". ---- edit: it's easy to be negative, so on the positive (and contrarian) side, I believe rideshare is undervalued, and has a long way to go. I bought both Uber and Lyft stock at $16 yesterday (they may go down again, but ping me in 2 years). This comment in this thread is a nice take and makes me even more optimistic: https://news.ycombinator.com/item?id=22631151 https://news.ycombinator.com/item?id=22631151

chrisco255 7y ago
I still think that AI shuttles and buses driving predetermined routes and schedules would be tremendously useful. Especially if they could operate continuously 24/7. And that should be possible, with maybe tele-operators dialing in to override the edge cases or sending dispatch crews to takeover for any issues that come up.
- stefan8r 7y ago
  This is true, but essentially every transit system in the world (minus Hong Kong and Singapore's) operates at a loss. Maybe you can save money vis a vis unionized drivers, but I'm not wildly optimistic.

ChuckMcM 7y ago

Such an excellent and cogent discussion of the challenges of making an autonomous vehicle startup work. I really resonate with the challenge they had making safety "sexy" from a coverage point of view. It seems it shares that attribute with "security" which, when done well, means nothing happens. I had a lot of hope that we would get four lane (two each way) "freight ways"[1]. These would be paved roads designed specifically for autonomous trucks to move freight from one point to another. With sufficient freight ways between key cities on the coast and further inland, and efficient way to move goods with somewhat worse costs than rail and quite a bit better in terms of reaction time. [1] Like a free way but designed specifically to exclusively service autonomous trucks carrying standard containers.

Arbalest 7y ago
This begs the question, how much does it cost to lay road than to lay rail? If it's going to be exclusively autonomous, the more narrow confines of the rail should be well suited.
- stefan8r 7y ago
  Another common fallacy. The US actually has a better rail network than Europe - most people just think it's worse because passenger rail here sucks. That's the case because we prioritize freight trains over passenger (in America Business is #1). To paraphrase Churchill, trucking is the worst form of transportation, besides all the others. If you can put freight on a train, you do. It's slower, point-to-point, and less good a less-than-trainload freight than trucks are.
stefan8r 7y ago
Many many smart people share this thought. I don't. Freeway costs roughly $4m/lane/mile. Building a duplicate system for trucking would make UBI look cheap. Truck taxes also pay for much of current hway infrastructure, so you'd need to replace that $

sitkack 7y ago

The VSSA (Voluntary Safety Self-Assessment) referenced in the post [1] https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1be1791a7332594e0ff28b_Voluntary%20Safety%20Self-Assessment_Starsky%20Robotics.pdf https://uploads-ssl.webflow.com/599d39e79f59ae00017e2107/5c1...

malandrew 7y ago

> In 2019, our truck became the first fully-unmanned truck to drive on a live highway. Uber drove an autonomous truck on a live highway to deliver beer for Anheuser-Busch back in October 2016 https://fortune.com/2016/10/25/uber-anheuser-busch-first-autonomous-truck-beer-delivery/ https://fortune.com/2016/10/25/uber-anheuser-busch-first-aut...

stefan8r 7y ago
There was a person in the vehicle. Amusingly - many VCs who "should know better" fell for the same error. I'd regularly get in arguments with investors about whether others had done unmanned or not.
- malandrew 7y ago
  In the sleeper cabin monitoring the vehicle. The main cabin was unoccupied and there wasn't a single intervention for the entire trip. I knew people involved with that trip at the time.
  - stefan8r 7y ago
    The difference between "in the sleeper cab" and "outside the truck" are huge. We first started doing in the sleeper cab in Jul'17, took us 2 years of dedication to get confidence to remove from vehicle. To take the person out of the vehicle, the system itself needs to be able to self-monitor and pull itself over if it stops working. That's the big lift.
    - malandrew 7y ago
      That's it. The team at Uber did that like a year or so later but with no public fanfare because it wasn't considered noteworthy. That still puts it at ~2 years prior to this accomplishment.

Havoc 7y ago

This to me very much assumes no leap forward. See the start of deep learning - sudden rapid progress. I don’t see why there can’t be another jump like that. Much like the chip manufacturers mentioned in the post. No idea what that advance may be but it’s out there I think

larrydag 7y ago
Perhaps there is no leap forward in long haul trucking. Yet there are many supervised machine learning products in the market and doing well (Tesla autopilot, iRobot vacuums, etc). I bet the founder can find a niche that could be marketable and can be mass produced or scaled.

Quarrelsome 7y ago

If we'd sold AV as "grandma driving" instead of "autopilot" do you think we could have avoided some of this mess? The public are only ever going to accept perfection which prevents us from getting our hoverboards here.

keenmaster 7y ago

Sad news. Apart from improving upon ML, can more sensors and physical modeling get us past the AV plateau? An example of an additional sensor would be an infared camera. I don't know how well infared cameras work during the day or when they are occluded, but they seem like they can provide an additional datapoint to classify an object as "person" for example. That way, it doesn't matter if they're wearing a paper bag Halloween costume, the car still knows what's underneath. Physical modeling seems like it would help with prediction. If a deer is idle on the side of road ahead, a car with a physical "deer" model would account for the probability that the deer will suddenly dart across the road and the speed at which it can possibly do so. It will slow down and watch for certain behaviors, such as the deer standing up from a prone position, and modify its probabilities accordingly. That is, after all, what we humans do. The ML wouldn't need to perfectly model all animals. Some animals would be put into a "mammal" class or more vague "animal" class, paying special attention to size and stature. Additionally, depending on how long it might take to develop truly autonomous self-driving cars, we might want to develop self-driving car friendly infrastructure. I'm talking about AV-only lanes with safeguards in place to minimize variability, vehicle-to-vehicle communication, infrastructure-to-vehicle communication, and better than average maintenance. It seems like it might be worth it, given the gains to be had. Here's an underappreciated angle to the AV-friendly infrastructure: it's a hedge against incorrect predictions. What if truly autonomous vehicles are actually 15 years away? In that case, if an "AV heaven" is developed in a corridor somewhere in Los Angeles 5 years from now, that'll be the only place where the AV dream is realized for the ensuing 10 years. The profit from "AV heaven" can be reinvested into AV development. It can also be used to expand into other areas. Moreover, AV heaven might galvanize the government, public, academia, and investors to eliminate any bottlenecks in AV development. There's no immutable stone tablet, only viewable by God, that says "AV development must take X years." We can modify the timeline to some extent. Lastly, AV heaven would allow us to observe the true value of self driving cars. What if they're better than we ever imagined? What if they radically change human behavior? What if they increase development in surrounding areas by a lot? What if they change the nature of work and where people live by more than we even expected? What if self-driving trucks + last mile bot delivery radically push down prices on everything and enable previously impractical business models or behavior? We actually know many of the changes that AVs will bring, but, critically, we don't know the magnitude of those changes.

ackbar03 7y ago

Not sure about him blaming the "professorly pace" of research though. That's the rate good research is done at. If you were expecting anything faster than you don't really understand what it was going to take in the first place

machinelearning 7y ago

For any engineers reading this who want to solve the computer vision/machine learning problems explained in the blog post - reach out at info@nuronlabs.com.

smabie 7y ago

I'm not sure it's intellectually honest to put all the blame on stupid investors, as a founder it's your job to deliver what the market and the investors want, or convince them otherwise. And if in fact investors and the market is just stupid, that's actually a great sign: you were just early and have a shot at doing it again in the future.

johntiger1 7y ago

Just this hiring season, I still saw positions open...and now this

punnerud 7y ago

Why does every self-driving car company focus on driving like we do today? Seems more viable to make a system where the trucks dock at speed on the highway. This is where the power of AI come in to play by coordinating the two trucks next to each other, switching load and continue driving without ever have to stop. The end goal would be to also move people, but we have to start with something safer that makes economical sense.

DrNuke 7y ago

It is pretty cynical and hindsight is always 20/20, but... is self-driving automotive just trains and metros? There is a fundamental business case flaw in these startups, because regulators worldwide will never approve them go together with human traffic, which is messy, chaotic, smart in its own sense at local level (you need to know the uses of local people, not the general rules... eg. driving a car in Naples is very different from driving a car in Milan). Only dedicated lanes or tunnels then, trains!

blackrock 7y ago

Maybe it’s time they got bought out by a larger company, and go dark. Continue their research in private, until they build the perfect robot.

dwnvoted2hell 7y ago

It all seems rather tragic, especially when the solution to preventing the spread of disease seems to be largely in robotics.

baylessj 7y ago

Thank you for the excellent write-up here. I'm sad to hear that Starsky is dissolving, but I have hope that the lessons learned here will be applicable elsewhere. The point about safety being a low priority with investors makes sense but is unfortunate. I hoped that Google's investment in Waymo would push past this hurdle, but they're latest funding round makes it clear that they also have to deal with investor wants over the success of the technology. Really hoping to see someone get the funding to make this tech work reliably in the future and apply the same safety-first design that Starsky used.

putroce 7y ago

I’ve been developing software for self driving (both road and offroad) the past two years and what I’ve noticed is that there isn’t much innovation in the field. It seems like everyone follows the exact same design paradigm which is using something like Autoware. It’s pretty disheartening to see how much money has been poured into this field and what do we have out of it now? GM Supercruise came out in like 2013 [0] with similar performance to the level of autonomy we’re at in 2020. To clarify, this was in a production cadillac in 2013, so anyone who knows about automotive products knows how long this must have been in development before they were allowed to put it in production. To the author, it’s interesting to read how quick you rose and fell, but it’s good you got out now instead of holding on for years and years. What do you think is next? I’m honestly tired of the lack of progress in self driving, and was wondering what anyone’s thoughts were on home robotics. I feel like that’s the next big thing that VC’s will pour billions into. [0]https://media.gm.com/media/us/en/gm/news.detail.html/content/Pages/news/us/en/2013/Apr/0429-cadillac-super-cruise.html https://media.gm.com/media/us/en/gm/news.detail.html/content...

tnorthcutt 7y ago

> Around November 12 of 2019, our $20m Series B fell apart. We furloughed most of the team on the 15th (probably the worst day of my life), and then started work on selling the company and making sure the team didn’t go without shelter (or visa status, or healthcare for the new and expectant parents). We were able to land many of the vulnerable jobs by the end of January and I’m in the process of selling the assets of the company (which includes a number of patents essential to operating unmanned vehicles). Like the captain of a sinking ship, I’ve gotten most of the crew on lifeboats and am now noticing the icy waters at my ankles while I start to think about what I do next. Well done.

mleonhard 7y ago

Stefan & team: I'm sorry your company failed. I hope you can and will take time now to rest and recover. Things that help recovery from burnout: long walks, therapy, exercise, sleep, meditation, and spending time with people you love.

anandkulkarni 7y ago

Thanks for sharing, Stefan. Lots to be proud of in what Starsky achieved, and factors outside your control at work at the end. Excited to see what you do next.