7 ms·
Better bus predictions (a lot better)
- BugsJustFindMe 8y agoDear MBTA CTO, why isn't this on the MBTA website instead of Medium?
- kuanbutts 8y agoPerhaps it could be cross-posted?
- vcryan 8y agoI work as a software for the MBTA but don't speak for the organization. There are pages on the main site about this project, for example: https://www.mbta.com/projects/better-bus-project https://www.mbta.com/projects/better-bus-project I think the Medium blog was set up for posts intended for a more tech-savvy audience than the broader audience that visits MBTA.com to check schedules, plan trips, view alerts, etc. Maybe at some point it will be merged in. The website was relaunched ~year ago as an Elixir / Phoenix application and there are still quite a few features and content being added.
- kuanbutts 8y agoThis is awesome! I fully believe that government procurement needs serious reform, which this article and effort is clearly attempting to address. So, props to David Block-Schachter (the CTO). There are far too many large, bloated consultancies that specialize not in delivering quality products and services, but rather in "surviving" the government procurement process. Props as well to the Swiftly team - I had a change to take a peak at some of the APIs they expose to their customers and it's quite valuable. In particular, they roll up stop-pair segment performance on routes by time of day, which allows someone to query for bus performance by discrete route-schedule-segments. Gathering this type of data is quite labor intensive and a significant technical lift (I was once part of a project doing this with GTFS-RT data from the NYC MTA). This type of information, and the broader ecosystem of performance related API services they provide to their users (based off the limited amount I have seen), can enable operators to extract highly articulated performance statistics about their fleet, on their own.
- evmar 8y agoIn case anyone else was wondering, it appears that MBTA here refers to a Boston transit agency.
- benrapscallion 8y agoThat’s correct. It has the reputation of being one of the worst commuter systems in existence. http://www.bostonherald.com/news/local_coverage/2017/10/data_mbta_service_worst_among_carriers http://www.bostonherald.com/news/local_coverage/2017/10/data...
- bsder 8y agoTell me where the bus IS, dammit. Why is this rolling out after prediction instead of before? Why are we "predicting" at all? Why doesn't every bus have a tracker that tells you exactly where it is at all times? This is 2018. GPS with refinement isn't rocket science anymore. Tell me where the bus is and I'll do my own prediction thanks.
- vcryan 8y agoYou can see bus locations on the individual bus route pages, for example: https://www.mbta.com/schedules/741/line https://www.mbta.com/schedules/741/line
- bsder 8y agoThey commented that they just started rolling it out this week. The problem is that they should have spent 0% of their time on prediction and 100% of their time on actual tracking. Once you have tracking, prediction pretty much comes along for the ride.
- fastball 8y agoNot really. Just tracking the location in an app isn't going to tell you anything about traffic conditions. Seeing that the bus is only a mile from your house might not be that useful if one day it's in gridlocked traffic and the next it's nothing but open road and green lights.
- calebh 8y agoI use the MBTA buses sometimes to get from my house to the nearest train stop after which I make a transfer to another bus. A real time map of the bus locations would be 100x more valuable than just a prediction. Same with the train - I use a website that shows the exact locations of all the MBTA trains, and I am able to accurately predict how long the train will sit outside of the Alewife station (which is infuriating BTW) based on the number of trains currently at the platform. A prediction would not be able to take into account this domain knowledge and would be completely worthless.
- vcryan 8y agoMy apologies for being such an opportunist, BUT, we are hiring for various engineering and product roles at the moment: https://jobs.lever.co/mbta https://jobs.lever.co/mbta
- DoreenMichele 8y agoIn case you don't know: There is a post on the first of every month called "Who is Hiring." You might do better planning to participate in that. Most recent: https://news.ycombinator.com/item?id=18113144 https://news.ycombinator.com/item?id=18113144
- vcryan 8y agoThanks!
- cypherpunks01 8y agoJust curious if you're finding it challenging to hire for Erlang? Have you generally hired people with some experience, or do you typically expect new devs to pick it up upon hiring?
- katehedgpeth 8y agoI also work in the customer tech department for the MBTA. We use Elixir and all of our web interfaces use Phoenix on top of that. We don’t require anyone to have experience with elixir or erlang; most of our developers have learned on the job. We look for solid general programming skills and a willingness to learn. We’ve had a lot of success with that strategy so far!
- fastball 8y agoDiscord uses Elixir/Erlang VM too!
- cozzyd 8y agoIn theory, one should be able to use information about the stop lights to improve the prediction. Anyway, I really would like to see a histogram of the predicted distribution of arrival times. Sometimes I 1-sigma need to make the bus and sometimes I 3-sigma need to make the bus...
- crote 8y agoEven better, give buses priority at stop lights when they are behind schedule, or intentionally delay them when they are ahead of schedule. This is being done in a few countries and it seems to work quite well. Why predict when you can control it?
- romed 8y agoAwesome. I hope SFMTA can learn from this. They are taking proposals to replace Nextbus currently [1]. When I last measured Nextbus reliability, the first prediction for a bus was weakly correlated with arrival, and the second and third predictions were totally free of information. One of the main problems around here seems to be buses with the route and destination sign set to the wrong thing (e.g. the bus says it is headed to downtown, it's actually going the other way) which apparently throws a huge wrench in Nextbus. 1: https://www.sfchronicle.com/bayarea/article/Muni-looks-to-replace-aging-NextBus-system-with-13204933.php https://www.sfchronicle.com/bayarea/article/Muni-looks-to-re...
- tzs 8y agoI just skimmed the article, so apologies if this was covered. The article suggests people will use these predictions to spend more time at home before leaving their bus stop. If that is the case, what happens if the prediction says their bus will be, say, 10 minutes later, but then that bus catches some lucky breaks with lights, heavy traffic unexpectedly clears up, etc., and the bus makes up 5 minutes of that? People who relied on the prediction could miss their bus. Is there a mechanism to address this? The only one that I can think of offhand that would cover almost all cases would be if the predicted arrival times are also sent to the buses, and if the buses arrive ahead of the prediction they wait until the predicted time before leaving. You wouldn't want to do that for all predictions, though. Maybe only predictions made within 10 minutes of the original scheduled arrival time are binding, or maybe only predictions made when the bus is within three stops of the stop in questions, or something like that.
- why_only_15 8y agoIt explicitly talks about this in the article where he talks about how they try to predict the bus will arrive earlier rather than later. In the 12-30 minute bucket, "12% of weekday predictions were up to 4 minutes early, 71% were up to 6 minutes late, and 17% were outside both of those windows".
- tzs 8y agoSo they are weighting it to under-predict lateness, which will reduce the chances someone misses the bus by arriving at the stop at the predicted time, it seems.
- cimmanom 8y agoAs a bus rider who uses apps to track expected arrival times (not to be confused with scheduled arrival times), here are some thoughts: Timing when you leave the house/office to catch the bus matters a lot more when weather is miserable and/or when buses arrive infrequently than when they arrive frequently and the weather is nice. The bus I take most often is scheduled to run every 3 minutes on average during my morning commute. That means that even with severe bunching I rarely have to wait more than 10 minutes. Unless it's under 10F outside or is raining buckets, I just leave the house when I'm ready to leave the house, and use the bus predictions mostly as a way to manage frustration levels. It takes me about 90 seconds to get from home to the bus stop if I make the light, BUT there's a really long traffic light in between. If the bus is listed as 4 minutes away, I have a 50% chance of catching it (depending on whether it makes the previous light). Less than that and there's a 90% chance I'll miss it. Also, the arrival predictions seem to be based on distance and stop count rather than typical local traffic conditions. It also doesn't take into account times when the bus is empty and skips most stops or is full and not only stops everywhere but boarding and debarking take forever. Which means that while it's eerily accurate for certain stops and times of day, at others I can count on the bus to arrive in half the time predicted and at yet others in twice the time. Unless there's a crash or construction along the route, in which case all bets are off. During my evening commute, the buses are scheduled much further apart (and are a much longer walk from the office). A 10 or 15 min scheduled headway means leaving whenever you're ready works well if the buses are reasonably spaced (regardless of whether they're actually on schedule). But bunching sometimes causes a 20-25 minute wait (and occasionally more). What I find the bus arrival predictions best for is to be informed on those occasions when the bus is half an hour away and your best bet is to say "screw it" and walk or take a cab. On another route I frequent, I get on at the first stop on the line. The next bus is always parked around the corner (unless it's super late completing the incoming leg of the route). The app I use lists it as due in 1 or 0 minutes. The actual arrival time will in fact be the scheduled time. What it comes down to is mostly about knowing your route and adjusting your perception of the predictions to take that into account.
- ajuc 8y agoIn my city there is a system like this (all buses have gps, you can look where they are at any time http://www.sip.ztm.lublin.eu/Default.aspx?lang=EN http://www.sip.ztm.lublin.eu/Default.aspx?lang=EN ), and on bus stops there are these dynamic timetables http://rpw.ztm.lublin.eu/typo3temp/_processed_/csm_01_01_594179becd.jpg http://rpw.ztm.lublin.eu/typo3temp/_processed_/csm_01_01_594... It's working pretty well, the key thing is the point 4 in the article - the timetables show "in X min" if they know how long it will take for the bus to get here, or just "at XX:XX" if they don't know for some reason (transmitter or bus broke, whatever) so they just show the scheduled arrival. So you can be pretty sure that if it says it's coming in 4 minutes you won't be waiting 30 minutes. Clearly marking how sure you are of your prediction is IMHO more important than making very good predictions. I don't care if I have to wait 3 minutes or 6 minutes, I do care if I skip another bus waiting for a more convenient one in 3 minutes that never arrives.
- prolikewh0a 8y agoSeattle has GPS on most busses that integrates into apps like Transit. Transit is also crowdsourced if people leave it running when they're on the bus. Some bus stops also have screen timetables that update live information regarding arrival time. I know I have to leave my condo when Transit says 3 minutes and it has the "this bus is reporting" logo.
- edejong 8y agoHaving developed similar systems in the past, I would like to advice against arbitrary boundaries on accuracy. We used a couple of metrics to assess the quality of the predictions. The first one is sMAPE (Scaled Mean Absolute Precision Error), which tells us quickly where and how our predictions fail. We plot this with the precision on the y-axis and the minutes till arrival on x. Also plotted is bias, which is important, especially since you want to be slightly biased towards being too early. Similar axes. Other metrics we used are MAE (Mean Absolute Error), RSE (Root Square Error) and 2D kernel density plots. In the end, for a contract, I would take into account the density of use of these predictions. It's nice that you can predict perfectly during the middle of the night, but if that line is not used, it is next to useless. So something like sMAPE * passengers or sth like that. Also, even though I cannot relay this one-to-one to the passengers, a confidence interval on predictions is gold.