10 ms·
All BART trains were stopped due to ‘computer networking problem’
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- edgineer 1y ago[flagged]
- deleted 1y ago[deleted]
- CoffeeOnWrite 1y agoI’m worried about BART. The new gate tap scanners work so poorly, somebody really screwed those up, and they haven’t publicly explained what went wrong. I don’t understand why the media doesn’t cover this apparent bad screw up. There were a number of articles on the selection of the design and the planned rollout and the demonstrated susceptibility to ongoing fare evasion, but silence on the adverse impact on paying passengers. Every day the last week at my station there are piles of commuters held up by the semi-broken scanners. Kudos to the front line staff down there apologizing. I am not holding my breath it’ll be fixed anytime soon.
- abeppu 1y agoYesterday I went through Glen Park Bart, and every one of the scanners had a taped-on hand-written note in sharpie that said "hold 4 seconds". I doubt that the taller gates will ever pay for themselves (I thought in general fare-enforcement costs meaningfully more than systems lose in evaded fares), but I definitely don't understand why we needed to "update" gates with scanners that are clearly worse.
- scarmig 1y agoThe gates are less about direct fare recovery and more about limiting the externalities that gate jumpers impose on other passengers. BART is definitely calmer and cleaner than it was a year ago.
- jeffbee 1y agoThe point of the new scanners is you can pay to ride with any payment instrument, not just a Clipper card.
- joshuamorton 1y ago> (I thought in general fare-enforcement costs meaningfully more than systems lose in evaded fares) This is generally true for active fare enforcement, since you have to pay employees to do enforcement, there's appeals processes, and some people just don't pay the fine. If the new faregates result in some more people (lets say 1%!) paying for trips (less freeriding), that's directly 1-2 million per year. If they also increase real-ridership, that's additional income. To make the cost back quickly you do need a significant increase (10-15%), that's not totally out of the question, though it's probably not only due to faregates.
- kaladin-jasnah 1y agoIt's interesting this is such a large problem there. In NYC, the tap-to-pay OMNY system is literally excellent. I swear it processes cards in a second or less and you save a lot of time by not buying a MetroCard. Are the BART scanners for some sort of RFID/NFC cards that are local to the train system, or do they accept credit/debit card payment? I'm curious how OMNY is so fast (compared to even going to a store and paying with tap to pay) and why BART is slow.
- trollbridge 1y agoAnd OMNY will either accept a tap to pay credit card, a phone with tap to pay (including Apple's express card option, where it works even if your phone is dead), or a prepaid value dedicated OMNY card. It does not actually run the transaction through the entire way - if a transaction fails, the card info gets placed on a blacklist and that particular NFC device won't receive an instant authorisation next time. Generally speaking, people don't have an easy way to generate lots and lots of fake NFC devices, so this hasn't been a problem for widespread fraud (vs just jumping over the turnstile).
- beezlebroxxxxxx 1y agoI don't know if this is the case with BART, but I've seen in my own local public transit network an incredible inability or lack of will to use systems proven in other areas. Instead of going with quick, proven, and reliable, systems, they'll default to going for the cheap option, which is usually slow, re-inventing the wheel, and unreliable. I visit a city like NYC or London or Tokyo and see a transit system with decades of accrued understanding, technical experience, and optimization. Then I come home and ride something shiny but slow, janky, and bug ridden.
- Symbiote 1y agoNYC wisely decided to license London's system.
- kelnos 1y agoMy one huge gripe with Tokyo's/Japan's transit system is that you can't load money onto an IC card using a credit card. I was just in Japan last month; my last trip before then had been in 2017, and I really assumed that they would have fixed this deficiency by now. I forgot to get a Tokyo metro pass along with my train ticket from Narita to Tokyo, so I ended up wandering around Nippori station looking for an ATM that would take my Visa debit card so I could get some cash in order to top up my Pasmo card[0] and take a local train to Shinjuku. I'd also forgotten how most random ATMs don't take foreign debit cards; after it dawned on me, I left the station, found a 7-11, and used the ATM there. This was not something I enjoyed having to do, while carrying my bags, after a 10-hour flight, plus 2 hours waiting in line at immigration, followed by another hour-long train ride to the city. I was pleasantly surprised to find that many many more businesses in Japan take credit cards nowadays though. [0] I was also pleasantly surprised to find that Pasmo cards are still good up to 10 years after the last time you use them. I feel like something similar in the US would expire after no more than a year or two.
- jeffbee 1y agoBART isn't the only agency suffering from those new readers. The problem is the vendor, Cubic. They supply this junk to many agencies.
- whalesalad 1y agoI lived in DC for a few years before I moved to SF, spending a few years there as well. DC Metro is not perfect by any means, but I was really surprised at how much better transit was in the district than the bay area considering they have zero weather issues to contend with and are - ahem - home to a tremendous amount of technological talent. My personal observations are really that California are just truly fucking terrible at this sort of thing. Ironic considering they are such a huge economy and so wealthy. In northern California PCH (pacific coast highway) has been closed for over 15 months due to a rockslide. In southern california, a huge segment of ACH (angeles crest highway, one of my favorite places on earth) has been closed since 2023! You cannot drive from one end of the range to the other at this time. China would have fixed these issues in weeks. For all the cash and people they have, Cali really manages to drop the ball on these things constantly. Don't even get me started on high speed rail that was built out in the middle of buttfuck farmland from Madera to Shafter. Like a stairway to nowhere.
- Hilift 1y agoDC Metro can use funds from developing property near Metro stations. This has been one of the positive things about Metro. The Purple line has evolved into a financial disaster. It will be the most expensive train line ever constructed, $10 billion for a 16 mile segment. BART ridership is 40% of what it was in 2019. There's no way you can lose 60% of your customers and not have major problems.
- simoncion 1y ago> There's no way you can lose 60% of your customers and not have major problems. There is a way, but (with how much they used to crow that (unlike those other big, fat, LOSER mass-transit companies in the area) 80->95% of their budget came from rider fares) the BART management pretty clearly really hates to do it: Make up the shortfall with government money.
- hedora 1y agoAre they actually building housing near the new bart stations? South Bay is doing everything it can to make sure no new houses are serviced by existing train route. They even tore out the line that ran through most of the residential areas decades ago. That line serviced Cupertino, Saratoga, Los Gatos, the San Lorenzo Valley and Santa Cruz. It ran all the way to SF, so it must have serviced many other cities on the peninsula.
- a_t48 1y agoIIRC it’s the same sensors as the old ones
- CoffeeOnWrite 1y agoI have no idea and didn’t comment on what the sensors are. Before the new gates, they worked. Now they only half work.
- fhkatari 1y agoI used to live in Oakland, and am really sad to see continued failures of BART. They had started significant expansions right before the pandemic, allowing a long (but no traffic) ride possible from Oakland to San Jose. A key challenge for BART is that they depend a lot more on ticket sales than subsidies, and as a result, have been hit much harder with lower ridership.
- ryanmcbride 1y agoI'd say the BART administrators overpaying themselves is the bigger moneysink. Not to mention the spending 73M dollars to overhaul the gates to stop fare evaders, when it SHOULD just be free for bay area residents.
- darth_avocado 1y agoThe problem is that the Bay Area needs a single transit agency. All the agencies fighting each other for funding doesn’t help. It’s a much easier task to levy a minuscule tax on the entire region that pays for free public transit. If BART wants to do that at the moment, they can’t.
- trollbridge 1y agoI live in an area with a county-wide transit sales tax, and it just seems to result in them running a lot of empty buses around and building (and eventually abandoning) transit centres. Full price is $1.50 (which if paying cash means fiddling around with coins) except for the 50+ mile express routes which are $2.50.
- darth_avocado 1y agoWell you could be in the Bay Area where you’d have 27 agencies, that still run a lot of empty routes and build useless and expensive transit centers, and then charge way more than what you have.
- jeffbee 1y ago
- scarmig 1y agoI don't know the last time the entire BART system got shut down. There was a loud explosion earlier this morning, which may or may not be related, but there aren't any power outages. No information as to the actual cause right now. Easy to speculate that it's a cyberattack, but I'm going to go for the free square and wildly guess it is a DNS issue. Best wishes and godspeed to the folks who are working on fixing the issue, whatever it is.
- AStonesThrow 1y ago> wildly guess it is a DNS issue Riding around in Waymos here, in the Jaguar models there is a button on the console labeled "DNS" and I have no desire to grab the steering wheel or adjust any other control, except every time I climb in, I am sorely, sorely tempted to press this "DNS" button because (1) I do not know what it does and (2) I have always had a soft spot in my sysadmin's heart for DNS in particular. Please do not reply to tell me what "DNS" means in a motor vehicle, because you will ruin the mystique.
- scarmig 1y agoOh, that just updates the AAAA record for the Waymo. Totally harmless.
- ToucanLoucan 1y agoDo Not Slide. So just hit that if you lose traction, or if you're having a hard time getting your kid out of the park.
- dmoy 1y agoSorry I will ruin the mystique: It's Do Not Schedule, carry over from when there was a human behind the wheel.
- jeffbee 1y agoI don't see any reason other than sheer speculation to suspect a cyberattack. It can just as easily have been some car crashed into a fiber optic thingy. The last time in recent memory there was a large BART disruption it had been caused by a motorcyclist who somehow flung himself over a fence into the trackway and died. That stopped service in and north of Oakland, which is more than half of the system by riders.
- ujkhsjkdhf234 1y agoI took a trip to Europe last month and watching America fail at all manners of transportation is embarrassing
- TulliusCicero 1y agoHey, at least we're ahead in self-driving cars! But yes, public transit in the states is rather pathetic.
- breakingcups 1y agoThey can even go through specially bored tunnels! Oh wait...
- Holo-epiphyte 1y ago[flagged]
- omnimus 1y agoThat's not exactly common. You might say it's very very rare. Probably as rare as in US.
- ujkhsjkdhf234 1y ago[flagged]
- joshcsimmons 1y ago[flagged]
- spacemadness 1y agoTrust me it’s embarrassing for a lot of us that have traveled to countries with decent rail travel. Part of it is the shamelessness of the car industry for lobbying for decades to shackle everyone to a car. Now the car is an engrained part of American culture and makes our cities much more ugly, isolating, loud, and dangerous than they need be.
- 1y ago
- neilv 1y ago> Due to a computer networking problem BART service is suspended system wide until further notice. Seek alternate means of transportation. That's worth a screenshot. Any idea whether the political and technical will is there, to post-mortem this, and make the system more robust and resilient?
- quickthrowman 1y agoRight now I am a subcontractor on a project to replace the Liebert unit in the server room that runs my metro area’s transit system, my worst nightmare is my tech calling and telling me the servers are down, glad this isn’t me!!
- notpushkin 1y agoNow this could make a great anime episode. [0]: https://news.ycombinator.com/item?id=43806281 https://news.ycombinator.com/item?id=43806281
- quotemstr 1y agoDid they finally run out of old new stock 5.25-inch floppy disks?
- more_corn 1y agoThat was my guess.
- whalesalad 1y agoit's always DNS
- gjvc 1y agoI would like to find the person who started this stupid meme and shake them by the neck. The majority of times it's SSL.
- whalesalad 1y agoto some extent SSL depends on DNS lol
- gjvc 1y agofair point
- Spooky23 1y agoThe network guys ducked up again.
- LeoPanthera 1y agoSometimes it's an ethernet loop.
- ashayh 1y ago"Poorer" countries with a per-capita GDP that's 1/5th of Bay Area have exceptional public transport. How about fix the country public transport first instead of spending 2 Trillion on BS AI?
- oldpersonintx 1y ago[dead]
- sershe 1y agoVery poor countries also have exceptional public water wells where people can draw water with buckets. That is because those are inconvenient, slow but necessary amenities in the areas where most people are poor. And that is why US currently doesn't need either. Source: lived in a poor country with exceptional public transit and didn't ever drive till 29; and carried water with buckets from a well only a block away for an aggregate of 1-2 years.
- estebank 1y agoEquating having access to public transport with lack of running water is wild to me. A rich country is not one where poor people drive but rather one where rich people take public transport.
- sershe 1y agoI'm not doing that. I am equating on a spectrum not being able to, as a society, afford running water infrastructure to not being able to afford cars and infrastructure. Note that this includes environmental arguments - saying that 8bil people cannot all live like people in Houston may be true but it's basically saying we cannot afford to have nice things. But the thing is US, and especially major cities, currently can afford them! Why would "rich people take public transit"? Except for extremely dense areas, driving is faster especially accounting for overhead; goes exactly where one needs, any time; and is way more comfortable. Only those who prefer extremely dense areas and also cannot live close to work/amenities (kind of the point of density) would want it.
- a_t48 1y agoOnce the initial shock of “how the hell do I get to work” wore off, it was nice taking the ferry and F to SOMA. Took an extra 20 minutes, but better than the time it would have taken waiting in bridge traffic.
- kelnos 1y agoIf the ferry+F commute is only 20 minutes longer than your car commute, why don't you do it more often?! If I was in that situation, I would never drive! The last time I had a long car commute, it was 45-60 minutes driving, or 2+ hours via various transit options, so driving was unfortunately the only reasonable choice. But 20 minutes extra to avoid driving? I'd do that in a heartbeat. (Hell, I do that often enough just inside SF, taking a bus or train instead of an Uber/Lyft or driving myself... sadly it's not uncommon for a 15-minute car ride to be the equivalent of a 40 minute transit ride.)
- a_t48 1y agoMy usual commute is a <15 minute walk to BART, and a 5 minute walk off BART. The car is mostly for driving around South/East Bay and getting bulk goods in Berkeley. Trying to wean myself off of using it in Oakland but bus service isn’t amazing where I am. I’m with you on avoiding the car commute - would much rather the walk, and feels safer when I’m sleepy. Plus, fuck the 880, drivers on it are crazy even on a calm day - every time my wife and I drive down it, she has the same reaction - “you were driving in this every day?!”
- satiated_grue 1y agoNot quite but nearly totally unrelated, but I love that Debian ships a package of a PDP-8 PAL-like cross-assembler written at BART: https://tracker.debian.org/pkg/palbart https://tracker.debian.org/pkg/palbart http://www.pdp8online.com/ftp/software/palbart/ http://www.pdp8online.com/ftp/software/palbart/
- throwaway81523 1y agoSome system wouldn't power up this morning. System is back in service now, with major delays in all directions. https://www.sfgate.com/bayarea/article/no-bart-trains-running-friday-morning-system-wide-20318982.php https://www.sfgate.com/bayarea/article/no-bart-trains-runnin...
- noitpmeder 1y agoTheir HQ lost power yesterday, maybe related?
- jbverschoor 1y agoDue to human failure when operating computer systems*
- dang 1y agoWe changed the URL from https://www.bart.gov/ https://www.bart.gov/ to an article that provides more information about what happened. See also: https://www.sfchronicle.com/bayarea/article/bart-shut-down-transit-20318959.php https://www.sfchronicle.com/bayarea/article/bart-shut-down-t... (https://archive.ph/LnvJ1 https://archive.ph/LnvJ1) https://sfstandard.com/2025/05/09/bart-service-shuts-down-computer-networking-problem/ https://sfstandard.com/2025/05/09/bart-service-shuts-down-co... https://www.nytimes.com/2025/05/09/us/bart-train-shutdown.html https://www.nytimes.com/2025/05/09/us/bart-train-shutdown.ht... (https://web.archive.org/web/20250509152319/https://www.nytimes.com/2025/05/09/us/bart-train-shutdown.html https://web.archive.org/web/20250509152319/https://www.nytim...) https://abc7news.com/post/systemwide-bart-shutdown-due-train-control-system-problem-agency-says/16365873/ https://abc7news.com/post/systemwide-bart-shutdown-due-train...
- OptionOfT 1y ago[flagged]
- talldatethrow 1y agoMost people I know wouldn't take Bart if it was free. Dirty, noisy, and unsafe. I can't imagine anything can be done to make Americans take trains at scale unless atleast the unsafe part is handled, and then the dirty part atleast.
- kiroguku 1y agoPostmortem thread posted: https://x.com/SFBART/status/1920945151419003225 https://x.com/SFBART/status/1920945151419003225 >The root cause of the disruption was related to network devices having intermittent connectivity. Staff in the Operations Control Center lacked the visibility of the track circuits and the train positions necessary for safe operations. Visibility of this system in the Operations Control Center is required to run service. > BART’s Network Engineering team identified and isolated a redundant sector of the network that was causing intermittent visibility and disconnected it. This allowed service to begin.
- kelnos 1y agoA "redundant sector of the network that was causing intermittent visibility and disconnected it"? That sounds like some sort of network loop?
- great_wubwub 1y ago137 comments as I write this and not a single one says "spanning tree loop".
- nashashmi 1y agoNYC MTA has an old mechanical system in place to prevent collisions. And they also have a new system in place to keep things safe. And the directors insisted both run side by side. This delayed so many projects for so many years. It took ten years to put in place train time estimation. I guess today those same people feel a level of validation on what they insisted.
- LeoPanthera 1y agoI don't know if years of delays is better than an outage of only a few hours.
- nashashmi 1y agoDepends on what you prioritize: reliability of service VS features.
- teleforce 1y ago>“Once the crews isolated that exact section that had the devices that were not properly communicating to each other, they were able to just simply disconnect them,” she told KQED. “That is what allowed us to finally get service back up and running.” Should it not be "they were able to just simply reconnect them"? Surprise to see no exact problem been given in the article and comments section. Curious to see is it a legit computer networking problem, and if yes what they actually were? Could they install a proper fail over connection to prevent the outage in the first place?
- AStonesThrow 1y agoIt seems that many BART supporters are suffering from a fatal fallacy which I also believed until recently. What is the purpose of public transit? Do you believe that public transit is designed so that poor people can commute to jobs? Wrong. Public transit is designed so that people can go shopping. Buses and trains move people around to shopping centers and stores and malls. It is only by accident that poor people can commute to some jobs with public transit. There are far, far more jobs that are more-or-less unreachable by bus or train, and poor people end up walking, cycling, or moving closer to those jobs. When I was a young child, Grandma didn't drive, and so our weekends were consumed by walking around the neighborhood, shopping and eating in restaurants. We'd hang out at the 7-Eleven playing Centipede, Missile Command, and Asteroids. We'd pick up some sodas and Cracked magazine and go home. Then later in life, we started going to malls. None of the malls were walking-distance from Grandma's house, so we rode the bus. And I fell in love with public transit; we'd ride the bus to any one of 3-4 malls nearby, walk around to our heart's content, and ride the bus back home carrying our spoils for the day. It was always a treat to do this. What I definitely noticed was that a lot of poor people rode the bus. What I didn't realize is that mostly, they didn't have anywhere else to go. It wasn't a matter of commuting to their jobs, but just hanging out for the day. Here in Phoenix, most bus stations are in shopping malls, or they become de facto shopping centers, and the light rail corridor is basically a commerce incubator by the way stores and shopping centers are popping up now that the track is permanently laid down. On the weekends it may be common to see working-class Hispanic moms take their children to church on the free buses, but the free buses are intended to get tourists and residents into the shopping areas and/or connect them to the full-fare routes so that they can really do some hardcore "shop 'til you drop" activity. In fact, the public transit sectors that are designed for commuting are the Express buses, which have a higher base fare and serve 9-5 white-collar office workers. It's transparently upper-crust. None of those Express routes can possibly help poor people get to poor-people jobs. Express buses get people into the city center so they don't need cars there: attorneys, civil servants, accountants, clerks. That's the Express system only. The rest of transit: shopping, shopping and more shopping.
- albert_e 1y agoRequest: Make titles more accessible to global audience. BART doesn't automatically mean anything unless you are familiar with the region. I lived 7 years in the US and still took a while (even after opening the linked article -- given the website was a local media outlet, there were no immediate clues in the article which country/state/city it is referring to)
- hglover945 1y ago[dead]
- micahsebastian8 1y ago[dead]
- sr9669604 1y ago[dead]