8 ms·
The author of this article comes off as really very clueless. Yes, in fact, if a store's computers go down they don't just let customers walk out the door with
by colechristensen 6y ago
The author of this article comes off as really very clueless.
Yes, in fact, if a store's computers go down they don't just let customers walk out the door with products and hope they get paid eventually?
As far as I understand colonial is basically a big gas station connected with a series of tubes. Look ma, the gas pump still works but it doesn't keep track of how much we pumped or charge us any more!
Of course they shut down if they couldn't bill?
It's being treated like some kind of greed when in reality they didn't want to give away millions of dollars of fuel they couldn't track or bill for.
The colonial pipeline was fine... except it couldn't do the thing it does which is sell gas to distributors, it could only give it away.
Sigh.
This isn't "fake news", but it is... shitty news? News which gives facts and a ridiculous analysis?
I'm still trying to find out what actually happened in Israel but god forbid a news source or commentator actually lay out the details of a complicated situation... (if you're not getting everybody pissed off about X, then you're not doing your job as a reporter?)
- sam_lowry_ 6y ago> if a store's computers go down they don't just let customers walk out the door with products and hope they get paid eventually? This is exactly what we expect them to do in a crisis situation. This is also how Hollywood portrays ethical behavior. The store owner in Jericho (2006 TV series) is the first to come to mind, but there should be plenty of other examples. And good deeds come back to us.
- andrewstuart2 6y agoI think maybe you have to think about the total economic impact for the article and frustration to make more sense. Fuel is a very fundamental part of our economy and almost always goes toward producing much more value than the value of the fuel itself. Add to that, shortages can and do contribute to irreversible losses like loss of life. If every gas distributor shut down because ransomware hit them all, you could completely grind the country to a halt. It's not too far fetched either to hope that their customers would be willing to self-report. Sure you might get some people who'd take advantage of the situation, but it's not a the total loss scenario of giving away your product for free that you're talking about. Add in the opportunity cost and shutting down delivery if you have alternatives that may not be 100% accurate billing is probably a net loss for the company, though it's impossible to say. > I'm still trying to find out what actually happened in Israel but god forbid a news source or commentator actually lay out the details of a complicated situation... (if you're not getting everybody pissed off about X, then you're not doing your job as a reporter?) It's a bit ironic to take a morals-over-profit stance on reporting right after having taken the profit-over-morals stance of gas pipelines.
- londons_explore 6y agoI wonder if there is a capitalistic approach so solving this problem (making incentives align between the needs of the fuel distributor and that of the economy in the case of technical failure)? The only approach I see is perhaps to have written into contracts with all buyers some kind of compensation for the pipeline being unavailable per hour. Then colonial will prefer to have the pipeline turned on handing out free fuel because it's cheaper than the compensation for turning off.
- MattGaiser 6y ago> It's not too far fetched either to hope that their customers would be willing to self-report. Sure you might get some people who'd take advantage of the situation Would you bet millions of dollars on the goodness of the typical person?
- thanatos519 6y ago
- phreeza 6y agoIsn't the argument that it should be a utility and not a purely for-profit business? Would you expect your local water company to shut off supply if they have a similar bookkeeping problem?
- salawat 6y agoPeople, and particularly software people I've noticed, seem to chronically underestimate the magnitude of B2B where you have a potential number of different vendors to settle with in the 6 digits, that changes over time according to vendor internal reorg, closings, openings, relocations, changes of payment and financial service outsourcing arrangement, etc. That doesn't at all come for free. Entire departments and businesses revolve around making that complexity disappear. Gettting locked out of your infrastructure to do that is essentially paralysis of your business period. It isn't a case of "Just put some meters on the tap of the pipe" . You have to be able to read them, you have to know to read them and when, you've got to know where the invoice goes, what terms were negotiated, etc. If you don't have access to that dataset, the only physical way for the human network to recreate that web is by people picking up phones and blowing up and slamming your point of contact which runs into the issue of having enough people on your end to handle things in the meantime, and your customers actually knowing who to call, and javing planned a procedure for a massive degradation in your info system, even if it means going back to pen and paper.
- romwell 6y ago>Yes, in fact, if a store's computers go down they don't just let customers walk out the door with products and hope they get paid eventually? They better do that if they get the permission from the gov't to operate the only grocery store in the area, and people weren't able to get food otherwise.
- FeepingCreature 6y agoOkay, so establish some sort of "crisis service fund" that pays out when payment processing breaks through no fault of your own. This is something the state can do at any time. Honestly, allowing a monopoly without planning something like this just seems careless. Providing service without hope of ever being paid is stupid. The market is pretty good at selecting against stupidity.
- sam_lowry_ 6y agoI lived both in a high-trust society and in a low-trust society. That you seem to have experienced only low-trust society may impact you worldview, but take my word for it, things are different elsewhere ;-)
- brobdingnagians 6y agoI really like this comment. It is an indictment of our society if we feel like we can't trust anyone around us. I have people around me with whom I'd trust my life; it is much better to surround ourselves with those people. And there are lots of examples of societies historically and now where things operated very largely on trust.
- MrBuddyCasino 6y agoDon't be too harsh. The experience of living in a high-trust society is becoming more and more rare for future generations, due to certain policy decisions. They will just assume things being broken is the natural order.
- manicdee 6y ago
- cm2187 6y agoThe other thing is when you have hackers in your network, shutting down the things that could be physically dangerous if they access it, until you know what you are facing and you identified the compromised machines, doesn't seem unreasonable to me.
- flaubere 6y agoThat depends on what the alternative is. If it involves clueless people filling up garbage bags with gasoline and storing them in their kitchens perhaps letting the pipeline continue to run would have been safer.
- mschuster91 6y ago> Yes, in fact, if a store's computers go down they don't just let customers walk out the door with products and hope they get paid eventually? It's one thing with a supermarket and anonymous masses of customers, but an oil pipeline where each customer is supposed to have meters running on their side as well (as double verification to protect against technical issues)? If you can't trust your known (!) customers enough to not rip you off, something is very much off. And even in that case: the government could have told the pipeline owners "we'll pay off any differences after settling, now go and get that pipeline up and running again".
- Stranger43 6y agoit's also sounding kind of like the simplification an IT clueless manager might make after too much coffee and not enough sleep. it whats likely happened was that the hardened industrial control network was pretty much fine meaning that engineers inside of the pumping facilities could manually direct/redirect flow with all the required safeguards still in place, but that they had no plan for where to direct the flow as all information about who needed/ordered what sat in the unhardened corporate network. This is kind of similar to what happens to warehouses and "just in time" factory lines when the order printer/queue ceases up, or is no longer fed data from the back end system, and while one might expect that the operators of critical infrastructure have a plan for minimal services when/if the unhardened corporate network fails but that plan(even if it existed) likely sat on the same unsecured and now encrypted network.
- helsinkiandrew 6y agoThe difference is that the Colonial Pipeline is seen as an important part of the US Critical National Infrastucture - without oil large parts of the economy and security system breakdown, people can die. It's protected by Homeland Security and the FBI too a much greater extent than your average store.
- mike_d 6y agoOk, so the government steps in and says the pipeline operates without a billing system... now what? Pipes are useless without something to flow through them, and no oil producer is going to fill the pipe without a system to get paid.
- helsinkiandrew 6y agoThe story was that hackers had closed the pipeline (or the owners had closed it because its systems become compromised). If the pipeline is seen as a piece of critical infrastructure [1] - where its closure can damage other parts of the US - then if Colonial didn't tell the FBI, CISA, the Department of Transport (or whoever is in charge of that part of infrastructure) that the reason they were closing it was because they couldn't bill their customers rather than it being a safety issue, I imagine they are in trouble. They were lucky in that it was only down for a short period, perhaps its OK - if it had been down long enough so ambulances stopped running and food and goods weren't delivered - then I'm sure the government would have told them to restart the pipeline and billing would have been sorted out some other way. [1] https://en.wikipedia.org/wiki/Critical_infrastructure_protection https://en.wikipedia.org/wiki/Critical_infrastructure_protec...
- thenoblesunfish 6y agoI think the point they’re really trying to make is that gas should be treated like a utility - you shouldn’t provide it solely based on profitability for a corporation. But you are right that the way things are now, it’s quite clear why they shut down, and that outrage at this particular company seems to really be outrage at the system which it operates in.
- cowsandmilk 6y agoFrankly, you seem to lack knowledge here. Your example of a grocery store is B2C with no knowledge of the customers. The pipeline is B2B with legal contracts in place and use invoices. You can order dozens of types of oil through the pipeline. With travel distances of thousands of miles, many destinations, and the need for product segregation, the pipeline knows where all the oil is going. If they couldn’t track segregation and destinations in the pipeline, that is a good reason to shut down. If they were just going to be delayed in sending out invoices or had to input some items into the system manually after recovery, they most definitely should not have shut down.
- croutonwagon 6y ago>If they couldn’t track segregation and destinations in the pipeline, that is a good reason to shut down. Wouldnt that then conceded the pipeline was not fine? >If they were just going to be delayed in sending out invoices or had to input some items into the system manually after recovery, they most definitely should not have shut down. I am willing to bet a signifigant number of "customers" (businesses in this case with teams of lawyers) would challenge the vailidity of invoices based on estimates and not actual usage. Theres no free lunch, the pipeline owners understand this and decided its in the best interest for their company to stop deliveries until the system that manages their order tracking is back online. Thats totally reasonable. The US government (and states for that matter) made 0 guarantees of coverage of any thing. And even if they did, its not like the government does much to make anything a straightforward process.
- Rastonbury 6y ago> The pipeline is B2B with legal contracts in place and use invoices. What if billing is tied to metered usage and it was impossible to meter? Or what if the billing system is needed to shut off access when customer has reached a quota? The article is scant on details but "legal contracts and invoices" are not going to save a company legal fees with when multiple customers dispute their bills.
- crispyambulance 6y agoIt's understandable why they would shut it down temporarily if they could no longer accurately bill customers for usage. At some point, however, it starts impacting the economy and there would have to measures in place to get the fuel going regardless of whether the corporate bean-counters (and their malware infested computers) were fully "ready". I don't know what that point in time should be. But the fact that there were news stories of imbeciles hoarding gasoline and emptying fully loaded gas stations within a couple of hours certainly makes it urgent to get the pipeline running sooner. Can't we all just get along?
- whoopdedo 6y agoI was in a fast-food restaurant when their credit card processor went down. They continued to take orders and if the customer didn't have cash the food was free. The cost of poor service and losing repeat customers was greater than the reduction in profit. Now, this was a semi-independent franchise owned by a local family. Across the street, the chain big-box store had the same thing happen and they put a sign on the door saying "cash only".
- vsareto 6y ago>Of course they shut down if they couldn't bill? I know it's a commercial operation but there's strategic reasons to keep it up. The attention it got from the US is evidence of that. Plus, they could have just made an educated guess on billing and resolved it later. Even the US could have made cash flow happen if that was an issue. And anyone who tried to take advantage of that would likely get the US's attention. Not worth it long-term. But all this was overshadowed by the risk of ransomware possibly attacking the industrial systems which is really why it was shut down.
- macspoofing 6y agoI agree with you that the company can't make a decision to simply ship fuel for free. This is where the government should step in. You can't have this kind of wide-scale economic disruption occurring. Government could mandate that they resume shipments, have them bill manually as best they can, and maybe use tax dollars can plug any holes after the fact (and if necessary). They should also be fined for letting their billing system get compromised.
- chiefalchemist 6y agoRegardless, the moral of the story is this: To hack and shut down infrastructure does *not* require penetrating the actual infrastructure. Instead there are (likely far more vulnerable) "support systems" that if compromised will have the same effect. Why pick a hardened target when a soft one will do? It's like arresting Al Capone. He didn't go down for all the violence, etc., instead it was tax evasion.
- koheripbal 6y agoThe author here is misrepresenting his sources to spin a specific agenda point, in order to construct a false narrative. It's as fake as fake news gets. From the author's own sources... > Colonial’s corporate IT network and the process control network are connected and exchange information about how much fuel each supplier or distributor receives in order to bill them for it. > ...they could change [data about the] flow rates, they could modify the data > Although infecting Colonial Pipeline’s process control network would be disruptive, it isn’t the only concern. Colonial’s control system also connects to the control systems at tank farms that feed fuel into Colonial’s pipeline.... An attacker can potentially pass through Colonial’s control systems into the control systems of these farms. I really hate advocacy journalism. It's such a fundamentally dishonest medium.
- sam0x17 6y agoThis wasn't even a shitty take -- most of the developed world has government-owned utilities so the thought that the entire east coast gas infrastructure could go down just because some private company had their billing infrastructure locked up is not only laughable, it's morally outrageous. This situation is just begging for federal regulation, and that's the conversation we should be having -- should laws be passed requiring companies like Colonial to keep pumping gas (on their on dime) even when their billing infrastructure goes down? I think yes and I think most of the general public would as well. Instead of having that discussion, we have no discussion because people like you have decided to flag this issue. This is the problem with HN.