6 ms·
Ask HN: What's behind all the UK IT failures this month?
This month we've had McDonald's, Sainsbury's and now Greggs forced to shut temporarily because their payment systems have gone down.
Surely these can't be unconnected? Can anyone shed any light on what's happening here behind the scenes?
- jjgreen 3y agoMcDonalds, Sainsbury, Tesco, barely matters. But Greggs, that's some serious shit depriving people of pies. We need answers.
- 0898 3y agoIs Putin hitting us where it hurts?
- gertrunde 3y agoQuite probably. This could be an attack on national security, after all, Greggs sausage rolls are the fuel that runs the UK police forces, much like donuts in the US ;)
- abirch 3y agoAs an American an attack on donuts should trigger NATO article 5
- fifteen1506 3y agoPlease watch "The Undeclared War", on UK-Russia Cyberwar [fictional TV series] so the 2nd season gets produced.
- timthorn 3y agoHannah Khalique-Brown taught herself Python and C++ as well as playing with IDA Pro to prepare for her role in that.
- gadders 3y agoThe south is organising food drops to the North of England as we speak.
- jjgreen 3y agoPoncey southern bastards. Still, pies come before pride ...
- buggeryorkshire 3y agoPukka Pies can pissoff!
- gadders 3y agoHaving said this, though, my nearest Kent town has 2 Greggs, 3 if you count a garage on the outskirts of town. Even Canary Wharf has a Greggs now.
- drcongo 3y ago"Pies before pride" would be a great t-shirt.
- dsattt 3y agoProbably leetcode.
- i_have_an_idea 3y agoRandom chance and years of underinvestment and tech debt
- abulman 3y agoMaybe all the interns that were left in charge didn't properly do what the AI had said?
- InsomniacL 3y agomore likely all the interns that were left in charge DID properly do what the AI had said...
- jesterson 3y agoMost plausible explanation of all
- Joeboy 3y agoMaybe the dog[0] failed? [0] https://quoteinvestigator.com/2022/01/30/future-factory/ https://quoteinvestigator.com/2022/01/30/future-factory/
- usui 3y agoMcDonald's wasn't unique to the UK as it was global. McDonald's Japan was affected.
- password54321 3y agoWork is getting outsourced to India. Most CS grads don’t know how a computer works. People get hired for reasons other than merit. And ChatGPT…
- smcl 3y agoOnly 3 comments into a story about a UK problem and here's a comment blaming it on foreigners. Incredible.
- deleted 3y ago[deleted]
- soco 3y agoI take the spirit of that comment that they didn't blame the foreigners, they blamed the outsourcing. Because even if the provider was local the result would have likely been similar (and we do have recent examples, Horizon and not only)
- smcl 3y agoThey didn't say "Work is getting outsourced" they said said "Work is getting outsourced to India"
- alephnerd 3y agoWhich is even more jarring as McD's platform team is in their Chicagoland office. The amount of unbridled racism on HN is getting annoying. We outsource to Israel, Poland, Romania, etc as well, yet no one is acting racist about them.
- p_l 3y agoBecause TCS, HCL, Wipro, Infosys, and some western companies also keeping outsourcing teams there (i.e. Atos) provide consistently below-absolute-zero quality that pushes and enforces bad stereotypes. Because the real issue is not Indian people or their skills, but corporations that established a pattern of supplying people whose only skill is alleged knowledge of english language. The best thing I can say about those companies is that their IT sweatshops are better than starving, and those who learn anything seem to escape to better workplaces. I have yet to meet as consistent (negative) quality signal as finding out the job has been outsourced to consulting company in India. I have worked with, well, low-quality coworkers (and betcha been one myself more than once) from many countries - but nowhere else there is such negative selection. For comparison, there's no such correlation if the coworker is indian but working elsewhere.
- jl6 3y agoThey could definitely all be unconnected. It’s not that suspicious. I guess we’ll find out if they ever publish post mortems.
- ksec 3y agoMcDonald was worldwide. Sainsbury, Tesco and Greggs seems to be related somehow? At least they are all payment related.
- buggeryorkshire 3y agoJust Eat went down Friday, too...
- thinkingemote 3y agoSainsburys said it was due to an error with "an overnight software update". Perhaps they all use the same software but updated at different times? https://twitter.com/sainsburys/status/1768972295622553900 https://twitter.com/sainsburys/status/1768972295622553900
- nonrandomstring 3y agoPoliticisation of technology. It's a layer 8 problem. Human political expectations of technology are outpacing the engineering reality. Over the past two decades overselling of "utopias" mixed with professional management arrogance and ignoring engineers, experts and what people actually want. That, plus a mixture of crony contracts and bad project management. We've ended up with over-complex systems that we don't have the human capacity or money to maintain and secure. A telling remark by a Labour politician to the question "What is the greatest concern about a 'cashless economy' was "Making sure people are not left behind". In other words, we're forcing this on people whether they want it or not, and whether it works or not. The idea that there might be "risks" or necessary safeguards was simply not conceivable. That's what happened with the Post Office Horizon system and it's what's happening with other new systems pushed out "for our own good" That's not a remark against technology or modernisation, it's a criticism of bloody-minded recklessness and anti-democratic hubris. Good technology requires care. We talk about it a lot here [0]. [0] https://cybershow.uk https://cybershow.uk
- bilekas 3y ago> Politicisation of technology. > It's a layer 8 problem. Human political expectations of technology are outpacing the engineering reality. Over the past two decades overselling of "utopias" mixed with professional management arrogance and ignoring engineers, experts and what people actually want. > That, plus a mixture of crony contracts and bad project management. > We've ended up with over-complex systems that we don't have the human capacity or money to maintain and secure. That is a LOT of VERY sweeping statements to a pretty specific question it seems of payment processors, can you give some examples of what you're talking about ?
- nonrandomstring 3y ago> can you give some examples of what you're talking about ? PLease see the podcast linked above where we have spent many hours researching, analysing, writing and presenting on specific cases. In episodes to come we're working on joining the dots, finding common factors in these cases. Enjoy, and please give feedback if you feel able.
- damvigilante 3y agoMandatory cert rotation forced by a dependency that had a leak, and they all didn’t handle it properly. (Just a guess)
- DarkNova6 3y agoAll of these enterprises are examples where IT is a "cost center", not a "value center". So you have this conundrum where everything goes smooth and the higher ups ask "why do we spend so much money on this without anything in return?". And when something does go wrong they are likely to repeat said question.
- phtrivier 3y agoDo you have sources about the "outages" ? (did not make the headlines internationally..) It they all shut down because of "a payment system", then a simple explanation would be a failure of... the same payment system that they all happen to use ? (It's not like Greggs is going to fully develop a payment system in house without relying on any infra, I suppose ?)
- oakesm9 3y agoSainsbury's and Tesco: https://www.theguardian.com/business/2024/mar/16/sainsburys-grocery-deliveries-hit-by-technical-issue https://www.theguardian.com/business/2024/mar/16/sainsburys-... Greggs: https://www.theguardian.com/business/2024/mar/20/greggs-shops-close-it-glitch-payments-tech-problems https://www.theguardian.com/business/2024/mar/20/greggs-shop... The Sainsbury's and Tesco ones happened on the same day (last Saturday), but the scale was very different. Sainsbury's were unable to take card payments nationwide. My local one said it was due to the internet connection in their store not working. It also affected planned deliveries, likely because the scanners they use to fetch and pick the orders didn't have a connection. Sainsbury's blamed this on a botched "software update". Greggs happened today and was card payment related but doesn't seem to be nationwide (I've not been to one today so not 100% sure if that's true).