12 ms·
Bank of England to crack down on 'secretive' cloud computing services
- politician 5y agoTLDR "Secretive" means "big providers could dictate terms and conditions - as well as prices - to key financial firms."
- DecoPerson 5y agoThanks, I only skimmed this headline and assumed “secretive” meant groups of developers set up their own AWS/GCP/Azure accounts because they were hitting walls with the company’s IT dept.
- daniellarusso 5y agoThat is what I thought the article was about, too.
- bob1029 5y agoWe do business in the US financial sector, and the sentiment we are getting with regard to cloud vs on-prem seems to be growing into a bimodal distribution. I would say it's nearly a 50/50 split until we have conversations about how our product actually works and the incredibly sticky problem that is PII... Once the risks are reviewed in open and honest ways, we find that virtually all of our clients would prefer to keep our solution on-prem. Only those who have already made a full step into cloud compute have to continue to take exception for obvious practicality reasons.
- matheusmoreira 5y ago> Once the risks are reviewed in open and honest ways, we find that virtually all of our clients would prefer to keep our solution on-prem. I hope this kind of risk assessment becomes more common. I'm used to people not caring until things blow up on their faces.
- jiriknesl 5y agoWell, I guess you are speaking about companies who don't know what they do. But this article speaks about banks, who have hundreds of servers, ability to recover anything, full-time employed ops teams, monitoring & automation in place for 20 years already. Moving to cloud provides very little advantage (definitely not financially) to such companies. They are not SaaS who might need to double their infrastructure overnight. Lots of advantages provided by cloud has been in place years before clouds because of VMWare. Another thing is existing bank apps are often monoliths (when you are happy, in Java with Spring, C# .NET, when not, in Visual Basic, Cobol, PowerBuilder, SAP) or huge interconnected services, huge databases with thousands of stored procedures, SOAP APIs in place. This is not a place where you would leverage ElasticBeanstalk or AWS Lambda.
- garethmcc 5y agoIts not as simple as that. Just because its the case of "we have lots of old stuff" doesn't mean you need to ignore the new stuff. Building solutions with traditional data centers and staff, even if you have a lot of it, is often a lot slower. You can spin up entire fleets of servers (or even use services such as AWS Lambda, API Gateway and DynamoDB so you never use servers) and get a solution out in much less the time. Its not just about responding to load but also to be fast enough to get new stuff out there. Traditional financial services organisations are notoriously slow to adapt to changes in the market. Using cloud resources alongside the legacy infrastructure is one way to try and remain competitive.
- bob1029 5y ago> Traditional financial services organisations are notoriously slow to adapt to changes in the market. Using cloud resources alongside the legacy infrastructure is one way to try and remain competitive. Or, you could have a single executive action revamp IT resource acquisition policies. Simply mandate that internalized self-service resource provisioning capabilities be developed. It is not rocket science to put a web dashboard around vmware or some other virtualization solution. Most of them already sell something like this as part of their feature set. You could set something like this up in a week if you had enough buy-in. There are no excuses if you want to win at this kind of game. The bad guys are way more patient in aggregate. A 2nd perspective - One of our customers has a "traditional" process for setting up new IT workloads, and we were still able to get 3 servers provisioned within 8 hours along with 3 new publicly-routable IPv4 addresses and matching DNS+TLS certs. Anything less than this in 2021 for any organization is indicative of sheer incompetence IMO. We do have some customers that are really slow, but they are also really small. I don't think any F500 is taking weeks to provision SQL Server anymore.
- pwarner 5y agoHow is the problem of PII better solved on premises?
- someguydave 5y agobecause you can control physical access to the hardware
- LinuxBender 5y agoJust my observations, but some business entities run into legal challenges that vary greatly by industry. B2B customers have a contractual relationship with you, not your hosting provider. If your hosting provider has an "oops we leaked your data" they only breached the contract with their vendor, not themselves. The contract can specify how data is managed. Adding to this, some companies/banks legal controls are compatible with SOC2 controls of a 3rd party data processor being audited and some companies are not. Some companies can cite the 3rd parties audited certifications and some can not based on preexisting legal contractual agreements with their own customers. I am not a lawyer, but had to sit in many meetings with lawyers and businesses and this is a real issue they have to address. I have also worked for a large bank. Rules, regulations and contracts around 3rd party data processors and financial institutions can get very complicated. There are a myriad of additional complicating variables that go beyond regulations. Legal obligations also vary by relationship. If a bank has preexisting relationships with other banks, they may be obligated to get approval from the other banks to change how their data is managed. Amending contracts is non-trivial. This rabbit hole can get very deep.
- elliekelly 5y agoI'm a banking attorney and I think you've hit the nail on the head. Vendor management is popular with regulators, particularly with respect to info sec and data privacy, and the navigating the contractual responsibilities can quickly become burdensome. Getting the lawyers on both sides to agree to language for the ongoing certification(s) that will also satisfy the internal auditors, the outside auditors, the regulators... suddenly something that sounded simple when the contract was signed takes several meetings and a lot of back and forth.
- twic 5y agoDoes "on prem" ever include (a) on your own hardware, but in a third-party colo and (b) on rented hardware, where only you have root? Where do those sit between full cloud and traditional servers-in-the-basement? To look at it another way, how much of the risk is about the physical location of machines, and how much is about who operates them?
- MrBuddyCasino 5y ago> But big providers could dictate terms and conditions - as well as prices - to key financial firms. What exactly is the concern here? Cloud compute is becoming cheaper over time due to market forces. Its not like Amazon is cornering the market for CPUs.
- pmlnr 5y ago> What exactly is the concern here? Top of the article: > Concentration of compute could threaten financial stability. If BigCloud goes down, so does banking, and banking doesn't seem to like that - and I'm with them.
- cm2187 5y agoAlso changing IT in a bank is slow, dead slow. A bank wouldn’t be able to cope with being booted overnight from its infrastructure Parler-style.
- nindalf 5y agoThat FUD is unwarranted. No bank is going to be kicked off it’s infrastructure. Bringing up Parler seems completely irrelevant.
- gizdan 5y agoWhile Parler is somewhat irrelevant and the likelihood of any bank being taken down for similar reasons as Parler, it isn't completely incomprehensible. There have been many horror stories of businesses being banned, blocked, or messed around by Google and Amazon. Their policy does not protect anyone but themselves. It is within the realms of reality for bank to be taken down by them in a matter of days.
- wdb 5y agoGuess, as Bank you just need to make sure you have Amazon and Google as customer :)
- dsign 5y agoThe risk Bank of England doesn't like comes with the territory, and everybody who uses public cloud is subject to it. The only difference is that smaller companies don't have a big megaphone or the clout that they have, and must either accept the terms and shut up or find an alternate solution. If Bank of England wants to use their power constructively, they can literally go shopping for data center and IT companies to serve their needs.
- pjc50 5y ago> risk Bank of England doesn't like comes with the territory The BoE is the central bank; to a great extent they get to define the territory. Especially with regard to risk. Basel III and that kind of thing. > they can literally go shopping for data center and IT companies to serve their needs. No, it's not about what they themselves buy, it's about what all the banks in the UK buy. A situation where all the banks are using the same cloud provider(s) is unacceptable because of the high risk of correlated failure. It's bad enough when one bank goes down and its ATMs stop working; if EU-west-1 goes down and takes out all the banks, that's a disaster, and the BoE is rightly taking steps to prevent it.
- dsign 5y agoAhh, I stand corrected.
- PeterisP 5y agoWhat BoE can say is that certain risks are unacceptable for UK banks, period. It's not BoE shopping for data centers, it would be BoE setting the rules for every bank shopping for data centers. If BoE does not like some risk that really does come with the territory and everybody who uses public cloud is subject to that risk, oh well, then that would imply that UK banks have to keep out of that territory and can't use public cloud for their key services - it is a bit drastic, but certainly something that BoE has the right to rule if it really wanted. On the other hand, if there's a way to use public cloud and meet the requirements, but current major vendors are simply not offering it, then BoE can make up arbitrary requirements, followed by the major UK banks going to the major cloud vendors with a proposal "meet these requirements or we won't use your services, because we'd be prohibited to".
- kaydub 5y agoIf you're that size why not split the workload between a couple or all 3 of the big providers. Get them to work against each other. All the concerns about vendor lock-in are ridiculous right now. Prices have consistently lowered for all cloud providers.
- deleted 5y ago[deleted]
- ExposeThem 5y agoBoE wants all to use the big cloud providers, no on premise things or boutique clouds, because they want to monitor EVERYTHING.
- DrBazza 5y agoOne of the (legal ?) requirements we had to meet back in the 00s was that financial software (quant code, options trading) was "reproducible" for some length of time (7 years?). That meant: version control (SCCS, CVS, svn), archiving OS versions (SunOS, Solaris), and kit (Sun workstations and servers). I have no idea if the last 2 ("OS", hardware) is even achievable in 2021. I can always get my source code from git, and possibly the 'configuration', but can I expect to run cloudy code in 7 years time?
- WJW 5y agoI'd expect that if the cloud providers wish to acquire customers that have to comply with those regulations, the cloud providers will have clause in the contract that they are contractually obligated to keep the same services running for that period of time.
- throwaway_2047 5y agoWhat about having regulations updated instead of bending the industry backward to fit the outdated regulations?
- handrous 5y agoWhat's outdated about it?
- DrBazza 5y agoI can't testify to my statement being 100% true without a lot of tedious searching on the UK FSA/SFA/FCA/whatever they are now's website. But that's what I have a vague recollection of. It's similar to HMRC's "we can come after you for unpaid tax for 7 years", but you can only go after the HMRC for 5 years for overpaid tax. The rules are ultimately for the benefit of long term investigations such as the LIBOR rigging, and Guinness trials. https://www.theguardian.com/business/2016/jul/04/libor-rigging-scandal-three-former-barclays-traders-found-guilty https://www.theguardian.com/business/2016/jul/04/libor-riggi... https://en.wikipedia.org/wiki/Guinness_share-trading_fraud https://en.wikipedia.org/wiki/Guinness_share-trading_fraud
- 5y ago
- allyant 5y agoSeems a bit like a nothing-article - doesn't make any real point. The FCA/FSA have already been noticing a lot of the financial providers placing 'all their eggs in one basket' and have been putting guidelines in place for banks to have a multi-'cloud' strategy. [0] https://www.fca.org.uk/publication/finalised-guidance/fg16-5.pdf https://www.fca.org.uk/publication/finalised-guidance/fg16-5...
- lbriner 5y agoThat's what I thought but also, when a statement is made by a senior executive, I usually assume they don't really know what they are talking about anyway. Loads of "keynote" speeches are very hand-wavy and have a feel of importance with little of real value.
- beermonster 5y agoDoes anyone know if CSPs can be managed via supplier agreements? I’ve always assumed it’s a commodity service and as such there are standard T&Cs, TOS etc. Certainly those in the financial sector wish to manage other types of supplier risk in that way where possible.
- lbriner 5y agoIsn't that the million dollar question (or probably billion now). Theoretically, the Ts & Cs cover everything but clearly you cannot reasonably mitigate risk on the basis that the "supplier told me that it wouldn't happen". You have a (sometimes legal) requirement to do due diligence and at least decide what additional controls you can have to help with compliance and what your BC/DR process is if everything really goes south. Realistically, I don't really see AWS or Azure etc. being able to promise that their systems can never be hacked/broken. On the other hand, the assumption that doing things on-prem removes this risk is naiive since we can make just as many cock-ups even if we work for the company. Anyone ever forgotten to setup a firewall or vpn properly? True story: When I worked at a security company, the sales team wanted to demo a digital camera-over-IP system and the IT Manager told them not to use our internal network. They ignored him and plugged it in, flooding the network with packets that took down the computers and phones for about 30 minutes until we worked out what had happened. That was inconvenient but imagine if they had plugged in something much worse "because sales"
- surfingdino 5y agoOne secretive group used to dictating their own terms unhappy about another secretive group used to dictating their own terms...
- wolverine876 5y agoThe Bank of England is under the democratic control of the people of England.
- andrewzah 5y ago*In theory. However, in practice...
- lbriner 5y agoHow exactly do you figure that? Like any other quango, they are almost entirely unaccountable to anyone except their own staff. A Politician might be able to cause a stink but that's assuming they knew enough about what was going on and they probably don't.
- wolverine876 5y agoThe elected representatives of the English people could shut down the Bank tomorrow, or today.
- alasdair_ 5y ago>The elected representatives of the English people could shut down the Bank tomorrow, or today. There is a certain irony in this statement, given that such an act would require approval from the queen.
- wolverine876 5y agoLet's be serious.
- alasdair_ 5y ago
- game_the0ry 5y agoI worked in a brand-name bank, so I am familiar with the goings in the tech side of finance. Call me crazy, but I would trust aws, microsoft, and google with my PII and finances before I would trust Wells, BofA, JPMC, Goldman, et al. The cloud giants pay their engineers more and technologists are second-class citizens at the financial institutions - infer what you want from that.
- onlyrealcuzzo 5y agoMy understanding is that at least Chase, Blackrock, and Visa are rapidly raising their pay for new engineering / analyst hires. Is this not true?
- game_the0ry 5y agoI can speak for the bank I worked at - this is true, but the base pay level was very low to begin with and the increase is nowhere near competitive for the most talented engineers. Also, the experienced engineers at those companies are salty about new hires making more then they are. Again, this stems from the culture of financial institutions where tech workers are viewed as support staff, not strategically critical, despite what financial institutions say publicly.
- ZeroCool2u 5y agoI know for a fact that base pay for a first year SWE with a BS+MS in CS at Goldman just broke $100k a couple years ago. That probably sounds like a lot to most people, but if you're hiring staff to live and work in NYC, especially when there's an absolutely massive Google campus 10 minutes North of the GS HQ, that's pretty much table stakes. Never mind the glaring cultural differences.
- elliekelly 5y agoDoes that take bonuses into account? Goldman (and others in the industry) tend to have lower base pay with a much higher bonus. It's typical for low level employees to get a year-end bonus equal to about 50-60% of their salary. As they advance to mid-level their bonus will equal their salary and, if they're in a senior position, it will greatly exceed their salary.
- ChrisArchitect 5y agooriginal/source article on Reuters: https://www.reuters.com/business/retail-consumer/bank-england-crack-down-secretive-cloud-computing-services-2021-07-13/ https://www.reuters.com/business/retail-consumer/bank-englan... (https://news.ycombinator.com/item?id=27819016 https://news.ycombinator.com/item?id=27819016) any other more detailed breakdown of this somewhere?
- ChrisArchitect 5y agoHow reliant are banks and insurers on cloud outsourcing? https://www.bankofengland.co.uk/bank-overground/2020/how-reliant-are-banks-and-insurers-on-cloud-outsourcing https://www.bankofengland.co.uk/bank-overground/2020/how-rel... (January 2020)
- nobodyandproud 5y agoThis is a good conversation to have. There are no meaningful guarantees that cloud providers won’t scrape data or ideas. Then there’s the cost: Cloud is extremely expensive over the long run; more-so than the equivalent on-prem. The long term solution here is to add better Internet in all regions, similar to our highway system today. Then any company of a large enough size can build out their own DCs on the cheap. It would also inject much needed dollars into the non-coastal areas.
- nobodyandproud 5y agoSome ups and downs, but cloud’s value isn’t really for its costs.
- asaikali 5y agoI think a big risk is a cpu level security issue similar to meltdown or spectre that ends up weakening the hardware isolation between tenants to the point where it can be exploited on mass on the cloud providers to wreak havoc. The probability of something like this happening is very low but not zero, I would say same level of probability as datacenter fire or earthquake banks should be planning for how to handle this type of event.
- davidgerard 5y agoSeparate hardware for your stuff is a standard AWS product, for example - you can just buy this.
- asaikali 5y agoThat’s one way of dealing with hardware isolation risk but not every bank is doing this on public cloud.
- lbriner 5y agoI don't see that this risk is any different than a similar apocolyptic failure happening to your on-prem equipment. There's not much you can do about it differently than the cloud just add some extra controls and hope for the best. I very much doubt that anyone would not use the cloud because of a theoretical de-isolation bug. Also, by the time you found out, it would probably already be too late anyway if you were a victim. If not, you just switch it off.
- asaikali 5y agoI am not saying that they should not use cloud just that it is important to have a plan in place to deal with a unlikely but high impact security event affecting a cloud provider. Just like companies have business continuity plans in case a data center disaster they need to have plans for evacuate a cloud provider should they need too. I put on my seat belt when I drive on the highways even though a nasty crash at 120 kph would likely kill me. Not using a seat belt because you will be severely injured anyway is not wise. Given the amount of profit banks make what is the Downside of having them be resilient against public cloud failures?
- peteretep 5y agoCloud computing feels like one area we have fantastic competition in, even if there are only 3 major players (with one or two smaller ones). Feature and price competition between AWS and Azure is fierce, and if you’re dumb enough to have any trust at all in Google as a supplier then there’s even GCP.
- 908B64B197 5y agoSomething to keep in mind here: The employees of the Bank of England are ultimately government employees. They are subjected to the pay scales dictated by bureaucrats and politicians. Of course, they can't attract the caliber of engineers that works for large commercial cloud providers, so they have to settle for programmers that didn't make the cut. And the leadership is non-technical and from the financial and government worlds (you know how these "elites" see programmers and software engineers in the UK...). The existing bureaucrats running the bank probably see the Cloud as something that will reduce their headcount (why would they have sysadmins and datacenter employees on their payroll when they can simply buy it from a reliable provider), thus making them look less important to other managers. The existing unionized employees see it as a threat to their stable jobs (they are now competing with engineers that are 10x their caliber). That's a pretty bad thing for both.
- anon98356 5y agoFrom my understanding of the article, the BoE aren't concerned about their own infrastructure. They're concerned about all the major retail banks using the same cloud provider and then that provider having a major outage. Individual banks having an outage is an issue but one that can be handled. Three or four of the main banks all going down at once could be catastrophic.
- 908B64B197 5y ago> They're concerned about all the major retail banks using the same cloud provider and then that provider having a major outage. I wouldn't worry too much about that. Major outages are exceedingly rare. I can't say the same for on-prem infrastructure.
- jollybean 5y agoWe need a better 'Open Stack' for cloud stuff. So that you can just run a bunch of your own servers, lay over the 'Stack' and then get nice Lambdas, provisioning, and other things. Imagine if you could just buy some hardware, and have some regular IT guys reproduce most of Amazon without the Amazon? That would be a 'reverse revolution'. National Regulators could also support strategic investment in the sector, i.e. 'financial ops have to be backed-up in-nation by a local provider' i.e. some kind of forced local diversity in the system by regulatory fiat. As an idea.