9 ms·

Monzo Stand-In

130 points by coffeefuel 1mo ago

qmarchi 1mo ago
Something like this has been top-of-mind for us since we have a heavy reliance on AWS, which was made evident with the VPC Origin failure earlier this year. Multi-cloud is best cloud.
sdcfgy 1mo ago
Due to the recent outage where it didn’t work properly, this should be renamed Monzo Limp-In.
- lozenge 1mo ago
  Monzo I've Fallen And I Can't Get Up
- nunez 1mo ago
  Simple Bank, when they were a thing, published this "look at how awesome we are" post about moving to Riak from Cassandra. (I think that's what it was; it's been a long time.) "We're going to be so reliable after this; big banks don't stand a chance" was what I took away from it. The migration went horribly. Tier 0 clusterfuck. Debit cards were completely unusable and they lost ledger data. That and the occasional duplicate transaction (like the duplicated student loan payment that put me in a really bad spot at the time, which I was only given $50 for the inconvenience) was enough for me to _run_ bank to Chase. So, yeah, I get why the big banks are totally fine with keeping their mostly-rock-solid core banking functions on the mainframe and stick with datacenters everywhere.
  - sdcfgy 1mo ago
    100% why I bank with an old bank that runs mainframes.
guiriduro 1mo ago
By using another US hyperscaler for your backup platform you're still unhedged against US governance failure/breakdown.
- sdcfgy 1mo ago
  An increasing risk for sure.
- lmz 1mo ago
  Surely by being a bank, you are already exposed to that.
  - guiriduro 1mo ago
    Exposed yes, but not fatally if you're working with alternative de-dollarised banking systems in the eurozone, for example.
    - lmz 1mo ago
      Won't help if they sanction you and cut you off from SWIFT (or the card networks) so it just depends on the degree of the governance issues here.
      - guiriduro 1mo ago
        Most european banking entities already have and use SEPA.
        lmz 1mo ago
        Any alternative non US, Europe wide card network? I suppose you could use Mir, or UnionPay.
        Abstract_Typist 29d ago
        This is an issue, but one Europe is acutely aware of and working on. WERO is one strand of that. https://www.reneweuropegroup.eu/news/2026-06-23/independence-from-us-payment-systems-todays-vote-on-the-digital-euro https://www.reneweuropegroup.eu/news/2026-06-23/independence... https://europeanbusinessmagazine.com/eu-payments-sovereignty-why-europe-seeks-independence-from-visa-and-mastercard/ https://europeanbusinessmagazine.com/eu-payments-sovereignty... https://stripe.com/en-fr/resources/more/wero-how-europes-unified-digital-wallet-is-changing-payments https://stripe.com/en-fr/resources/more/wero-how-europes-uni...
ascorbic 1mo ago
[2025]
danpalmer 1mo ago
I can't help but think this isn't the best idea, it raises so many questions. How does fail over work (it failed for me!), can you safely fail over, how does product think about the stack, is it funded well enough, how do you scale a system from zero to all traffic instantly, how do you effectively keep this environment warm, how do you determine which services are critical enough to be in it, how do you manage finding the bugs in two separate stacks, and so on and on and on. It seems like it would be better to invest in a platform that is distributed across multiple failure domains. Some companies call this a cell architecture. For Monzo this might mean true multi cloud, but they already have bare metal services for bank integrations so they are already hybrid in some respects. It's not the decision I would have made, and I'm not sure I agree with their reasoning, but I'll also admit that I have not run a bank, I don't know all their constraints, and there are some good tech folks at Monzo. That said, I am an SRE at Google so I'd like to think I'm not completely naïve here. Edit: after reading in more detail I'm puzzled. The justification for Stand-In and not traditional scale-out is the difficulty of data consistency (agreed!), but then Stand-In is just eventually consistent anyway, and introduces sync problems in both directions. Admittedly I'm spoilt by Spanner, but I do wonder if Monzo is being bitten by past architecture choices here.
- ricardobayes 1mo ago
  No, it's a great idea. Any systems engineer knows redundancy is good, even better if it's across providers. I see no issues with this from a system architecture point of view. As to how the implementation is done that's up to them to do right.
  - danpalmer 1mo ago
    Redundancy is great, absolutely, but there are multiple ways to do it and this is not necessarily the right one. Which is better: 3 DCs for 2 DCs worth of traffic, running the same stack but isolated failure domains... or 4 DCs for 2 DCs worth of traffic, that work in completely different ways, where 2 are left cold until needed. I can see some advantages of the latter, but not a lot. This also isn't about fault tolerance at the code level like in, say, avionics. In those scenarios having multiple distinct implementations makes sense, but that's very different, you run all paths at the same time and check they agree etc.
IshKebab 1mo ago
Monzo is great. Rare example of a consumer banking company that makes things easy. I just signed up for their phone service in literally 3 minutes. Didn't even need to talk to anyone to port my old number. I'm going to move all my ISAs and SIPP to them next I think.
- danpalmer 1mo ago
  > Didn't even need to talk to anyone to port my old number That seems like pretty low stakes – I wouldn't expect to talk to anyone to sign up and transfer to a consumer service in the UK. I don't think I've spoken to anyone for something like this in the last 10 years or so.
  - OJFord 1mo ago
    I've never talked to someone for it, and I would've first done it in something like 2008. (It's 'always' been done by SMS, well before 'apps' that might let you (and I don't recall it on websites).)
  - IshKebab 1mo ago
    Well I haven't moved my number for a couple of decades at least, but the last time I did you had to phone them and go through the whole "are you sure? Let me see if I can get you a better deal" nonsense.
    - danpalmer 1mo ago
      Cancellation is different to porting a number. You'll still need to cancel your previous plan, but legislation fixed carriers holding numbers hostage a while ago.
      - IshKebab 1mo ago
        No, when you port the number it automatically cancels the service.
- broadben12 1mo ago
varispeed 1mo ago
> Monzo Stand-in is an independent set of systems that run on Google Cloud Platform (GCP) Given the US Cloud Act, how Monzo is protecting customers' financial data from being accessed by foreign government without Monzo's knowledge? Everyone does it, does not excuse it. Also shows lack of faith in British business.
- danpalmer 1mo ago
  Which British businesses provide this sort of infrastructure? I ask this as a Brit who ran stuff on a few UK providers, and ultimately moved to a cloud platform because of the combination of terrible service and lack of features. The landscape may have changed, but in my experience it was bleak circa 2020.
  - varispeed 1mo ago
    [dead]
  - danpalmer 29d ago
    There's a now dead comment giving a few options. One of those options was the main driver for my previous company moving fully to GCP, they were atrocious and caused a major security incident for us. They also regularly took days to action urgent fixes. I'm not saying the UK doesn't have data centres, but it's easy to do a crap job of running one. In my experience there's a level of competence that is missing from UK operations. Hetzner and OVH are noticeably better, but still not quite at the level of the main cloud providers.
greatgib 1mo ago
There are 3 things I have in mind when reading this: - it should be so frustrating for a dev to work on a full core banking that is mostly useless and userless except maybe 1h every 3 years. - if you build a totally different implementation to handle high traffic transaction from third parties that don't operate for days or months, i would expect it to be fragile. Like for example suddenly you receive unicode special chars that you didn't expect in labels and boom. - last, I think that in most incident cases, the situation is not clear enough to be able to switch to the full different system. Rarely it will be your whole infra but part of it, so there is always the question to what to do in such a case, or if your main infra goes crazy with loops for example, or if you lose control because of a network, aws, infrastructure issue thr problem might be that you have no visibility on what is working and what is not, not necessarily the control to stop the faulty services, etc...
- rjh29 1mo ago
  "it should be so frustrating for a dev to work on a full core banking that is mostly useless and userless except maybe 1h every 3 years." When other banks go down and Monzo stays up, even for 1 hour, that is a massive PR win and will bring in new customers. It's also been used multiple times as of the article (2025), and they are always exposing some users to it for testing.
  - greatgib 1mo ago
    I don't say that it is not useful, and that it is not a good thing for the company. More than I would be very frustrated to spend my life on that if it was my job.
    - Imustaskforhelp 1mo ago
      Offtopic but I remembered one HN comment about where someone was working for an internal app for a company for the fast food/ice-cream or food app in general (or maybe it was for the contractor) I think some people found that job really peaceful when others might've found it more frustrating. To me, I think though the issue isn't the scale or even the internal part of it but rather in this case, there's a difference because its being used infrequently (1 hour) as compared to that other HN comment was still being used quite frequently (it was used every day) Though, I must admit, I don't find it too frustrating. I can be wrong I usually am but, Sysadmin/DevOps/Security engineer does the work as well where sometimes 1 hour can matter much more than months and they do work for months so that they don't get to face that 1 hour issue just because of how devastating it can get, Though Sysadmins do a lot of regular work as well. There are many jobs where they exist for reasons where the employee exists to handle the bad things. You would wish for the bad thing to not happen and the employee might feel like they might not be doing much if the bad thing doesn't happen, BUT when that bad thing happens, You would be happy that they would then be there for you. Maybe not a direct 1:1 comparison but hopefully I can express my point. I feel like there are many jobs in tech, or maybe in the world in general which fall into a similar fashion?
Valodim 1mo ago
Almost seems like they can do anything at monzo.com
- stavros 1mo ago
  The only limit is themselves.
janpeuker 1mo ago
Minor thing but I do like they use the word "Stand In" here in the payments / financial sense of Stand-in processing (STIP), which is usually used when a bank is offline to keep essential services such as public transport working.
tarun_anand 1mo ago
I think this is what distributed systems has a phrase for: "graceful degradation".
jmkni 1mo ago
Just pointing out this is from February 2025, so has been a thing for a while
syou1024 1mo ago
[flagged]
FreezingKeeper 1mo ago
Monzo is nothing more than a science project
just60sec 29d ago
[flagged]
nancysahu 28d ago
[flagged]

130 points by coffeefuel 1mo ago

Something like this has been top-of-mind for us since we have a heavy reliance on AWS, which was made evident with the VPC Origin failure earlier this year. Multi-cloud is best cloud.

sdcfgy 1mo ago

Due to the recent outage where it didn’t work properly, this should be renamed Monzo Limp-In.

lozenge 1mo ago
Monzo I've Fallen And I Can't Get Up
nunez 1mo ago
Simple Bank, when they were a thing, published this "look at how awesome we are" post about moving to Riak from Cassandra. (I think that's what it was; it's been a long time.) "We're going to be so reliable after this; big banks don't stand a chance" was what I took away from it. The migration went horribly. Tier 0 clusterfuck. Debit cards were completely unusable and they lost ledger data. That and the occasional duplicate transaction (like the duplicated student loan payment that put me in a really bad spot at the time, which I was only given $50 for the inconvenience) was enough for me to _run_ bank to Chase. So, yeah, I get why the big banks are totally fine with keeping their mostly-rock-solid core banking functions on the mainframe and stick with datacenters everywhere.
- sdcfgy 1mo ago
  100% why I bank with an old bank that runs mainframes.

guiriduro 1mo ago

By using another US hyperscaler for your backup platform you're still unhedged against US governance failure/breakdown.

sdcfgy 1mo ago
An increasing risk for sure.
lmz 1mo ago
Surely by being a bank, you are already exposed to that.
- guiriduro 1mo ago
  Exposed yes, but not fatally if you're working with alternative de-dollarised banking systems in the eurozone, for example.
  - lmz 1mo ago
    Won't help if they sanction you and cut you off from SWIFT (or the card networks) so it just depends on the degree of the governance issues here.
    - guiriduro 1mo ago
      Most european banking entities already have and use SEPA.
      - lmz 1mo ago
        Any alternative non US, Europe wide card network? I suppose you could use Mir, or UnionPay.
        Abstract_Typist 29d ago
        This is an issue, but one Europe is acutely aware of and working on. WERO is one strand of that. https://www.reneweuropegroup.eu/news/2026-06-23/independence-from-us-payment-systems-todays-vote-on-the-digital-euro https://www.reneweuropegroup.eu/news/2026-06-23/independence... https://europeanbusinessmagazine.com/eu-payments-sovereignty-why-europe-seeks-independence-from-visa-and-mastercard/ https://europeanbusinessmagazine.com/eu-payments-sovereignty... https://stripe.com/en-fr/resources/more/wero-how-europes-unified-digital-wallet-is-changing-payments https://stripe.com/en-fr/resources/more/wero-how-europes-uni...

ascorbic 1mo ago

[2025]

danpalmer 1mo ago

I can't help but think this isn't the best idea, it raises so many questions. How does fail over work (it failed for me!), can you safely fail over, how does product think about the stack, is it funded well enough, how do you scale a system from zero to all traffic instantly, how do you effectively keep this environment warm, how do you determine which services are critical enough to be in it, how do you manage finding the bugs in two separate stacks, and so on and on and on. It seems like it would be better to invest in a platform that is distributed across multiple failure domains. Some companies call this a cell architecture. For Monzo this might mean true multi cloud, but they already have bare metal services for bank integrations so they are already hybrid in some respects. It's not the decision I would have made, and I'm not sure I agree with their reasoning, but I'll also admit that I have not run a bank, I don't know all their constraints, and there are some good tech folks at Monzo. That said, I am an SRE at Google so I'd like to think I'm not completely naïve here. Edit: after reading in more detail I'm puzzled. The justification for Stand-In and not traditional scale-out is the difficulty of data consistency (agreed!), but then Stand-In is just eventually consistent anyway, and introduces sync problems in both directions. Admittedly I'm spoilt by Spanner, but I do wonder if Monzo is being bitten by past architecture choices here.

ricardobayes 1mo ago
No, it's a great idea. Any systems engineer knows redundancy is good, even better if it's across providers. I see no issues with this from a system architecture point of view. As to how the implementation is done that's up to them to do right.
- danpalmer 1mo ago
  Redundancy is great, absolutely, but there are multiple ways to do it and this is not necessarily the right one. Which is better: 3 DCs for 2 DCs worth of traffic, running the same stack but isolated failure domains... or 4 DCs for 2 DCs worth of traffic, that work in completely different ways, where 2 are left cold until needed. I can see some advantages of the latter, but not a lot. This also isn't about fault tolerance at the code level like in, say, avionics. In those scenarios having multiple distinct implementations makes sense, but that's very different, you run all paths at the same time and check they agree etc.

IshKebab 1mo ago

Monzo is great. Rare example of a consumer banking company that makes things easy. I just signed up for their phone service in literally 3 minutes. Didn't even need to talk to anyone to port my old number. I'm going to move all my ISAs and SIPP to them next I think.

danpalmer 1mo ago
> Didn't even need to talk to anyone to port my old number That seems like pretty low stakes – I wouldn't expect to talk to anyone to sign up and transfer to a consumer service in the UK. I don't think I've spoken to anyone for something like this in the last 10 years or so.
- OJFord 1mo ago
  I've never talked to someone for it, and I would've first done it in something like 2008. (It's 'always' been done by SMS, well before 'apps' that might let you (and I don't recall it on websites).)
- IshKebab 1mo ago
  Well I haven't moved my number for a couple of decades at least, but the last time I did you had to phone them and go through the whole "are you sure? Let me see if I can get you a better deal" nonsense.
  - danpalmer 1mo ago
    Cancellation is different to porting a number. You'll still need to cancel your previous plan, but legislation fixed carriers holding numbers hostage a while ago.
    - IshKebab 1mo ago
      No, when you port the number it automatically cancels the service.
broadben12 1mo ago

varispeed 1mo ago

> Monzo Stand-in is an independent set of systems that run on Google Cloud Platform (GCP) Given the US Cloud Act, how Monzo is protecting customers' financial data from being accessed by foreign government without Monzo's knowledge? Everyone does it, does not excuse it. Also shows lack of faith in British business.

danpalmer 1mo ago
Which British businesses provide this sort of infrastructure? I ask this as a Brit who ran stuff on a few UK providers, and ultimately moved to a cloud platform because of the combination of terrible service and lack of features. The landscape may have changed, but in my experience it was bleak circa 2020.
- varispeed 1mo ago
  [dead]
- danpalmer 29d ago
  There's a now dead comment giving a few options. One of those options was the main driver for my previous company moving fully to GCP, they were atrocious and caused a major security incident for us. They also regularly took days to action urgent fixes. I'm not saying the UK doesn't have data centres, but it's easy to do a crap job of running one. In my experience there's a level of competence that is missing from UK operations. Hetzner and OVH are noticeably better, but still not quite at the level of the main cloud providers.

greatgib 1mo ago

There are 3 things I have in mind when reading this: - it should be so frustrating for a dev to work on a full core banking that is mostly useless and userless except maybe 1h every 3 years. - if you build a totally different implementation to handle high traffic transaction from third parties that don't operate for days or months, i would expect it to be fragile. Like for example suddenly you receive unicode special chars that you didn't expect in labels and boom. - last, I think that in most incident cases, the situation is not clear enough to be able to switch to the full different system. Rarely it will be your whole infra but part of it, so there is always the question to what to do in such a case, or if your main infra goes crazy with loops for example, or if you lose control because of a network, aws, infrastructure issue thr problem might be that you have no visibility on what is working and what is not, not necessarily the control to stop the faulty services, etc...

rjh29 1mo ago
"it should be so frustrating for a dev to work on a full core banking that is mostly useless and userless except maybe 1h every 3 years." When other banks go down and Monzo stays up, even for 1 hour, that is a massive PR win and will bring in new customers. It's also been used multiple times as of the article (2025), and they are always exposing some users to it for testing.
- greatgib 1mo ago
  I don't say that it is not useful, and that it is not a good thing for the company. More than I would be very frustrated to spend my life on that if it was my job.
  - Imustaskforhelp 1mo ago
    Offtopic but I remembered one HN comment about where someone was working for an internal app for a company for the fast food/ice-cream or food app in general (or maybe it was for the contractor) I think some people found that job really peaceful when others might've found it more frustrating. To me, I think though the issue isn't the scale or even the internal part of it but rather in this case, there's a difference because its being used infrequently (1 hour) as compared to that other HN comment was still being used quite frequently (it was used every day) Though, I must admit, I don't find it too frustrating. I can be wrong I usually am but, Sysadmin/DevOps/Security engineer does the work as well where sometimes 1 hour can matter much more than months and they do work for months so that they don't get to face that 1 hour issue just because of how devastating it can get, Though Sysadmins do a lot of regular work as well. There are many jobs where they exist for reasons where the employee exists to handle the bad things. You would wish for the bad thing to not happen and the employee might feel like they might not be doing much if the bad thing doesn't happen, BUT when that bad thing happens, You would be happy that they would then be there for you. Maybe not a direct 1:1 comparison but hopefully I can express my point. I feel like there are many jobs in tech, or maybe in the world in general which fall into a similar fashion?

Valodim 1mo ago

Almost seems like they can do anything at monzo.com

stavros 1mo ago
The only limit is themselves.

janpeuker 1mo ago

Minor thing but I do like they use the word "Stand In" here in the payments / financial sense of Stand-in processing (STIP), which is usually used when a bank is offline to keep essential services such as public transport working.

tarun_anand 1mo ago

I think this is what distributed systems has a phrase for: "graceful degradation".

jmkni 1mo ago

Just pointing out this is from February 2025, so has been a thing for a while

syou1024 1mo ago

[flagged]

FreezingKeeper 1mo ago

Monzo is nothing more than a science project

just60sec 29d ago

[flagged]

nancysahu 28d ago

[flagged]