40 ms·

> Raj thought that checking the "principal" checkbox and entering the number of a Citibank wash account would ensure that the principal payment would stay at Ci

> Raj thought that checking the "principal" checkbox and entering the number of a Citibank wash account would ensure that the principal payment would stay at Citibank. He was wrong. To prevent payment of the principal, Raj actually needed to set the "front" and "fund" fields to the wash account as well as "principal." Raj didn't do that. I don't even have anything to add. The paragraph speaks for itself... You can't make this up

gzer0 6y ago
If you read literally the next few lines: > Citibank's procedures require that three people sign off on a transaction of this size. In this case, that was Raj, a colleague of his in India, and a senior Citibank official in Delaware named Vincent Fratta. All three believed that setting the "principal" field to an internal wash account number would prevent payment of the principal. As he approved the transaction, Fratta wrote: "looks good, please proceed. Principal is going to wash."
- airstrike 6y ago
  I'm not sure that changes anything. The human error here is in the design, not in the execution.
  - hjek 6y ago
    It speaks even worse of that interface. It means that the design was not only bad enough to confuse a single individual but also the two others meant to act as safety mechanism.
- yrgulation 6y ago
  Unrelated, but is it a good idea make the names of those employees public? Wondering how this will end up affecting their lives, when in fact it was caused by a UI issue and lack of proper training on how to use it.
- m12k 6y ago
  Thing is, even with a UI as shitty as this, you would still have caught this if before actually committing the transaction, the software had a "before" and "after" view to show the effect of the change. Raj caught the error during a routine inspection the next morning, so the same view he used for that, presented before the change had been committed, would have allowed him to avoid the error.
  - pdpi 6y ago
    That would, by itself, make the UI less shitty. Also, if you're the sort of person who thinks in those terms, you're also likely the sort of person who'd have come up with a better design overall anyway.
    - dylan604 6y ago
      This sounds like one of those situations where devs tasked with programming something that they have little to no understanding of what they are programming does. Yes, the dev programmed button A to perform action B, but doesn't truly understand how/why/what/when/where action B. It's just a check box of the requirements that has now been completed. However, a dev that does understand action B would be able to anticipate various forms of input from button A and how that could cause undesired results from action B. This isn't a knock on the dev that is less knowledgable about the full thing. It is a knock on the people upstream that should be more aware of issues. Also, the people most knowledgeable about action B tend to have little to no understanding of programming. While they might understand that providing bogus data to action B can have bad results, they are not aware enough to tell the devs that sometimes valid looking data is just not sane.
      - acdha 6y ago
        I wouldn’t even assume lack of understanding as much as being locked into a work-to-order situation where they’re not rewarded, possibly punished, for doing anything which isn’t on the assignment. This is super common in outsourced hellscapes where the MBAs like to LARP as architects and UI designers and there’s no effective feedback loop from the actual users.
- mumblemumble 6y ago
  There's an old(ish) episode of the Stack Overflow podcast where they interview a couple folks from the site reliability engineering team. I think this is the one: https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasionally-maintaining-infrastructure-hate/ https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasi... Somewhere in there, one of the SREs said something that's stuck with me. Roughly, he said that the concept of human error is useless. Every time a human error escalates into a real problem, the deeper problem is a design flaw. At scale, human errors are a certainty, and, while documentation and training is far from useless, no amount of it can change that fact. It's therefore a critical responsibility of systems designers to design their systems to be resilient to mistakes.
  - mybandisbetter 6y ago
    This is also a big concept in the medical world. There's an influential report on this from the 90s called "To Err is Human." https://pubmed.ncbi.nlm.nih.gov/25077248/ https://pubmed.ncbi.nlm.nih.gov/25077248/
  - capableweb 6y ago
    This is also not a new idea that even comes from the software world. I'm not entirely sure of the original of such ideas, but the first time I heard about was in conjunction with airplanes and their UX. Basically, crashes and other issues from flying is never attributed to the operator, but the UX of the controls as if it was done better, would have prevented the operator of committing the error in the first place.
    - CPLX 6y ago
      Indeed. One of the really core things you learn when you dig deeper into aviation is that essentially every major disaster is a chained combination of multiple extremely unlikely failures in sequence. Which is a testament to both the incredible success this approach has had, as well as the impossibility of perfection.
      - Gare 6y ago
- goatcode 6y ago
  It looks like they were all under-qualified. I wonder how much the decision to hire them all will end up costing anyone other than them, and the people who hired them.
  - read_if_gay_ 6y ago
    No. It looks like the people who built the software were underqualified.
    - goatcode 6y ago
      Maybe I misunderstood, but all of them either "thought" that was the right thing to do or signed off on it. Even if the software should have warned them, shouldn't they have known the correct transactions to make?
      - CydeWeys 6y ago
        They knew the correct transactions to make; the problem is the software was unbelievably counter-intuitive. At some point the user can't be blamed if the UX design is that bad.
      - kemotep 6y ago
        That's issue here with this specific software's UX. They all thought they were making the right transaction. There was no warning that they were not. It would be like if I hit the reply button here on Hacker News and instead of the comment I typed out it my reply contained my bank account information. And I was not told something I do not want to happen would happen by doing so.
        goatcode 6y ago
        I think I understand now: the software was doing the wrong thing. They thought they were making one transaction, but they made a different one. One wonders how long that had been going on for, and how it could have been going on for so long. Thanks for the explanation. It saddens me how brutal people here are in response to a misunderstanding, judging by the downvotes, but oh well.
koheripbal 6y ago
To me this speaks more to the need for major money transfers to have more programmatic checks in place as well aa 2nd person signoffs.
- toomuchtodo 6y ago
  The entire story is the cost of not recognizing you're an engineering firm that handles money, versus a bank that bolts on IT at the lowest cost possible. Citi learned the hard way, which is good, as it's the only way for such an org to learn.
  - DoofusOfDeath 6y ago
    > Citi learned the hard way Do we have evidence that they actually learned anything from this?
    - deleted 6y ago
      [deleted]
    - m12k 6y ago
      No, we just know that they paid for a lesson
    - dvfjsdhgfv 6y ago
      In all my previous companies whenever we earned more than expected from a given project, we'd routinely analyze it and it was our job to make sure we can somehow repeat or benefit from this lesson for the future. In the same way, whenever there was a significant drop, it was our job to make sure we know the reasons behind it and implement preventive measures if applicable. The only factor that changed was the level of 'why?'s asked. I can't imagine it's any different in Citibank.
      - DoofusOfDeath 6y ago
        > I can't imagine it's any different in Citibank. Having worked for companies the size of small startups up through megacorps, my experience has been that megacorps can be unfathomably foolish in these kinds of things. I'd expect Citibank to be less likely to learn (and apply) the right lessons from this experience, than a small or medium-sized business would.
Shivetya 6y ago
as many pointed out, it still isn't uncommon to repurpose or "Extend" and existing UI to do stuff it was never meant to do and then never go back and clean up the efforts when such use is common place. resulting in issues like Citi got. Seriously who thought that UI, requirements, text, and all, was a long term solution for handling any amount of money?
dheera 6y ago
I've used Citizens Bank before for business reasons and their UI is probably one of the worst I have ever seen. I don't understand why consumers get such good stuff and enterprises get such crap.
- MattGaiser 6y ago
  Do enterprises change banks because of bad UI/UX? There's your answer.
  - joshgoldman 6y ago
    Did you even read the article?
    - MattGaiser 6y ago
      Did you read the context of my reply? I know the article is about bad internal UI/UX. But my understanding of what the person I am replying to said was that businesses have to put up with terrible UI/UX for their banking portals. Which I anecdotally know to be true, at least with Canadian banks. The business banking portal is often much worse than the consumer one.
    - jacurtis 6y ago
      Did you even read his question? Nowhere in the article (which I read in its entirety) does it answer the above reply's question. "Do companies consider good UI a competitive advantage when looking for a commercial bank or lender?" I don't know. But that's not addressed in the article. My guess is that it probably isn't a huge consideration. Very few people interact with these interfaces. Maybe one or two controllers at a company would ever log into this. Even CEOs and top management would never actually log into these accounts directly. They are going to get their information from internal reports and BI dashboards. So I bet no one really cares. If Citi offers a $400M loan at 1.1% interest and JP Morgan offers the same loan at 1.3% interest, my guess is that that company is going with Citi. I doubt UI/UX makes a difference here. Although in the consumer world it does have an impact which is why we have seen many banks improving their apps and websites lately to compete in the consumer banking space.
samizdis 6y ago
Ha! For a few seconds after reading your comment I thought I was experiencing déjà vu, but ... https://news.ycombinator.com/item?id=26170694 https://news.ycombinator.com/item?id=26170694 :-)
- airstrike 6y ago
  Oh no, I've been found out!
- Debug_Overload 6y ago
  You can tell it's very insightful because OP felt the need to repost it so the wisdom doesn't go to waste.
jacurtis 6y ago
It is also worth noting that Raj wasn't alone in the confusion on how to perform this task. Before he could complete the transfer his boss in India also had to look at what Raj did. Then their boss in Deleware also approved the transaction. ALL THREE PEOPLE INVOLVED IN THIS TASK WERE CONFUSED by the interface and all three thought that this was the correct way to do this. This wasn't the case of one person checked the wrong box. Everyone who looked at this was confused. This problem was not individual, but systemic.
- TuringNYC 6y ago
  Looking at the UI, anyone would be confused. I feel bad for the employee put into the spotlight. The spotlight should be on whomever designed and approved this software and UI. Unfortunately because of the court case, instead, people associated are the users.
- crispyambulance 6y ago
  Totally NOT SURPRISED that this was an awful Oracle enterprise app. These things were put together in haste in the 90's and Oracle keeps pushing them and their corporate customers keep these things going long past when the interface becomes utterly unfamiliar. The blame has to be shared, however, with the battle-axe personalities of the people that use this stuff. They would like nothing more than to retire while still using the _exact_ same applications and dated interfaces they've used for 20+ years. Literally, these are default "grey-theme" java applications, with all kinds of inconsistent UI quirks and absolutely no shits given for discoverability or even being able to create hyperlinks so that people can share stuff without screenshots. It's fine if it's the same people keep using these apps forever, but the problems start when new people get on-boarded. If it's like most places, the newbies will be given no training and the only help they will get would be from some dour in-house oracle-application jockey whose been doing the same thing for decades. That's when people make very, very expensive mistakes. We hear about this one because the sum was vast, but people make $10,000 mistakes on shit software from Oracle all the time-- that's partly why Oracle has a very deep bench of retained lawyers, I suppose. I have to say Citibank deserves it.
  - 6y ago
cosmodisk 6y ago
You look at the screenshot,then read the summary on what happened and then wonder how on earth the bank still has any money left on their accounts with systems like this.
- rowland66 6y ago
  Looking at the screen tells you nothing about the quality of this system. The screens look a bit dated because the system is 20 years old. It is also a back office system used by a small number of what should be trained professionals. Pretty html and javascript would not have helped. This was a failure of the whole system. The software and the organization.
  - cosmodisk 6y ago
    I didn't mean that it looks ugly- I take functionality over pretty design any time. The screenshot itself doesn't provide much info,but when combined with comment from the article,the who picture is pretty bleak tbh..
  - murermader 6y ago
    This is not about ugly vs beautiful. Good UI is not primarily about being nice to look at, it is to be usable. The UI has to be really bad to be able to screw up this bad without even knowing it. Horrendously bad.
redshirtrob 6y ago
These issues are everywhere if you bother to notice. My in-laws had a coffee maker with a built-in grinder. The grind function was default enabled, so if you went to brew some coffee from pre-ground beans and just turned it on, the grinder would kick on with an awful sound. The solution they came up with was a "Grinder Off" button that lit up to indicate the grinder was turned off. It was...confusing to say the least. And as an infrequent user of this coffee maker it was all too easy to screw up. To add insult to injury, that off light turned off (i.e. enabled grinding) every time you brewed a new pot. You had to be sure to press it every time you brewed from pre-ground beans, which was always for us.
- CydeWeys 6y ago
  To be honest, that setup would make more sense for me; I always brew from freshly ground whole beans. It sounds like they flat out bought the wrong coffee maker. They should have gotten one that doesn't come with an integrated grinder at all. That coffee maker is well designed for its intended happy path use case, which is not the same as you can say for this Oracle software.
  - redshirtrob 6y ago
    No doubt it was the wrong coffee maker for them. I don't think it was well designed though. I think it was "eh, ok" designed for the happy path, and very poor for the unhappy (but probably pretty normal) path. I think what threw me is the "grinder off" button with a red light. The red light made sense in the context. But, if I were designing this, I would have a "grinder" button with a green light that was default "on", indicating that grinding was imminent. There was no green light on this part. It was red or off. That would have been consistent with the rest of the interface, where a green light on the "brew" dial meant the timer was set and a red light meant the coffee had brewed.
    - wging 6y ago
      In this situation, a physical toggle switch seems like a good idea. Its state is always inspectable and it stays the way you left it.
1024core 6y ago
Let's not blame the lowly sub here. The next paragraph continues: Citibank's procedures require that three people sign off on a transaction of this size. In this case, that was Raj, a colleague of his in India, and a senior Citibank official in Delaware named Vincent Fratta. All three believed that setting the "principal" field to an internal wash account number would prevent payment of the principal. As he approved the transaction, Fratta wrote: "looks good, please proceed. Principal is going to wash."
SpaceManNabs 6y ago
And Raj will still probably get fired for jointly discovering this vulnerability.
- rchaud 6y ago
  Hopefully he just gets moved to a different project. At least according to the story and what was discussed in court, he was not singled out for blame.
  - chris_wot 6y ago
    His name is now in a court filing AND published on Ars Technica. He's toast.
woobar 6y ago
Not defending this horrible UI, but this is a case of a non-standard operation and one should be extra careful when trying to override default behavior. Matt Levine has more details.[1] If I am an approver of a $900M transfer using an edge case I will follow instructions: "Citibank’s internal Fund Sighting Manual provides instructions for suppressing Flexcube’s default. When entering a payment, the employee is presented with a menu with several “boxes” that can be “checked” along with an associated field in which an account number can be input. The Fund Sighting Manual explains that, in order to suppress payment of a principal amount, “ALL of the below field[s] must be set to the wash account: FRONT[;] FUND[; and] PRINCIPAL” — meaning that the employee had to check all three of those boxes and input the wash account number into the relevant fields." [1] [1] https://www.bloomberg.com/opinion/articles/2021-02-17/citi-can-t-have-its-900-million-back https://www.bloomberg.com/opinion/articles/2021-02-17/citi-c...
kables 6y ago
Interesting pattern: 787 boeing crash, Citibank 500m confused transaction, are outsourced to the same place, India.
- dang 6y ago
  Please keep nationalistic flamewar off HN. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
w23j 6y ago
Well one could add, that other possible options are "COLLAT", "COMPINTSF", "DEFAUL" and "DFLFTC". Hard to believe that anyone could possibly be confused by that or miss the clearly necessary "FRONT" and "FUND".
fortran77 6y ago
I also wonder what the action of setting an account number in the "PRINCIPAL" field does if the system basically ignores it unless other fields are set. Why is this configuration even possible?
joejerryronnie 6y ago
I can understand if you’re venmoing your buddy $20, but if your are transferring millions of dollars, and faced with that horrendous interface, you might think to consult the standard operating procedures.
JJMcJ 6y ago
So there wasn't a final screen that said something like You are about to send $83,932,198.83 to (various account information fields) Unlike, say, a site that sells hand knit mittens or maybe organic kibble.
- JumpCrisscross 6y ago
  No, but it did ask for the email address twice.
dopidop 6y ago
I work on a large financial transaction system. A dumb backbone for bank the same size as citi. This UI is pretty good ! Haha. No seriously today we had a one hour conversation on what happen when a user cancel a trade twice. I don’t think we’re gonna find a elegant solution within the contraints of the clusterfuck of sub-system we maintaining. Kinda disheartening.
Igelau 6y ago
Ars Technica must have redacted the name. It just says "subcontractor" now.
DrBazza 6y ago
Having previously worked at "Megabank", this does not surprise me at all. Conway's Law applies here. It's a hugely dysfunctional bank, saddled with bureaucracy, fragile software (company change control ran on a IBM mainframe as late as 2015, requiring IE6), teams that are 'silo-ed', and talking with other teams are often the equivalent of a divorce lawyer's communication between spouses. Sandy Weill's crowning glory of mashing together Salomon Brothers, Smith Barney, Primerica, Travellers, Citicorp, and Diner's Club.
bluishgreen 6y ago
Amusing and mildly racist use of the name Raj in the comment, not from the original article as far as I can see. Please note that John of Delaware, his boss confirmed that principal goes to wash and said to “please proceed”
- Igelau 6y ago
  > Amusing and mildly racist use of the name Raj Interesting. My gut reaction was that they must have initially published his name and silently redacted it. Your gut reaction is that HN is seething with many racist commenters who use R__ as an epithet for any Indian contractor. Wayback Machine if you haven't already figured out which one of us is right.

gzer0 6y ago

If you read literally the next few lines: > Citibank's procedures require that three people sign off on a transaction of this size. In this case, that was Raj, a colleague of his in India, and a senior Citibank official in Delaware named Vincent Fratta. All three believed that setting the "principal" field to an internal wash account number would prevent payment of the principal. As he approved the transaction, Fratta wrote: "looks good, please proceed. Principal is going to wash."

airstrike 6y ago
I'm not sure that changes anything. The human error here is in the design, not in the execution.
- hjek 6y ago
  It speaks even worse of that interface. It means that the design was not only bad enough to confuse a single individual but also the two others meant to act as safety mechanism.
yrgulation 6y ago
Unrelated, but is it a good idea make the names of those employees public? Wondering how this will end up affecting their lives, when in fact it was caused by a UI issue and lack of proper training on how to use it.
m12k 6y ago
Thing is, even with a UI as shitty as this, you would still have caught this if before actually committing the transaction, the software had a "before" and "after" view to show the effect of the change. Raj caught the error during a routine inspection the next morning, so the same view he used for that, presented before the change had been committed, would have allowed him to avoid the error.
- pdpi 6y ago
  That would, by itself, make the UI less shitty. Also, if you're the sort of person who thinks in those terms, you're also likely the sort of person who'd have come up with a better design overall anyway.
  - dylan604 6y ago
    This sounds like one of those situations where devs tasked with programming something that they have little to no understanding of what they are programming does. Yes, the dev programmed button A to perform action B, but doesn't truly understand how/why/what/when/where action B. It's just a check box of the requirements that has now been completed. However, a dev that does understand action B would be able to anticipate various forms of input from button A and how that could cause undesired results from action B. This isn't a knock on the dev that is less knowledgable about the full thing. It is a knock on the people upstream that should be more aware of issues. Also, the people most knowledgeable about action B tend to have little to no understanding of programming. While they might understand that providing bogus data to action B can have bad results, they are not aware enough to tell the devs that sometimes valid looking data is just not sane.
    - acdha 6y ago
      I wouldn’t even assume lack of understanding as much as being locked into a work-to-order situation where they’re not rewarded, possibly punished, for doing anything which isn’t on the assignment. This is super common in outsourced hellscapes where the MBAs like to LARP as architects and UI designers and there’s no effective feedback loop from the actual users.
mumblemumble 6y ago
There's an old(ish) episode of the Stack Overflow podcast where they interview a couple folks from the site reliability engineering team. I think this is the one: https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasionally-maintaining-infrastructure-hate/ https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasi... Somewhere in there, one of the SREs said something that's stuck with me. Roughly, he said that the concept of human error is useless. Every time a human error escalates into a real problem, the deeper problem is a design flaw. At scale, human errors are a certainty, and, while documentation and training is far from useless, no amount of it can change that fact. It's therefore a critical responsibility of systems designers to design their systems to be resilient to mistakes.
- mybandisbetter 6y ago
  This is also a big concept in the medical world. There's an influential report on this from the 90s called "To Err is Human." https://pubmed.ncbi.nlm.nih.gov/25077248/ https://pubmed.ncbi.nlm.nih.gov/25077248/
- capableweb 6y ago
  This is also not a new idea that even comes from the software world. I'm not entirely sure of the original of such ideas, but the first time I heard about was in conjunction with airplanes and their UX. Basically, crashes and other issues from flying is never attributed to the operator, but the UX of the controls as if it was done better, would have prevented the operator of committing the error in the first place.
  - CPLX 6y ago
    Indeed. One of the really core things you learn when you dig deeper into aviation is that essentially every major disaster is a chained combination of multiple extremely unlikely failures in sequence. Which is a testament to both the incredible success this approach has had, as well as the impossibility of perfection.
    - Gare 6y ago
goatcode 6y ago
It looks like they were all under-qualified. I wonder how much the decision to hire them all will end up costing anyone other than them, and the people who hired them.
- read_if_gay_ 6y ago
  No. It looks like the people who built the software were underqualified.
  - goatcode 6y ago
    Maybe I misunderstood, but all of them either "thought" that was the right thing to do or signed off on it. Even if the software should have warned them, shouldn't they have known the correct transactions to make?
    - CydeWeys 6y ago
      They knew the correct transactions to make; the problem is the software was unbelievably counter-intuitive. At some point the user can't be blamed if the UX design is that bad.
    - kemotep 6y ago
      That's issue here with this specific software's UX. They all thought they were making the right transaction. There was no warning that they were not. It would be like if I hit the reply button here on Hacker News and instead of the comment I typed out it my reply contained my bank account information. And I was not told something I do not want to happen would happen by doing so.
      - goatcode 6y ago
        I think I understand now: the software was doing the wrong thing. They thought they were making one transaction, but they made a different one. One wonders how long that had been going on for, and how it could have been going on for so long. Thanks for the explanation. It saddens me how brutal people here are in response to a misunderstanding, judging by the downvotes, but oh well.

koheripbal 6y ago

To me this speaks more to the need for major money transfers to have more programmatic checks in place as well aa 2nd person signoffs.

toomuchtodo 6y ago
The entire story is the cost of not recognizing you're an engineering firm that handles money, versus a bank that bolts on IT at the lowest cost possible. Citi learned the hard way, which is good, as it's the only way for such an org to learn.
- DoofusOfDeath 6y ago
  > Citi learned the hard way Do we have evidence that they actually learned anything from this?
  - deleted 6y ago
    [deleted]
  - m12k 6y ago
    No, we just know that they paid for a lesson
  - dvfjsdhgfv 6y ago
    In all my previous companies whenever we earned more than expected from a given project, we'd routinely analyze it and it was our job to make sure we can somehow repeat or benefit from this lesson for the future. In the same way, whenever there was a significant drop, it was our job to make sure we know the reasons behind it and implement preventive measures if applicable. The only factor that changed was the level of 'why?'s asked. I can't imagine it's any different in Citibank.
    - DoofusOfDeath 6y ago
      > I can't imagine it's any different in Citibank. Having worked for companies the size of small startups up through megacorps, my experience has been that megacorps can be unfathomably foolish in these kinds of things. I'd expect Citibank to be less likely to learn (and apply) the right lessons from this experience, than a small or medium-sized business would.

Shivetya 6y ago

as many pointed out, it still isn't uncommon to repurpose or "Extend" and existing UI to do stuff it was never meant to do and then never go back and clean up the efforts when such use is common place. resulting in issues like Citi got. Seriously who thought that UI, requirements, text, and all, was a long term solution for handling any amount of money?

dheera 6y ago

I've used Citizens Bank before for business reasons and their UI is probably one of the worst I have ever seen. I don't understand why consumers get such good stuff and enterprises get such crap.

MattGaiser 6y ago
Do enterprises change banks because of bad UI/UX? There's your answer.
- joshgoldman 6y ago
  Did you even read the article?
  - MattGaiser 6y ago
    Did you read the context of my reply? I know the article is about bad internal UI/UX. But my understanding of what the person I am replying to said was that businesses have to put up with terrible UI/UX for their banking portals. Which I anecdotally know to be true, at least with Canadian banks. The business banking portal is often much worse than the consumer one.
  - jacurtis 6y ago
    Did you even read his question? Nowhere in the article (which I read in its entirety) does it answer the above reply's question. "Do companies consider good UI a competitive advantage when looking for a commercial bank or lender?" I don't know. But that's not addressed in the article. My guess is that it probably isn't a huge consideration. Very few people interact with these interfaces. Maybe one or two controllers at a company would ever log into this. Even CEOs and top management would never actually log into these accounts directly. They are going to get their information from internal reports and BI dashboards. So I bet no one really cares. If Citi offers a $400M loan at 1.1% interest and JP Morgan offers the same loan at 1.3% interest, my guess is that that company is going with Citi. I doubt UI/UX makes a difference here. Although in the consumer world it does have an impact which is why we have seen many banks improving their apps and websites lately to compete in the consumer banking space.

samizdis 6y ago

Ha! For a few seconds after reading your comment I thought I was experiencing déjà vu, but ... https://news.ycombinator.com/item?id=26170694 https://news.ycombinator.com/item?id=26170694 :-)

airstrike 6y ago
Oh no, I've been found out!
Debug_Overload 6y ago
You can tell it's very insightful because OP felt the need to repost it so the wisdom doesn't go to waste.

jacurtis 6y ago

It is also worth noting that Raj wasn't alone in the confusion on how to perform this task. Before he could complete the transfer his boss in India also had to look at what Raj did. Then their boss in Deleware also approved the transaction. ALL THREE PEOPLE INVOLVED IN THIS TASK WERE CONFUSED by the interface and all three thought that this was the correct way to do this. This wasn't the case of one person checked the wrong box. Everyone who looked at this was confused. This problem was not individual, but systemic.

TuringNYC 6y ago
Looking at the UI, anyone would be confused. I feel bad for the employee put into the spotlight. The spotlight should be on whomever designed and approved this software and UI. Unfortunately because of the court case, instead, people associated are the users.
crispyambulance 6y ago
Totally NOT SURPRISED that this was an awful Oracle enterprise app. These things were put together in haste in the 90's and Oracle keeps pushing them and their corporate customers keep these things going long past when the interface becomes utterly unfamiliar. The blame has to be shared, however, with the battle-axe personalities of the people that use this stuff. They would like nothing more than to retire while still using the _exact_ same applications and dated interfaces they've used for 20+ years. Literally, these are default "grey-theme" java applications, with all kinds of inconsistent UI quirks and absolutely no shits given for discoverability or even being able to create hyperlinks so that people can share stuff without screenshots. It's fine if it's the same people keep using these apps forever, but the problems start when new people get on-boarded. If it's like most places, the newbies will be given no training and the only help they will get would be from some dour in-house oracle-application jockey whose been doing the same thing for decades. That's when people make very, very expensive mistakes. We hear about this one because the sum was vast, but people make $10,000 mistakes on shit software from Oracle all the time-- that's partly why Oracle has a very deep bench of retained lawyers, I suppose. I have to say Citibank deserves it.
- 6y ago

cosmodisk 6y ago

You look at the screenshot,then read the summary on what happened and then wonder how on earth the bank still has any money left on their accounts with systems like this.

rowland66 6y ago
Looking at the screen tells you nothing about the quality of this system. The screens look a bit dated because the system is 20 years old. It is also a back office system used by a small number of what should be trained professionals. Pretty html and javascript would not have helped. This was a failure of the whole system. The software and the organization.
- cosmodisk 6y ago
  I didn't mean that it looks ugly- I take functionality over pretty design any time. The screenshot itself doesn't provide much info,but when combined with comment from the article,the who picture is pretty bleak tbh..
- murermader 6y ago
  This is not about ugly vs beautiful. Good UI is not primarily about being nice to look at, it is to be usable. The UI has to be really bad to be able to screw up this bad without even knowing it. Horrendously bad.

redshirtrob 6y ago

These issues are everywhere if you bother to notice. My in-laws had a coffee maker with a built-in grinder. The grind function was default enabled, so if you went to brew some coffee from pre-ground beans and just turned it on, the grinder would kick on with an awful sound. The solution they came up with was a "Grinder Off" button that lit up to indicate the grinder was turned off. It was...confusing to say the least. And as an infrequent user of this coffee maker it was all too easy to screw up. To add insult to injury, that off light turned off (i.e. enabled grinding) every time you brewed a new pot. You had to be sure to press it every time you brewed from pre-ground beans, which was always for us.

CydeWeys 6y ago
To be honest, that setup would make more sense for me; I always brew from freshly ground whole beans. It sounds like they flat out bought the wrong coffee maker. They should have gotten one that doesn't come with an integrated grinder at all. That coffee maker is well designed for its intended happy path use case, which is not the same as you can say for this Oracle software.
- redshirtrob 6y ago
  No doubt it was the wrong coffee maker for them. I don't think it was well designed though. I think it was "eh, ok" designed for the happy path, and very poor for the unhappy (but probably pretty normal) path. I think what threw me is the "grinder off" button with a red light. The red light made sense in the context. But, if I were designing this, I would have a "grinder" button with a green light that was default "on", indicating that grinding was imminent. There was no green light on this part. It was red or off. That would have been consistent with the rest of the interface, where a green light on the "brew" dial meant the timer was set and a red light meant the coffee had brewed.
  - wging 6y ago
    In this situation, a physical toggle switch seems like a good idea. Its state is always inspectable and it stays the way you left it.

1024core 6y ago

Let's not blame the lowly sub here. The next paragraph continues: Citibank's procedures require that three people sign off on a transaction of this size. In this case, that was Raj, a colleague of his in India, and a senior Citibank official in Delaware named Vincent Fratta. All three believed that setting the "principal" field to an internal wash account number would prevent payment of the principal. As he approved the transaction, Fratta wrote: "looks good, please proceed. Principal is going to wash."

SpaceManNabs 6y ago

And Raj will still probably get fired for jointly discovering this vulnerability.

rchaud 6y ago
Hopefully he just gets moved to a different project. At least according to the story and what was discussed in court, he was not singled out for blame.
- chris_wot 6y ago
  His name is now in a court filing AND published on Ars Technica. He's toast.

woobar 6y ago

Not defending this horrible UI, but this is a case of a non-standard operation and one should be extra careful when trying to override default behavior. Matt Levine has more details.[1] If I am an approver of a $900M transfer using an edge case I will follow instructions: "Citibank’s internal Fund Sighting Manual provides instructions for suppressing Flexcube’s default. When entering a payment, the employee is presented with a menu with several “boxes” that can be “checked” along with an associated field in which an account number can be input. The Fund Sighting Manual explains that, in order to suppress payment of a principal amount, “ALL of the below field[s] must be set to the wash account: FRONT[;] FUND[; and] PRINCIPAL” — meaning that the employee had to check all three of those boxes and input the wash account number into the relevant fields." [1] [1] https://www.bloomberg.com/opinion/articles/2021-02-17/citi-can-t-have-its-900-million-back https://www.bloomberg.com/opinion/articles/2021-02-17/citi-c...

kables 6y ago

Interesting pattern: 787 boeing crash, Citibank 500m confused transaction, are outsourced to the same place, India.

dang 6y ago
Please keep nationalistic flamewar off HN. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

w23j 6y ago

Well one could add, that other possible options are "COLLAT", "COMPINTSF", "DEFAUL" and "DFLFTC". Hard to believe that anyone could possibly be confused by that or miss the clearly necessary "FRONT" and "FUND".

fortran77 6y ago

I also wonder what the action of setting an account number in the "PRINCIPAL" field does if the system basically ignores it unless other fields are set. Why is this configuration even possible?

joejerryronnie 6y ago

I can understand if you’re venmoing your buddy $20, but if your are transferring millions of dollars, and faced with that horrendous interface, you might think to consult the standard operating procedures.

JJMcJ 6y ago

So there wasn't a final screen that said something like You are about to send $83,932,198.83 to (various account information fields) Unlike, say, a site that sells hand knit mittens or maybe organic kibble.

JumpCrisscross 6y ago
No, but it did ask for the email address twice.

dopidop 6y ago

I work on a large financial transaction system. A dumb backbone for bank the same size as citi. This UI is pretty good ! Haha. No seriously today we had a one hour conversation on what happen when a user cancel a trade twice. I don’t think we’re gonna find a elegant solution within the contraints of the clusterfuck of sub-system we maintaining. Kinda disheartening.

Igelau 6y ago

Ars Technica must have redacted the name. It just says "subcontractor" now.

DrBazza 6y ago

Having previously worked at "Megabank", this does not surprise me at all. Conway's Law applies here. It's a hugely dysfunctional bank, saddled with bureaucracy, fragile software (company change control ran on a IBM mainframe as late as 2015, requiring IE6), teams that are 'silo-ed', and talking with other teams are often the equivalent of a divorce lawyer's communication between spouses. Sandy Weill's crowning glory of mashing together Salomon Brothers, Smith Barney, Primerica, Travellers, Citicorp, and Diner's Club.

bluishgreen 6y ago

Amusing and mildly racist use of the name Raj in the comment, not from the original article as far as I can see. Please note that John of Delaware, his boss confirmed that principal goes to wash and said to “please proceed”

Igelau 6y ago
> Amusing and mildly racist use of the name Raj Interesting. My gut reaction was that they must have initially published his name and silently redacted it. Your gut reaction is that HN is seething with many racist commenters who use R__ as an epithet for any Indian contractor. Wayback Machine if you haven't already figured out which one of us is right.