21 ms·

Fintech engineering mistakes (2022)

153 points by 0x54MUR41 3y ago

fvdessen 3y ago
You can absolutely use doubles for money. Excel does it and so do many other financial tools. There are pros and cons but as long as you do rounding and comparisons correctly it works perfectly fine.
- gchadwick 3y ago
  The question is why? What does the double give you over a 64-bit integer? Sure when you divide and it leaves a fractional part you loose it and need to think about what happens there explicitly but you need to do the same for doubles to avoid pennies going missing and snowballing into larger errors.
  - jdiff 3y ago
    Using ints forces you to deal with all that. Using floats lets you easily ignore it and sweep it under the rug.
  - fvdessen 3y ago
    There are multiple factors: Doubles can exactly represent all 16 digit ints (IIRC) which is good enough for most use cases, you can catch the out of range cases (as you should do with ints as well) If you use long ints you must track the decimal precision along the value which is not always trivial if you use mixed currencies. Long ints are not guaranteed to correctly roundtrip through json serialisation / deserialisation Doubles are easier to handle in the frontend Currency math is different enough from regular math that you need special operator functions anyway so it’s not like ints are easier to handle either
  - cjs_ac 3y ago
    Using integer representations for currencies becomes very messy when dealing with more than one currency at a time. The United States dollar is famously subdivided into cents, but is also subdivided into 'mills' (one thousand to the dollar).[0] The Mauritanian ouguiya is divided into five khoums. The Madagascan ariary is divided into five iraimbilanja. The Maltese scudo is divided into twelve tarì, which are divided into twenty grani, which are divided into six piccioli. Historically, such currencies were ubiquitous. For example, prior to 15 February 1971, the pound sterling was divided into twenty shillings, each of which was divided into twelve pence, a system that originated with Roman currency and was used throughout the British Empire. Exchange rates are typically quoted in terms of the largest unit, whereas integer representations of currency would need to be done in terms of the smallest unit, so extensive information about currency structures would need to be used to correctly represent exchange rates. Floating point or binary-coded decimal representations are consequently much better. [0]: https://en.wikipedia.org/wiki/Mill_(currency) https://en.wikipedia.org/wiki/Mill_(currency)
- karmakurtisaani 3y ago
  If Excel uses doubles for money, it should be a warning sign. It uses simple numeric data type (I guess just ints) for freaking dates.. I can trace at least a couple of bugs in my career to just that fact.
  - mrbungie 3y ago
    Apart from its quirks Excel is fine, if you know its limitations. I think the real warning sign would be an analyst/programmer working with Excel and expecting high precision results.
    - Mordisquitos 3y ago
      Everything is fine apart from its quirks, if you know its limitations. The problem is when a tool's quirks and limitations are neither exhaustively documented nor can they be inferred from having reasonable knowledge about the base principles of the tool, but are rather learnt by experience (to be read as "through bad experiences").
    - recursive 3y ago
      I find Excel to be fine as long as I never use it.
- esotericimpl 3y ago
  [dead]
- indymike 3y ago
  > There are pros and cons but as long as you do rounding and comparisons correctly it works perfectly fine. This is exactly the issue with using floats where an arbitrary precision decimal with proper rounding is really needed. Easily solved with a good library and if your languages supports it, type, but it's really easy for a dev in a hurry to not use the library and roll some a=b+b*c_rate code that forces some type conversion. The rounding rules often are tied to contracts, and a subtle bug that's off a few mills here (total problem created $2.33) and there can lead to audits (total cost of audit $14,800) that cost a lot.
  - fvdessen 3y ago
    you can also easily hurry up subtly wrong money math with ints. (overflows, wrong rounding method, etc). So you should not in any case use the default math operators
    - lightbendover 3y ago
      At least with ints you can capture those issues with intent and handle them properly (as the better libraries that handle money tend to do).
  - phoehne 3y ago
    I'm tired of having to drag in another dependency and lose operators if I'm doing money math. I can create an experience in C++ that's almost rational, with operator overloading, but most other languages were designed well after we knew that doubles are not sufficient. And there's more than just arbitrary precision. For example,, some currencies use three decimal or no decimal places. Two just happens to be convenient for the Euro and Dollar. In addition, sometimes you carry prices to 3 or 4 places. But you still want banker's rounding. And I shouldn't be able to add Turkish Lira to US dollars, any more than the language allows adding floats and integers, without conversion. Then there's locale correct display for currencies (e.g. $ vs USD and before or after the money amount).
- phoehne 3y ago
  For example 16.10 is not exactly 16.10 in IEEE floating point. When you do enough operations, and depending on the order of operations, you can wind up several cents off. That sounds small, but can be enough to give your auditors heart-burn. COBOL does BCD arithmetic, (not really but at least conceptually), and it's penny accurate to 31 digits (as per the standard - implementations may have greater accuracy). Frankly, it's stupid that 63 years after COBOL we're still treating money and currency as an afterthought in languages that are supposed to be business oriented. Proper currency handling should be part of the language.
  - tiedieconderoga 3y ago
    It would be neat to have a fundamental "money" primitive alongside int, float, string, etc. It might not be simple to implement, though. Off the top of my head, * Can you confidently say that $100>¥10 in an offline environment? * You'd need to support different bases for splitting units to deal with eldritch values such as farthings. * Where is the line drawn? Is a Bitcoin a valid currency? A gold Krugerrand? A bundle of stock shares? A bushel of apples? All of that said, I would love to see a world where I could write code like: money balance = money.lumber.grade.kilograms(8);
    - jerf 3y ago
      "Can you confidently say that $100>¥10 in an offline environment?" A major problem money has is that it isn't a unit in the sense we usually take the term to mean. We expect, for instance, that translating one unit to another with a suitable level of precision should be translatable without loss back to the original unit, but that's not true for money, even ignoring transaction costs. If "US dollar" is a unit, it is a unit that technically stands alone in its own universe, not truly convertible to anything else, not even other currencies. All conversions are transient events with no repeatability. But that is very inconvenient to deal with, and with sufficient value stability of all the relevant values, often it's a sufficient approximation to just pretend it's a unit. But if you zoom in enough, the approximation breaks down. For that and similar reasons, while you could theoretically write that line of code, it would be implicitly depending on a huge pile of what would in most languages be global state. It would be a dubious line of code.
- jaclaz 3y ago
  If you want a simple real life example (unconnected to the way numbers are stored) here it is (accounting may have quirks that arrive unexpected). In EU prices to customers are required to be comprehensive of VAT, so a price is € 60,00 included VAT 10%. But in an invoice/receipt you have to explicit how much is the net and how much is the tax, so 60 / 1.10 = 54.55 and VAT is 54.55 x 0.10 = 5.46 which makes a nice 60.01. You may be tempted to round down the 60 / 1.10 = 54.54 and have VAT 54.54 x 0.10 = 5.45 but this makes 59.99.
- recursive 3y ago
  What do you mean by "correctly"? It sounds like victim blaming. For money, I don't think it's perfectly fine to use numbers where this isn't true. 0.1 + 0.2 <= 0.3 // sure looks true to me
- NovemberWhiskey 3y ago
  >as long as you do rounding and comparisons correctly it works perfectly fine. At least until you need to add things, at which point you need Kahan's algorithm.
- KRAKRISMOTT 3y ago
  It's sufficient for quantitative finance and trading but not for accounting. For financial engineering and modeling it doesn't matter if results are a few fractions of a cent off as long as your profit margins/model tolerances are greater than the error, because your broker/bank/exchange will keep track of the exact values in your account. But if you are building a bank/broker/exchange, then tracking it precisely enough for GAAP is now your problem.
indymike 3y ago
This is a great little article, and I can't emphasize what a big deal #2 is.
- JohnBooty 3y ago
  Some folks will laugh at this advice. Others (like me) will cry because we know of multibillion-dollar fintechs that still struggle with this. The one I'm thinking of didn't even have cents for a long time. After a pretty heroic migration effort they added cents. And they did it properly. But within weeks folks were using floats all over the place for money, leading to flaky tests and all kinds of other errors.
  - foobarian 3y ago
    JPY has entered the room
  - JackFr 3y ago
    I have worked at banks and fintechs for the past 30 years and honestly have never used anything but a doubles for money with no issues (and a simpler code base.) I understand the sentiment and the potential issues, but it's really kind of domain dependent.
    - Brystephor 3y ago
      Is storing $1.05 really that much simpler than storing something as 105? Use a long to store money, pretty easy.
      - gpderetta 3y ago
        I might or might not have stories about fix numerical types underflowing. Not that fix nums are wrong, but they are far from safe.
      - indymike 3y ago
        Storing is one thing, computation is another :-)
molsongolden 3y ago
One more: Timezones Various platforms use: UTC, the user’s timezone as set in their dashboard, the user’s detected timezone, the timezone of the server that is generating the reports. Aggregating or reconciling data from different platforms can be a pain if the timezones aren’t clearly indicated. I’ve had bank statements that didn’t agree to CSV exports of the same data because the servers generating the two reports were in different timezones.
- SilasX 3y ago
  Haha yeah for the past tax year I I looked at my transaction history and it was missing some that I expected to be there. Turns out, the exchange (Gemini, for crypto) uses UTC, but the IRS docs always tell you that a cutoff is relative to your own time zone. The transactions were in the late evening on New Year's Eve, central US time, but that was already the next year by UTC, so any readout of "transactions for 2022" would not include them. [1] e.g. https://www.irs.gov/taxtopics/tc301 https://www.irs.gov/taxtopics/tc301
  - koolba 3y ago
    That’d be a great subplot for a movie. The antagonist sets up an elaborate money laundering operation that leverages conflicting definitions of the taxable year and moves off shore profits that are “lost” between the systems. Could call it “Black Ink to the Future.
  - radiator 3y ago
    That is supposed to be a festive hour, not one for trading.
    - SilasX 3y ago
      Haha, you're not wrong. I think it was actually more like 8pm, shortly before I left to live it up. But either way, transacting near the new year shouldn't break financial systems.
- SideburnsOfDoom 3y ago
say_it_as_it_is 3y ago
Why should anyone have a look at TigerBeetle, a bleeding edge accounting system built with a language no one understands? Since no one understands Zig, no one will be able to maintain TigerBeetle. The company is exposing itself to a great deal of risk with this decision. Adopting TigerBeetle is a colossal business mistake, not an engineering mistake.
- jorangreef 3y ago
  Hey, Joran from TigerBeetle here! You raise a fair question. Coincidentally, one of the reasons we picked Zig was for how readable it was, and strikingly so, even for high level programmers who might not understand systems programming or C. Because Zig reads like TypeScript, and we were working in payment switches where the majority of programmers could read that. This particular switch, in fact, had this same business requirement, that programmers should be able to at least read the systems language. But generally, our experience has been that people who understand C will understand how to maintain Zig [0]. Zig's toolchain is also more accessible, and across all platforms. Zig's compiler is already being used by Uber for hermetic builds. It's also easy to learn. You can pick up Zig in a week and be comfortable in a month. Zig has a simple grammar. I love how, when we have someone join the team, we never have a discussion about how to learn Zig, as if it's a difficult language to master (like C++!). Rather, there's excitement around learning the language, even ahead of starting at work, and within a day or two they're committing. We made this decision for TigerBeetle in July 2020, and didn't take it lightly. We had already followed Zig's progress for 2 years by that point, and many factors were considered [1][2][3]. C was the other contender, given that we had to handle memory allocation failure. The crux of the decision, then, was whether to invest in a systems language of the last 30 years, or in a systems language of the next 30 years. A distributed database is a big investment. It made sense to invest for the future. If anything, it would have been a colossal business mistake to have picked C or C++, which would have crippled our development velocity. Furthermore, for TigerBeetle's design goals, especially w.r.t. our adoption of NASA's Power of Ten Rules for Safety-Critical Code and thus static memory allocation, Zig made (and continues to make) the most sense. We also liked the efficient performance culture surrounding Zig, with talented game developers like Michal Ziulek and Stephen Gutekanst, and embedded programmers like xq, Matt Knight, Jens Goldberg and others moving to it. These industries (gaming, embedded) are often a good litmus test of where systems programming is at. More details (our thinking on Zig through the lens of safety/performance/tooling/ecosystem/hiring/marketing) here: [0] https://kristoff.it/blog/maintain-it-with-zig/ https://kristoff.it/blog/maintain-it-with-zig/ [1] https://docs.tigerbeetle.com/FAQ#why-is-tigerbeetle-written-in-zig https://docs.tigerbeetle.com/FAQ#why-is-tigerbeetle-written-... [2] Building a Distributed DB in Zig: https://www.youtube.com/watch?v=0pHHb6ONTbw https://www.youtube.com/watch?v=0pHHb6ONTbw [3] TigerStyle! (How to Design Safer Systems in Less Time): https://www.youtube.com/watch?v=w3WYdYyjek4 https://www.youtube.com/watch?v=w3WYdYyjek4
gymbeaux 3y ago
This is a nit but is there a better way to say “RDBMS databases?” It feels like saying “ATM Machine”. Could we just say “relational database” or “state of the art database management system?”
- nickpeterson 3y ago
  A database that just got developed could be called a TardyBMS… I’m sorry I’ve drank too much coffee this morning.
  - gymbeaux 3y ago
    A musical database might be called a CardiB-MS
agentultra 3y ago
Don't be afraid to use formal methods. Queues, retries, event sourcing, payment state handling -- there are "global" properties we desire from these systems: certain things we want to make sure are always true and others that we want to make sure eventually happen. For the single process case, which nearly every developer thinks about, it can seem like a solved problem but we live in a concurrent world with network failures and vendors with errors in their own systems: it's nearly impossible for you to think really hard in your head and be sure that your queue retry strategy will maintain those properties that are important to your business. It's nearly impossible to do this with lightweight, informal testing strategies employed by busy software teams. By modelling your systems you will learn what the important failure modes are and you will get better at designing systems that are resilient and efficient. Card payment systems are fairly unreliable peer-to-peer messaging systems. Be prepared for a lot of complexity. Using an event-sourcing architecture is really useful here for that "auditing" requirement and for debugging transaction state when the network sends you messages in error, out of order, or they forget to retry themselves when they promised to, when merchants send bad data, when POS systems do weird things, etc.
phoehne 3y ago
RDMS databases are not a weak point. A bad model of your transaction is a weak point. Depending on the application, and not all applications require this, you may need a consistent read state for the data. Eventually consistent is like saying periodically wrong. Let's say a trader enters a trade for that exceeds there allowed VAR, because they looked at their screen thought they had more dry powder. (Cocaine jokes aside). That becomes a risk management problem. (Also the risk that traders can just use the 'It said I was below my limit' excuse). If you want an example where consistency is not important, you might be able to overdraw from your bank with your ATM card. The bank is happy for this to be inconsistent, since they can charge for the overdraft.
- boblob-law 3y ago
  The bank cannot charge for this unless you as the customer have opted in. See authorized positive settle negative. US based banks.
yafetn 3y ago
I’d also add: — If you’re using JSON to pass around monetary quantities (eg. from the frontend to the backend), put them in strings as opposed to the native number type. You never know what the serializers and deserializers across languages will do to your numbers (round them, truncate them etc.). — Start recording the currency of the transactions as early as possible. It can be a separate column in your table. — Use TIMESTAMPTZ. Always.
- hirundo 3y ago
  I'm generally not a fan of overloading fields, but we took the opportunity to add the currency code when we decided to encode monetary quantities in JSON as strings, e.g. "0.02USD". This has worked well, particularly because we use a money handling library that parses it unchanged.
  - zeisss 3y ago
    Do you always put the currency behind the number? I know some countries prefer the currency infront (e.g. USD), others behind (EUR). But since this is not a human visible field, this is probably irrelevant, right?
    - hirundo 3y ago
      I think I looked at this when I laid it out: https://ux.stackexchange.com/questions/9105/international-currency-formatting-guidelines-currency-codes https://ux.stackexchange.com/questions/9105/international-cu... So we put the ISO code at the end and ignore whitespace. But including a prefix sign as with `€1.234.567,89 EUR` would break it.
      - hermitdev 3y ago
        You also included localized formatting (the '.' and ',' ) in your example that'll likely break on parsing without special handling. For interchange, you'll want to avoid using localized formats, e.g. stick to the "C" locale or something specific or agreed to between the involved parties.
gunshai 3y ago
Shit, I wish I read this prior to my interviews at Cash App.
- neon_electro 3y ago
  In what way do you think it would've made a difference? Asking as someone who's applied to Block/Square/Cash App in the past.
  - gunshai 3y ago
    The difference is small, but mostly as probing questions to interviewers. I take the mind the more you can speak the language the more comfortable people become with you. Here's something small but I was doing a SQL interview one involved a a card_tx table with the amounts stored in integers values in cents. I immediately noticed this and chatted with the interviewer about this. My assumptions were wrong, and the floating point arithmetic was a simpler reason than what I was thinking. Anyway you can display you have thought about relevant business problems to where you are interview engineers tend to have a strong reflex to engage the person more.
    - neon_electro 3y ago
      Appreciate your reply!
      - gunshai 3y ago
        You're welcome, next time I'll try not to have a stroke at the end of my comment. hah. For some reason I can't edit it. shrug
gsatic 3y ago
And that's why I shall be sticking to cash for the next 10 years till all the beta testing is complete. By which time hopefully interest bearing CBDCs will show up, and make all these mindless intermediaries sitting between my wallet and someone elses wallet obsolete.
donatj 3y ago
I wish strictly INSERT-only tables were a standard SQL feature. It would be so very useful.
- lixtra 3y ago
  What’s the advantage of INSERT-only table over having a normal table and all normal users only have INSERT privilege on that table? As admin I can still recreate the whole insert table with modified data, etc.
dspillett 3y ago
> Over the last decade, fintechs using RDBMS databases to record transactions have built features onto their databases like a journal of all changes to all data and clever ways of making sure the data hasn’t been modified by storing checksums of data as transactions are added. (under “Updating transactions”) It is worth noting that many of these methods don't prevent tampering, they only make it visible when you look for it at which point the fact you are explicitly looking for it (rather than having been alerted to a potential issue) might imply it is too late.
- bostik 3y ago
  Which is also why auditors love git. The number one thing they crave is immutability. But when that's not an option, tamper-evident comes as a close second.
SideburnsOfDoom 3y ago
The recommendation that "developers should use integers to represent money" (with an implicit 2 decimal places, that will work for many but not all currencies) is not a great one. If your language has a dedicated type for monetary amounts, use that. (1) If it does not, but you can make a value object to represent, e.g. amount and currency code, then do that. If however, your language does not have a dedicated type for monetary amounts, or one cannot be trivially built or retrieved as a package (2), then you should ask yourself if it is really a suitable language for financial tasks. 1) https://learn.microsoft.com/en-us/dotnet/api/system.decimal https://learn.microsoft.com/en-us/dotnet/api/system.decimal 2) https://github.com/shopspring/decimal https://github.com/shopspring/decimal https://www.npmjs.com/package/ts-money https://www.npmjs.com/package/ts-money
- gytisgreitai 3y ago
  Stripe uses integers [1] Adyen uses integers [2] Square uses integers [3] So not sure that the industry would agree with you [1] https://stripe.com/docs/api/prices/create#create_price-unit_amount https://stripe.com/docs/api/prices/create#create_price-unit_... [2] https://docs.adyen.com/api-explorer/Checkout/70/post/payments#request-additionalAmount-value https://docs.adyen.com/api-explorer/Checkout/70/post/payment... [3] https://developer.squareup.com/reference/square/objects/Money https://developer.squareup.com/reference/square/objects/Mone...
  - SideburnsOfDoom 3y ago
    > Stripe uses integers [1] a) On web api, so this is by definition data interchange, not "in a language". b) with a currency code to cross check it. c) implicitly "in cents" which needs further documentation - does this mean "pence" when the currency is GBP? How does this work for for JPY? BHD? d) cannot represent fractions of a cent or penny. So: this int plus currency code plus docs is OK but not great, a lowest common denominator format for interchange, requires further documentation and cross-checking as "100 USD" does not mean 100 bucks and the conversion is currency-specific, and cannot represent all values. I wouldn't refuse to convert _to and from_ this format at the edges of my application, for data interchange, but the conversion to something clearer and richer IMHO should remain there. Payment APIs are far from the only use case in fintech, for interest calculations you do have to care about fractions of a cent or penny. In fact, if your case is "I call the stripe api" ... are you sure you're a fintech and not an online store? Get back to me when you have to interop with FiServ, MasterCard or SAP.
itsthecourier 3y ago
I'm surprised the article doesn't explain how float numbers are stored in memory according to IEEE that's the main reason you don't use them to represent money. If you are an engineer who doesn't know what a mantissa is, stop two minutes for a cool ride https://lashewi.medium.com/storing-currency-values-and-float-precision-99029c1e8b71 https://lashewi.medium.com/storing-currency-values-and-float...
submeta 3y ago
Have built a fintech startup here in Germany ten years ago. The article mentions lots of important things. Here‘s another one: Time synchronization. It is incredibly important in fintech applications for a number of reasons: 1. *Transaction Ordering:* Financial transactions often need to be processed in the order they were initiated. This is especially crucial in high-frequency trading where trades are often made in milliseconds or microseconds. A small difference in timing could potentially lead to substantial financial gains or losses. Therefore, accurate time synchronization ensures fairness and order in the execution of these transactions. 2. *Security:* Accurate timekeeping helps in maintaining security. For instance, time-based one-time passwords (TOTPs) are widely used in two-factor authentication systems. These passwords are valid only for a short period of time and rely on synchronized clocks on the server and client side. 3. *Audit Trails and Dispute Resolution:* Timestamping transactions can help create a precise audit trail, which is critical for detecting and investigating fraudulent activities. In case of any dispute, a detailed and accurate transaction history backed by synchronized time can help resolve the issue. 4. *Distributed Systems:* In distributed systems, time synchronization is important to ensure data consistency. Many financial systems are distributed over different geographical locations, and transactions need to be coordinated between these systems in an orderly fashion. This requires all servers to have their clocks synchronized. I am sure there are even more fields where this is relevant.
- jorangreef 3y ago
  This was also a surprise to me when we started TigerBeetle. How clock synchronization protocols can experience unaligned network partitions, where the ledger database cluster is able to continue running, but now with the risk of unsynchronized clocks and far-future timestamps, which can in turn then lead to money being locked up in 2PC payment protocols. We therefore spent considerable effort [0] on clock synchronization in TigerBeetle, not for the consensus protocol—we never risk stale reads or take a chance with clock error bounds—but rather simply for accurate audit trails and to keep inflight liquidity from being locked up if transactions take too long to get rolled back. [0] https://tigerbeetle.com/blog/three-clocks-are-better-than-one/ https://tigerbeetle.com/blog/three-clocks-are-better-than-on...
- troupo 3y ago
theptip 3y ago
I think this article covers many of the main points, but could be worded/structured better. There are terms of art that more precisely refer to the concepts you need. For “using floating point data types”, it’s even worse; you often need to use strings, for example if you need to store a bank account number “01234567789” will have the leading zero stripped if you use a numeric type. “Updating transactions” would be better phrased as “use an append-only log / evented architecture”. (Also, “use a double-entry ledger” is probably the most valuable advice I could have sent myself prior to getting into FinTech.) “Be careful with retry” should be more strictly “use idempotent operations” and link to the canonical Stripe article on idempotency keys (https://stripe.com/blog/idempotency https://stripe.com/blog/idempotency). Another important one to think about is bitemporality. “Created at” vs “effective at”. Not obvious at first and you’ll have some painful migrations if you don’t build it in. Fowler has a good overview here: https://martinfowler.com/eaaDev/timeNarrative.html https://martinfowler.com/eaaDev/timeNarrative.html. Edit to add - the advice that maybe using a NoSQL database is pretty bad IMO. I’d advise in the opposite direction - use SERIALIZABLE isolation in a SQL database. Read up on your Aphyr blog posts before trying to do anything distributed. Be paranoid about race conditions / serialization anomalies. If you eventually hit performance issues you need to think hard about what anomalies your access patterns might be subject to. (Obviously HFT won’t use serializable SQL).
- ahtihn 3y ago
  > if you need to store a bank account number “01234567789” will have the leading zero stripped if you use a numeric type. This goes for any "number" that is actually an identifier. Phone numbers aren't really numbers either for example. Rule of thumb: if it doesn't make sense to do math with it, it should probably be a numeric type unless there's a very good reason.
mhh__ 3y ago
Tip: you can express (city) time "zones" in the type system e.g. London time and New York time.
- gpderetta 3y ago
  You can, but because the full set if timezones is only known at runtime, it is not terribly useful. You can statically distinguish a canonical static timezone (say UTC) vs a dynamic ine though. Same for currency.
  - mhh__ 3y ago
    There are only a few useful times in markets e.g. London, new York etc. If you don't know then there's no point but the idea is to stop you comparing prices from London close to new York and so on
    - gpderetta 3y ago
      I've gone as far as encoding bid and ask prices with different types, but tz really has never occurred to me. Closing times are also exchange/contract specific, not just location.
TeeWEE 3y ago
I really mis the mistake of NOT using double entry bookkeeping in the way you store transactions. Next to that, ensure transactions are immutable, and ensure reporting is idempotent: Close your periods and generate reports for them, when you regenerate the report it should be identical. This is more domain tips. A lot of technical tips hold for non fintech too.
acadavid79 3y ago
A little off-topic, but I would have loved to see mifos / fineract[0] referenced in the article. Great open source banking core having many of the strengths listed [0] https://github.com/apache/fineract/ https://github.com/apache/fineract/
hannofcart 3y ago
All good points.. One more from me: - Payment/order processing systems typically involve complex logic. - One of the best ways we managed to keep complexity in check was to model these as state machines (with the state itself being persisted to DB)
potamic 3y ago
#2 and #3 are pretty solid advice. But the rest of them are nothing specific to fintech. I will also offer a few additional points. 1. Never record an amount without its currency. 2. Reconcile all your data all the time. Never let any data go unaccounted for. 3. Maker-checker is a powerful concept. Embrace it to the fullest across your system. 4. You will be dealing with all sorts of non-standardized financial integrations. A lot. Think adapter pattern as early as possible. 5. You will be answering to multiple regulatory agencies. Create boundaries between them within your system and reduce the surface of compliance as much as possible.

153 points by 0x54MUR41 3y ago

You can absolutely use doubles for money. Excel does it and so do many other financial tools. There are pros and cons but as long as you do rounding and comparisons correctly it works perfectly fine.

gchadwick 3y ago
The question is why? What does the double give you over a 64-bit integer? Sure when you divide and it leaves a fractional part you loose it and need to think about what happens there explicitly but you need to do the same for doubles to avoid pennies going missing and snowballing into larger errors.
- jdiff 3y ago
  Using ints forces you to deal with all that. Using floats lets you easily ignore it and sweep it under the rug.
- fvdessen 3y ago
  There are multiple factors: Doubles can exactly represent all 16 digit ints (IIRC) which is good enough for most use cases, you can catch the out of range cases (as you should do with ints as well) If you use long ints you must track the decimal precision along the value which is not always trivial if you use mixed currencies. Long ints are not guaranteed to correctly roundtrip through json serialisation / deserialisation Doubles are easier to handle in the frontend Currency math is different enough from regular math that you need special operator functions anyway so it’s not like ints are easier to handle either
- cjs_ac 3y ago
  Using integer representations for currencies becomes very messy when dealing with more than one currency at a time. The United States dollar is famously subdivided into cents, but is also subdivided into 'mills' (one thousand to the dollar).[0] The Mauritanian ouguiya is divided into five khoums. The Madagascan ariary is divided into five iraimbilanja. The Maltese scudo is divided into twelve tarì, which are divided into twenty grani, which are divided into six piccioli. Historically, such currencies were ubiquitous. For example, prior to 15 February 1971, the pound sterling was divided into twenty shillings, each of which was divided into twelve pence, a system that originated with Roman currency and was used throughout the British Empire. Exchange rates are typically quoted in terms of the largest unit, whereas integer representations of currency would need to be done in terms of the smallest unit, so extensive information about currency structures would need to be used to correctly represent exchange rates. Floating point or binary-coded decimal representations are consequently much better. [0]: https://en.wikipedia.org/wiki/Mill_(currency) https://en.wikipedia.org/wiki/Mill_(currency)
karmakurtisaani 3y ago
If Excel uses doubles for money, it should be a warning sign. It uses simple numeric data type (I guess just ints) for freaking dates.. I can trace at least a couple of bugs in my career to just that fact.
- mrbungie 3y ago
  Apart from its quirks Excel is fine, if you know its limitations. I think the real warning sign would be an analyst/programmer working with Excel and expecting high precision results.
  - Mordisquitos 3y ago
    Everything is fine apart from its quirks, if you know its limitations. The problem is when a tool's quirks and limitations are neither exhaustively documented nor can they be inferred from having reasonable knowledge about the base principles of the tool, but are rather learnt by experience (to be read as "through bad experiences").
  - recursive 3y ago
    I find Excel to be fine as long as I never use it.
esotericimpl 3y ago
[dead]
indymike 3y ago
> There are pros and cons but as long as you do rounding and comparisons correctly it works perfectly fine. This is exactly the issue with using floats where an arbitrary precision decimal with proper rounding is really needed. Easily solved with a good library and if your languages supports it, type, but it's really easy for a dev in a hurry to not use the library and roll some a=b+b*c_rate code that forces some type conversion. The rounding rules often are tied to contracts, and a subtle bug that's off a few mills here (total problem created $2.33) and there can lead to audits (total cost of audit $14,800) that cost a lot.
- fvdessen 3y ago
  you can also easily hurry up subtly wrong money math with ints. (overflows, wrong rounding method, etc). So you should not in any case use the default math operators
  - lightbendover 3y ago
    At least with ints you can capture those issues with intent and handle them properly (as the better libraries that handle money tend to do).
- phoehne 3y ago
  I'm tired of having to drag in another dependency and lose operators if I'm doing money math. I can create an experience in C++ that's almost rational, with operator overloading, but most other languages were designed well after we knew that doubles are not sufficient. And there's more than just arbitrary precision. For example,, some currencies use three decimal or no decimal places. Two just happens to be convenient for the Euro and Dollar. In addition, sometimes you carry prices to 3 or 4 places. But you still want banker's rounding. And I shouldn't be able to add Turkish Lira to US dollars, any more than the language allows adding floats and integers, without conversion. Then there's locale correct display for currencies (e.g. $ vs USD and before or after the money amount).
phoehne 3y ago
For example 16.10 is not exactly 16.10 in IEEE floating point. When you do enough operations, and depending on the order of operations, you can wind up several cents off. That sounds small, but can be enough to give your auditors heart-burn. COBOL does BCD arithmetic, (not really but at least conceptually), and it's penny accurate to 31 digits (as per the standard - implementations may have greater accuracy). Frankly, it's stupid that 63 years after COBOL we're still treating money and currency as an afterthought in languages that are supposed to be business oriented. Proper currency handling should be part of the language.
- tiedieconderoga 3y ago
  It would be neat to have a fundamental "money" primitive alongside int, float, string, etc. It might not be simple to implement, though. Off the top of my head, * Can you confidently say that $100>¥10 in an offline environment? * You'd need to support different bases for splitting units to deal with eldritch values such as farthings. * Where is the line drawn? Is a Bitcoin a valid currency? A gold Krugerrand? A bundle of stock shares? A bushel of apples? All of that said, I would love to see a world where I could write code like: money balance = money.lumber.grade.kilograms(8);
  - jerf 3y ago
    "Can you confidently say that $100>¥10 in an offline environment?" A major problem money has is that it isn't a unit in the sense we usually take the term to mean. We expect, for instance, that translating one unit to another with a suitable level of precision should be translatable without loss back to the original unit, but that's not true for money, even ignoring transaction costs. If "US dollar" is a unit, it is a unit that technically stands alone in its own universe, not truly convertible to anything else, not even other currencies. All conversions are transient events with no repeatability. But that is very inconvenient to deal with, and with sufficient value stability of all the relevant values, often it's a sufficient approximation to just pretend it's a unit. But if you zoom in enough, the approximation breaks down. For that and similar reasons, while you could theoretically write that line of code, it would be implicitly depending on a huge pile of what would in most languages be global state. It would be a dubious line of code.
jaclaz 3y ago
If you want a simple real life example (unconnected to the way numbers are stored) here it is (accounting may have quirks that arrive unexpected). In EU prices to customers are required to be comprehensive of VAT, so a price is € 60,00 included VAT 10%. But in an invoice/receipt you have to explicit how much is the net and how much is the tax, so 60 / 1.10 = 54.55 and VAT is 54.55 x 0.10 = 5.46 which makes a nice 60.01. You may be tempted to round down the 60 / 1.10 = 54.54 and have VAT 54.54 x 0.10 = 5.45 but this makes 59.99.
recursive 3y ago
What do you mean by "correctly"? It sounds like victim blaming. For money, I don't think it's perfectly fine to use numbers where this isn't true. 0.1 + 0.2 <= 0.3 // sure looks true to me
NovemberWhiskey 3y ago
>as long as you do rounding and comparisons correctly it works perfectly fine. At least until you need to add things, at which point you need Kahan's algorithm.
KRAKRISMOTT 3y ago
It's sufficient for quantitative finance and trading but not for accounting. For financial engineering and modeling it doesn't matter if results are a few fractions of a cent off as long as your profit margins/model tolerances are greater than the error, because your broker/bank/exchange will keep track of the exact values in your account. But if you are building a bank/broker/exchange, then tracking it precisely enough for GAAP is now your problem.

indymike 3y ago

This is a great little article, and I can't emphasize what a big deal #2 is.

JohnBooty 3y ago
Some folks will laugh at this advice. Others (like me) will cry because we know of multibillion-dollar fintechs that still struggle with this. The one I'm thinking of didn't even have cents for a long time. After a pretty heroic migration effort they added cents. And they did it properly. But within weeks folks were using floats all over the place for money, leading to flaky tests and all kinds of other errors.
- foobarian 3y ago
  JPY has entered the room
- JackFr 3y ago
  I have worked at banks and fintechs for the past 30 years and honestly have never used anything but a doubles for money with no issues (and a simpler code base.) I understand the sentiment and the potential issues, but it's really kind of domain dependent.
  - Brystephor 3y ago
    Is storing $1.05 really that much simpler than storing something as 105? Use a long to store money, pretty easy.
    - gpderetta 3y ago
      I might or might not have stories about fix numerical types underflowing. Not that fix nums are wrong, but they are far from safe.
    - indymike 3y ago
      Storing is one thing, computation is another :-)

molsongolden 3y ago

One more: Timezones Various platforms use: UTC, the user’s timezone as set in their dashboard, the user’s detected timezone, the timezone of the server that is generating the reports. Aggregating or reconciling data from different platforms can be a pain if the timezones aren’t clearly indicated. I’ve had bank statements that didn’t agree to CSV exports of the same data because the servers generating the two reports were in different timezones.

SilasX 3y ago
Haha yeah for the past tax year I I looked at my transaction history and it was missing some that I expected to be there. Turns out, the exchange (Gemini, for crypto) uses UTC, but the IRS docs always tell you that a cutoff is relative to your own time zone. The transactions were in the late evening on New Year's Eve, central US time, but that was already the next year by UTC, so any readout of "transactions for 2022" would not include them. [1] e.g. https://www.irs.gov/taxtopics/tc301 https://www.irs.gov/taxtopics/tc301
- koolba 3y ago
  That’d be a great subplot for a movie. The antagonist sets up an elaborate money laundering operation that leverages conflicting definitions of the taxable year and moves off shore profits that are “lost” between the systems. Could call it “Black Ink to the Future.
- radiator 3y ago
  That is supposed to be a festive hour, not one for trading.
  - SilasX 3y ago
    Haha, you're not wrong. I think it was actually more like 8pm, shortly before I left to live it up. But either way, transacting near the new year shouldn't break financial systems.
SideburnsOfDoom 3y ago

say_it_as_it_is 3y ago

Why should anyone have a look at TigerBeetle, a bleeding edge accounting system built with a language no one understands? Since no one understands Zig, no one will be able to maintain TigerBeetle. The company is exposing itself to a great deal of risk with this decision. Adopting TigerBeetle is a colossal business mistake, not an engineering mistake.

jorangreef 3y ago
Hey, Joran from TigerBeetle here! You raise a fair question. Coincidentally, one of the reasons we picked Zig was for how readable it was, and strikingly so, even for high level programmers who might not understand systems programming or C. Because Zig reads like TypeScript, and we were working in payment switches where the majority of programmers could read that. This particular switch, in fact, had this same business requirement, that programmers should be able to at least read the systems language. But generally, our experience has been that people who understand C will understand how to maintain Zig [0]. Zig's toolchain is also more accessible, and across all platforms. Zig's compiler is already being used by Uber for hermetic builds. It's also easy to learn. You can pick up Zig in a week and be comfortable in a month. Zig has a simple grammar. I love how, when we have someone join the team, we never have a discussion about how to learn Zig, as if it's a difficult language to master (like C++!). Rather, there's excitement around learning the language, even ahead of starting at work, and within a day or two they're committing. We made this decision for TigerBeetle in July 2020, and didn't take it lightly. We had already followed Zig's progress for 2 years by that point, and many factors were considered [1][2][3]. C was the other contender, given that we had to handle memory allocation failure. The crux of the decision, then, was whether to invest in a systems language of the last 30 years, or in a systems language of the next 30 years. A distributed database is a big investment. It made sense to invest for the future. If anything, it would have been a colossal business mistake to have picked C or C++, which would have crippled our development velocity. Furthermore, for TigerBeetle's design goals, especially w.r.t. our adoption of NASA's Power of Ten Rules for Safety-Critical Code and thus static memory allocation, Zig made (and continues to make) the most sense. We also liked the efficient performance culture surrounding Zig, with talented game developers like Michal Ziulek and Stephen Gutekanst, and embedded programmers like xq, Matt Knight, Jens Goldberg and others moving to it. These industries (gaming, embedded) are often a good litmus test of where systems programming is at. More details (our thinking on Zig through the lens of safety/performance/tooling/ecosystem/hiring/marketing) here: [0] https://kristoff.it/blog/maintain-it-with-zig/ https://kristoff.it/blog/maintain-it-with-zig/ [1] https://docs.tigerbeetle.com/FAQ#why-is-tigerbeetle-written-in-zig https://docs.tigerbeetle.com/FAQ#why-is-tigerbeetle-written-... [2] Building a Distributed DB in Zig: https://www.youtube.com/watch?v=0pHHb6ONTbw https://www.youtube.com/watch?v=0pHHb6ONTbw [3] TigerStyle! (How to Design Safer Systems in Less Time): https://www.youtube.com/watch?v=w3WYdYyjek4 https://www.youtube.com/watch?v=w3WYdYyjek4

gymbeaux 3y ago

This is a nit but is there a better way to say “RDBMS databases?” It feels like saying “ATM Machine”. Could we just say “relational database” or “state of the art database management system?”

nickpeterson 3y ago
A database that just got developed could be called a TardyBMS… I’m sorry I’ve drank too much coffee this morning.
- gymbeaux 3y ago
  A musical database might be called a CardiB-MS

agentultra 3y ago

Don't be afraid to use formal methods. Queues, retries, event sourcing, payment state handling -- there are "global" properties we desire from these systems: certain things we want to make sure are always true and others that we want to make sure eventually happen. For the single process case, which nearly every developer thinks about, it can seem like a solved problem but we live in a concurrent world with network failures and vendors with errors in their own systems: it's nearly impossible for you to think really hard in your head and be sure that your queue retry strategy will maintain those properties that are important to your business. It's nearly impossible to do this with lightweight, informal testing strategies employed by busy software teams. By modelling your systems you will learn what the important failure modes are and you will get better at designing systems that are resilient and efficient. Card payment systems are fairly unreliable peer-to-peer messaging systems. Be prepared for a lot of complexity. Using an event-sourcing architecture is really useful here for that "auditing" requirement and for debugging transaction state when the network sends you messages in error, out of order, or they forget to retry themselves when they promised to, when merchants send bad data, when POS systems do weird things, etc.

phoehne 3y ago

RDMS databases are not a weak point. A bad model of your transaction is a weak point. Depending on the application, and not all applications require this, you may need a consistent read state for the data. Eventually consistent is like saying periodically wrong. Let's say a trader enters a trade for that exceeds there allowed VAR, because they looked at their screen thought they had more dry powder. (Cocaine jokes aside). That becomes a risk management problem. (Also the risk that traders can just use the 'It said I was below my limit' excuse). If you want an example where consistency is not important, you might be able to overdraw from your bank with your ATM card. The bank is happy for this to be inconsistent, since they can charge for the overdraft.

boblob-law 3y ago
The bank cannot charge for this unless you as the customer have opted in. See authorized positive settle negative. US based banks.

yafetn 3y ago

I’d also add: — If you’re using JSON to pass around monetary quantities (eg. from the frontend to the backend), put them in strings as opposed to the native number type. You never know what the serializers and deserializers across languages will do to your numbers (round them, truncate them etc.). — Start recording the currency of the transactions as early as possible. It can be a separate column in your table. — Use TIMESTAMPTZ. Always.

hirundo 3y ago
I'm generally not a fan of overloading fields, but we took the opportunity to add the currency code when we decided to encode monetary quantities in JSON as strings, e.g. "0.02USD". This has worked well, particularly because we use a money handling library that parses it unchanged.
- zeisss 3y ago
  Do you always put the currency behind the number? I know some countries prefer the currency infront (e.g. USD), others behind (EUR). But since this is not a human visible field, this is probably irrelevant, right?
  - hirundo 3y ago
    I think I looked at this when I laid it out: https://ux.stackexchange.com/questions/9105/international-currency-formatting-guidelines-currency-codes https://ux.stackexchange.com/questions/9105/international-cu... So we put the ISO code at the end and ignore whitespace. But including a prefix sign as with `€1.234.567,89 EUR` would break it.
    - hermitdev 3y ago
      You also included localized formatting (the '.' and ',' ) in your example that'll likely break on parsing without special handling. For interchange, you'll want to avoid using localized formats, e.g. stick to the "C" locale or something specific or agreed to between the involved parties.

gunshai 3y ago

Shit, I wish I read this prior to my interviews at Cash App.

neon_electro 3y ago
In what way do you think it would've made a difference? Asking as someone who's applied to Block/Square/Cash App in the past.
- gunshai 3y ago
  The difference is small, but mostly as probing questions to interviewers. I take the mind the more you can speak the language the more comfortable people become with you. Here's something small but I was doing a SQL interview one involved a a card_tx table with the amounts stored in integers values in cents. I immediately noticed this and chatted with the interviewer about this. My assumptions were wrong, and the floating point arithmetic was a simpler reason than what I was thinking. Anyway you can display you have thought about relevant business problems to where you are interview engineers tend to have a strong reflex to engage the person more.
  - neon_electro 3y ago
    Appreciate your reply!
    - gunshai 3y ago
      You're welcome, next time I'll try not to have a stroke at the end of my comment. hah. For some reason I can't edit it. shrug

gsatic 3y ago

And that's why I shall be sticking to cash for the next 10 years till all the beta testing is complete. By which time hopefully interest bearing CBDCs will show up, and make all these mindless intermediaries sitting between my wallet and someone elses wallet obsolete.

donatj 3y ago

I wish strictly INSERT-only tables were a standard SQL feature. It would be so very useful.

lixtra 3y ago
What’s the advantage of INSERT-only table over having a normal table and all normal users only have INSERT privilege on that table? As admin I can still recreate the whole insert table with modified data, etc.

dspillett 3y ago

> Over the last decade, fintechs using RDBMS databases to record transactions have built features onto their databases like a journal of all changes to all data and clever ways of making sure the data hasn’t been modified by storing checksums of data as transactions are added. (under “Updating transactions”) It is worth noting that many of these methods don't prevent tampering, they only make it visible when you look for it at which point the fact you are explicitly looking for it (rather than having been alerted to a potential issue) might imply it is too late.

bostik 3y ago
Which is also why auditors love git. The number one thing they crave is immutability. But when that's not an option, tamper-evident comes as a close second.

SideburnsOfDoom 3y ago

The recommendation that "developers should use integers to represent money" (with an implicit 2 decimal places, that will work for many but not all currencies) is not a great one. If your language has a dedicated type for monetary amounts, use that. (1) If it does not, but you can make a value object to represent, e.g. amount and currency code, then do that. If however, your language does not have a dedicated type for monetary amounts, or one cannot be trivially built or retrieved as a package (2), then you should ask yourself if it is really a suitable language for financial tasks. 1) https://learn.microsoft.com/en-us/dotnet/api/system.decimal https://learn.microsoft.com/en-us/dotnet/api/system.decimal 2) https://github.com/shopspring/decimal https://github.com/shopspring/decimal https://www.npmjs.com/package/ts-money https://www.npmjs.com/package/ts-money

gytisgreitai 3y ago
Stripe uses integers [1] Adyen uses integers [2] Square uses integers [3] So not sure that the industry would agree with you [1] https://stripe.com/docs/api/prices/create#create_price-unit_amount https://stripe.com/docs/api/prices/create#create_price-unit_... [2] https://docs.adyen.com/api-explorer/Checkout/70/post/payments#request-additionalAmount-value https://docs.adyen.com/api-explorer/Checkout/70/post/payment... [3] https://developer.squareup.com/reference/square/objects/Money https://developer.squareup.com/reference/square/objects/Mone...
- SideburnsOfDoom 3y ago
  > Stripe uses integers [1] a) On web api, so this is by definition data interchange, not "in a language". b) with a currency code to cross check it. c) implicitly "in cents" which needs further documentation - does this mean "pence" when the currency is GBP? How does this work for for JPY? BHD? d) cannot represent fractions of a cent or penny. So: this int plus currency code plus docs is OK but not great, a lowest common denominator format for interchange, requires further documentation and cross-checking as "100 USD" does not mean 100 bucks and the conversion is currency-specific, and cannot represent all values. I wouldn't refuse to convert _to and from_ this format at the edges of my application, for data interchange, but the conversion to something clearer and richer IMHO should remain there. Payment APIs are far from the only use case in fintech, for interest calculations you do have to care about fractions of a cent or penny. In fact, if your case is "I call the stripe api" ... are you sure you're a fintech and not an online store? Get back to me when you have to interop with FiServ, MasterCard or SAP.

itsthecourier 3y ago

I'm surprised the article doesn't explain how float numbers are stored in memory according to IEEE that's the main reason you don't use them to represent money. If you are an engineer who doesn't know what a mantissa is, stop two minutes for a cool ride https://lashewi.medium.com/storing-currency-values-and-float-precision-99029c1e8b71 https://lashewi.medium.com/storing-currency-values-and-float...

submeta 3y ago

Have built a fintech startup here in Germany ten years ago. The article mentions lots of important things. Here‘s another one: Time synchronization. It is incredibly important in fintech applications for a number of reasons: 1. *Transaction Ordering:* Financial transactions often need to be processed in the order they were initiated. This is especially crucial in high-frequency trading where trades are often made in milliseconds or microseconds. A small difference in timing could potentially lead to substantial financial gains or losses. Therefore, accurate time synchronization ensures fairness and order in the execution of these transactions. 2. *Security:* Accurate timekeeping helps in maintaining security. For instance, time-based one-time passwords (TOTPs) are widely used in two-factor authentication systems. These passwords are valid only for a short period of time and rely on synchronized clocks on the server and client side. 3. *Audit Trails and Dispute Resolution:* Timestamping transactions can help create a precise audit trail, which is critical for detecting and investigating fraudulent activities. In case of any dispute, a detailed and accurate transaction history backed by synchronized time can help resolve the issue. 4. *Distributed Systems:* In distributed systems, time synchronization is important to ensure data consistency. Many financial systems are distributed over different geographical locations, and transactions need to be coordinated between these systems in an orderly fashion. This requires all servers to have their clocks synchronized. I am sure there are even more fields where this is relevant.

jorangreef 3y ago
This was also a surprise to me when we started TigerBeetle. How clock synchronization protocols can experience unaligned network partitions, where the ledger database cluster is able to continue running, but now with the risk of unsynchronized clocks and far-future timestamps, which can in turn then lead to money being locked up in 2PC payment protocols. We therefore spent considerable effort [0] on clock synchronization in TigerBeetle, not for the consensus protocol—we never risk stale reads or take a chance with clock error bounds—but rather simply for accurate audit trails and to keep inflight liquidity from being locked up if transactions take too long to get rolled back. [0] https://tigerbeetle.com/blog/three-clocks-are-better-than-one/ https://tigerbeetle.com/blog/three-clocks-are-better-than-on...
troupo 3y ago

theptip 3y ago

I think this article covers many of the main points, but could be worded/structured better. There are terms of art that more precisely refer to the concepts you need. For “using floating point data types”, it’s even worse; you often need to use strings, for example if you need to store a bank account number “01234567789” will have the leading zero stripped if you use a numeric type. “Updating transactions” would be better phrased as “use an append-only log / evented architecture”. (Also, “use a double-entry ledger” is probably the most valuable advice I could have sent myself prior to getting into FinTech.) “Be careful with retry” should be more strictly “use idempotent operations” and link to the canonical Stripe article on idempotency keys (https://stripe.com/blog/idempotency https://stripe.com/blog/idempotency). Another important one to think about is bitemporality. “Created at” vs “effective at”. Not obvious at first and you’ll have some painful migrations if you don’t build it in. Fowler has a good overview here: https://martinfowler.com/eaaDev/timeNarrative.html https://martinfowler.com/eaaDev/timeNarrative.html. Edit to add - the advice that maybe using a NoSQL database is pretty bad IMO. I’d advise in the opposite direction - use SERIALIZABLE isolation in a SQL database. Read up on your Aphyr blog posts before trying to do anything distributed. Be paranoid about race conditions / serialization anomalies. If you eventually hit performance issues you need to think hard about what anomalies your access patterns might be subject to. (Obviously HFT won’t use serializable SQL).

ahtihn 3y ago
> if you need to store a bank account number “01234567789” will have the leading zero stripped if you use a numeric type. This goes for any "number" that is actually an identifier. Phone numbers aren't really numbers either for example. Rule of thumb: if it doesn't make sense to do math with it, it should probably be a numeric type unless there's a very good reason.

mhh__ 3y ago

Tip: you can express (city) time "zones" in the type system e.g. London time and New York time.

gpderetta 3y ago
You can, but because the full set if timezones is only known at runtime, it is not terribly useful. You can statically distinguish a canonical static timezone (say UTC) vs a dynamic ine though. Same for currency.
- mhh__ 3y ago
  There are only a few useful times in markets e.g. London, new York etc. If you don't know then there's no point but the idea is to stop you comparing prices from London close to new York and so on
  - gpderetta 3y ago
    I've gone as far as encoding bid and ask prices with different types, but tz really has never occurred to me. Closing times are also exchange/contract specific, not just location.

TeeWEE 3y ago

I really mis the mistake of NOT using double entry bookkeeping in the way you store transactions. Next to that, ensure transactions are immutable, and ensure reporting is idempotent: Close your periods and generate reports for them, when you regenerate the report it should be identical. This is more domain tips. A lot of technical tips hold for non fintech too.

acadavid79 3y ago

A little off-topic, but I would have loved to see mifos / fineract[0] referenced in the article. Great open source banking core having many of the strengths listed [0] https://github.com/apache/fineract/ https://github.com/apache/fineract/

hannofcart 3y ago

All good points.. One more from me: - Payment/order processing systems typically involve complex logic. - One of the best ways we managed to keep complexity in check was to model these as state machines (with the state itself being persisted to DB)

potamic 3y ago

#2 and #3 are pretty solid advice. But the rest of them are nothing specific to fintech. I will also offer a few additional points. 1. Never record an amount without its currency. 2. Reconcile all your data all the time. Never let any data go unaccounted for. 3. Maker-checker is a powerful concept. Embrace it to the fullest across your system. 4. You will be dealing with all sorts of non-standardized financial integrations. A lot. Think adapter pattern as early as possible. 5. You will be answering to multiple regulatory agencies. Create boundaries between them within your system and reduce the surface of compliance as much as possible.