10 ms·

ISO 8583: The language of credit cards

407 points by ekmartin 2y ago

adamdecaf 2y ago
We’ve had a lot of success with our Go library for iso8583 https://github.com/moov-io/iso8583 https://github.com/moov-io/iso8583
Rygian 2y ago
Fun times reviewing the masking logic of credit card data spewed out in system logs, in base64-encoded (or god forbid, EBCDIC-encoded base64-encoded) ISO 8583.
aftbit 2y ago
I wonder if this is the standard that drove Charles Stross slightly insane and led to Accelerando. https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html https://www.antipope.org/charlie/blog-static/fiction/acceler... Actually based on the timing, this is probably the new better standard that replaced the obscure protocols of the 70s.
t0mas88 2y ago
The type of protocol (message type, bitmap to define fields, followed by a set of fixed and variable length values) is pretty normal for the time it was developed in. Many low level things are basically packed C-structs with this type of protocol. It comes with some pitfalls on the receiver side to be careful validating dynamic field length and refusing to read past end of message or allocate an infinite buffer. But all of those are well understood by now. What I find baffling is that this "standard" does not specify how to encode the fields or even the message type. Pick anything, binary, ASCII, BCD, EBCDIC. That doesn't work as a standard, every implementation could send you nearly any random set of bytes with no way for the receiver to make sense of them.
- mananaysiempre 2y ago
  A bitmap to define field presence doesn’t seem so offensive, as far as serialization formats go. FlatBuffers[1] use a list of offsets instead, but that’s in the context of expecting to see many identically-shaped records. One could argue that Cap’n Proto with zero-packing[2] amounts to the same thing if you squint, just with the bitmap smeared all over the message. I mean, this specific thing sounds like it should have been a fatter but much more unexciting TLV affair instead. But given it’s from 1987, it’d probably have ended up as ASN.1 BER in that case (ETA: ah, and for extensions, it mostly did, what fun), instead of a simpler design like Protobufs or MessagePack/CBOR, so maybe the bitmaps are a blessing in disguise. [1] https://google.github.io/flatbuffers/flatbuffers_internals.html https://google.github.io/flatbuffers/flatbuffers_internals.h... [2] https://capnproto.org/encoding.html#packing https://capnproto.org/encoding.html#packing
  - lxgr 2y ago
    I'd trade the field layer of ISO 8583 for some ASN.1 any day! Luckily, there's a bit of everything in the archeological site that is ISO 8583, and field 55 (where EMV chip data goes, and EMV itself is quite ASN.1-heavy, presumably for historical reasons) and some others in fact contain something very close to it :)
  - grishka 2y ago
    Telegram's "TL" serialization, that's part of its network protocol, also uses a bitmap for optional fields. It's an okay protocol overall. The only problem is that the official documentation[1] was written by Nikolay Durov who is a mathematician. He just loves to overgeneralize everything to a ridiculous degree and spend two screens worth of outstandingly obtuse text to say what amounts to, for example, "some types have type IDs before them and some don't because the type is obvious form the schema". [1] https://core.telegram.org/mtproto https://core.telegram.org/mtproto
- lxgr 2y ago
  > Many low level things are basically packed C-structs with this type of protocol. Not really: C structs notably don't have variable-length fields, but ISO 8583 very much does. To add insult to injury, it does not offer a standard way to determine field lengths. This means that in order to ignore a given field, you'll need to be able to parse it (at least at the highest level)! Even ASN.1, not exactly the easiest format to deal with, is one step up from that (in a TLV structure, you can always skip unknown types by just skipping "length" bytes).
  - david-gpu 2y ago
    > Not really: C structs notably don't have variable-length fields Feast your eyes: C99 introduced an ~~abomination~~ feature called flexible array members (FAM), which allows the last member of a struct to be a variable length array. If you want to ~~gouge you eyes~~ learn more, see section 6.7.2.1.16 [0]. > To add insult to injury, it does not offer a standard way to determine field lengths That's awful. You can sort of say the same about variable length struts in C, but at least the strict tupe definition usually has a field that tell you the length of the variable length array at the end. [0] https://rgambord.github.io/c99-doc/sections/6/7/2/1/index.html#p16 https://rgambord.github.io/c99-doc/sections/6/7/2/1/index.ht...
    - j16sdiz 2y ago
      FAM was a (not so successful) attempt to standardise some existing usage
    - lxgr 2y ago
      > [...] feature called flexible array members (FAM), which allows the last member of a struct to be a variable length array. Oh, ISO 8583 has these too! Sometimes they're even combined with the "feature" described in the article where there's a variable number of fixed-length elements, except for the last element, which is a variable-length string (or sometimes the last field type repeated n times). That's always "fun" to work with. ISO 8583 really is a living museum of all ideas people had about binary encoding in the last half century or so.
- quotemstr 2y ago
  As far as I'm concerned, we solved binary formats with ASN.1 and its various encodings. Everything afterwards has been NIH, ignorance, and square wheel reinvention.
  - wbl 2y ago
    ASN.1 DER, BER, or OER? Implicit and optional can really break compat in surprising ways. Then there are the machine unfriendly roster of available types. XDR was more tuned for that. Finally free tooling doesn't really exist. The connection to the OSI model also didn't help.
    - quotemstr 2y ago
      > ASN.1 DER, BER, or OER? Or XER or JER! One of the brilliant things about ASN.1 is that it decouples the data model from the serialization format. Of the major successor systems, only protobuf does something similar, and the text proto format barely counts. > Implicit and optional can really break compat in surprising ways Any implementation of any spec can be broken. You could argue that the spec should be simpler and require, e.g., explicit tagging everywhere, like protobuf. Sure. But the complexity enables efficiencies, and it's sometimes worth making a foundational library a bit more complex to enable simplifications and optimizations throughout the ecosystem. > Then there are the machine unfriendly roster of available type Protobuf's variable-length integers are machine-friendly now? :-) We can always come up with better encoding rules without changing the fundamental data structures. > Finally free tooling doesn't really exist. What do you mean? You use ASN.1 to talk to every server talking SNMP, LDAP, or the X.509 PKI. Every programming environment has a way to talk about ASN.1. > The connection to the OSI model also didn't help. Agreed. The legacy string types aren't great either. You can, of course, do ASN.1 better. No technology is perfect. But what we don't need, IMHO, is more investment in "simple" technologies like varlink that end up being too simple and shunting complexity and schema-ness that belongs in a universal foundation layer into every single application using the new "simple" thing.
      - devman0 2y ago
        XML also decouples the data model and serialization with the XML Infoset specification.
- immibis 2y ago
  Something similar is TLV which is extremely common in binary network protocols, because it's very flexible for compatibility.
TacticalCoder 2y ago
> "ISO 8583: The language of credit cards" "ISO 8583: The language of both debit and credit cards"
- lxgr 2y ago
  And sometimes even bank transfers (I believe at least FPS in the UK used it, or possibly still does). Also don't forget about prepaid cards, charge cards etc., depending on where they exist in your personal ontology of card funding methods ;)
bokohut 2y ago
A lot of payments chatter on here recently and patio11 throwing out some great content as well. May I ask where this pretty visual explanation website was 25 years ago? ;) Oh the woes of programming ISO8583 as I see another commented on EBCDIC which adds in a whole other level of mind numbing when passing between the endians. It was a fun experience however back in the early 2000s when I worked in isolation with Discover card to get the GUID field added to the ISO8583 specification. We are living in changing times on many fronts and the worlds financial systems is one of those new battlefields. Many are ignorant as to what is occurring but with big tech owning their own payments ecosystems this should be insight for others not aware as we are absolutely certain to see more following their lead. Some of those others following are entire countries, they are just a bigger business after all, as it is already happening for those aware and a small select few are doing i.t. Stay Healthy!
- nonrandomstring 2y ago
  I learned a lot more about this discussing the PCI/DSS [0] regulation framework here [1]. It's about to change to a new 4.0 in 2025 which means that to use or run any payments system you'll have to meet ever more stringent regulation. This is going to start applying to other pseudo currencies (in game value tokens etc) if they exceed certain value and scale. At present Visa and Mastercard have a big stake in defining this (capturing the regulator). Interestingly local real (non-digital) currencies like the Brixton Pound [2] and other local paper scrip seem to escape this, which seems a boost for paper technologies. [0] https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Sec... [1] https://cybershow.uk/episodes.php?id=36 https://cybershow.uk/episodes.php?id=36 [2] https://brixtonpound.org/ https://brixtonpound.org/
  - lxgr 2y ago
    PCI-DSS is an industry standard, not a law. If you don't think it should apply to your domain, complain to your legislators/regulators, not the authors of PCI-DSS or the payment industry covered by it! > Interestingly local real (non-digital) currencies like the Brixton Pound [2] and other local paper scrip seem to escape this And so do countless other digital (non-real?) payment systems across the globe. That's not to say that there aren't any other security regulations, but they're also most certainly not in PCI scope. Arguably, the original sin of the card payments industry in particular, and US American banking in general, is treating account numbers as bearer tokens, i.e. secret information; if you don't do that, it turns out that a lot of things become much easier when it comes to security. (The industry has successfully transitioned of that way of doing things for card-present payments, but for card-absent, i.e. online, card transactions, the efforts weren't nearly as successful yet.)
krab 2y ago
Oh, this format was fun. You could see history unfold when parsing it. The messages I parsed were ISO-8583 with ~EBCDIC~ no, BCD. But one field contained XML. And the XML had an embedded JSON. The inner format matched the fashion trend of the year when someone had to extend the message with extra data. :-)
- ekmartin 2y ago
  Fascinating, I don't think I've ever seen an XML field! Do you remember which network that was for?
  - krab 2y ago
    We were the issuer. So these were probably the payment processor's extensions. But we were issuing MasterCards.
- lxgr 2y ago
  > The messages I parsed were ISO-8583 with ~EBCDIC~ no, BCD. The "great" thing about most ISO 8583 implementations I've worked with (all mutually incompatible at the basic syntactic level!) is that they usually freely mix EBCDIC, ASCII, BCD, UTF-8, and hexadecimal encoding across fields.
indus 2y ago
(In the holiday spirit) The only language of credit cards is points, cashback, APYs, and hard to read TOS
heywire 2y ago
It has been fun seeing all the different ways companies have come up with to work around the limitations of ISO 8583. One I’ve been seeing a lot lately is making an API call before/after the ISO message (with non-PCI data) to confer additional information outside of the payment transaction. Definitely speeds up time to market, but opens up a whole new array of failure modes to deal with.
ocf 2y ago
Neither Visa nor Mastercard really implement ISO 8583 a standardized way. Which means they each issue many thousands of pages of documentation covering not only which of the standard fields they use and how, but also how they cram their proprietary data into the messages. Most card management/issuance platforms do a decent job of abstracting this away though. Transition to ISO 20022 would be a positive improvement, but I don't think it will ever meet the required ROI threshold (globally) for that to happen.
- lxgr 2y ago
  The large card networks have so many proprietary behaviors and extensions that I really doubt whether any common standard would even make sense at this point. And if you look at how "modern" ISO 8583 is evolving, almost all changes and extensions are already happening in TLV-like subfields (where a new field unexpectedly appearing doesn't make existing parsers explode spectacularly), and the top-level structure is essentially irrelevant. Of course, it's a significant hurdle to newcomers to get familiar with that outer layer, but I don't get the sense that catering to these is a particular focus by either network. ISO 8583 is also a great moat (one of many in the industry, really) for existing processors, which have no buy-in to switch to a new standard and the networks need to at least somewhat keep happy.
  - throwway120385 2y ago
    I thought that chip-in EMV was bad until I saw some of the stuff coming out of Discover cards for contactless EMV. Buying a test card set from somewhere like B2 Systems was very beneficial even just integrating an EMV reader from a hardware device to a payment processor.
    - lxgr 2y ago
      The problem is that the contactless stuff is all custom per network. Some of the implementations are reasonably close to contact EMV; others might as well be a completely different stack and technology.
  - BiteCode_dev 2y ago
TuringNYC 2y ago
Unlike Visa and Mastercard, I noticed that AMEX transaction notifications are near-instantaneous. There is something so magical about a notification popping up on my phone/watch literally the second i swipe a card. I always wondered about the layers on the stack which V/MC must have which AMEX doesnt.
- reaperducer 2y ago
  Unlike Visa and Mastercard, I noticed that AMEX transaction notifications are near-instantaneous. There is something so magical about a notification popping up on my phone/watch literally the second i swipe a card. I always wondered about the layers on the stack which V/MC must have which AMEX doesnt. Must be your bank, because both my Visa and MasterCard ping my phone instantaneously, too.
  - TuringNYC 2y ago
    >> Must be your bank, because both my Visa and MasterCard ping my phone instantaneously, too. Well thats sort of the thing...with Visa and MC, there is an extra layer or two of the bank or Fidelity Information Services. With Amex, they own the full stack end to end.
    - akira2501 2y ago
      Your card limit gets checked on every transaction. There doesn't seem to be a technical reason why information flow back to me should be limited in any way. If the extra layer fails to work the transaction fails to pass.
      - mschuster91 2y ago
        > Your card limit gets checked on every transaction. Nope. The merchant can choose the level of verification - in some cases, like copying the card with an imprinter [1] or running phone transactions (yes, that is possible - it's called MOTO [2]), it's obviously impossible to check card limits. Downside of CNP transactions is, the merchant is fully liable for anything from fraud over chargebacks to exceeding limits. And then you got card-present transactions but the network connectivity is down for whatever reasons... been a while since I messed with that, but at least for German cards you could configure the terminal to store the account details for later submission when connectivity was restored. [1] https://en.wikipedia.org/wiki/Credit_card_imprinter https://en.wikipedia.org/wiki/Credit_card_imprinter [2] https://docs.adyen.com/point-of-sale/mail-and-telephone-order-moto/ https://docs.adyen.com/point-of-sale/mail-and-telephone-orde...
rswail 2y ago
Great article that exposes why ISO20022 will replace 8583 over time, especially in areas not dominated by the M/V monopoly networks. Credit cards, with all their nonsense about cash backs and rewards can be imnplemented in the new payment systems with banks offering line of credit accounts that are part of the appropriate "national payment system", like UPI, PromptPay, Osko/PayID, FedNow etc. Instant settlement between accounts, low cost fixed price txns etc.
- sirn 2y ago
  Fun anecdote: Thailand's entire banking network (including regular wire transfer) was implemented with ISO 8583 (!). Part of the AnyID master plan (later renamed to PromptPay) was to replace the country's (ab)usage of ISO 8583 with ISO 20022. The Ministry of Finance hired a UK-based Vocalink to build this converter, among with other systems MoF hired them for. (AFAIK, the entire stack was written in Erlang.)
Copyrighted 2y ago
I got my last company certified with Visa and Mastercard for authorization and clearing. It's funny how they call it a standard but it's anything but that. There were some similarities but a lot of subtle differences which made the process 10X harder. Mastercard was the worst to deal with.
Copyrighted 2y ago
I really wonder what a future would look like without companies like Visa/Mastercard/Discover/AMEX.
Copyrighted 2y ago
So is ISO20022 the future then? There should* be a standard system that all the networks stick to... instead of the hodgepodge of bullshit there is now.
roordan 2y ago
An interesting side effect of this low-level bit mapping is that various banks authorization logics can be manipulated to increase auth rates by subtle bit flipping across various fields. All the big fintech companies have ML running over changes to identify what results in the highest auth rates on a per bin basis.
- mainde 2y ago
  I would be very surprised if bit flipping and ML were really used here, do you have any source? While for sure there's a lot of signal and value in monitoring auth rates per BIN per payload, flipping bits can be extremely disruptive and counterproductive. From doing the wrong operation to being fined by the schemes, it's a lot of risk for not a lot of gain when these fields can be tuned ad-hoc for the few card issuers that deviate from the standard/norm.
tibbydudeza 2y ago
Not only payment cards but value added services as well like prepaid airtime and bill payments and ISO8583 1993 or the 1987 version ???.

407 points by ekmartin 2y ago

We’ve had a lot of success with our Go library for iso8583 https://github.com/moov-io/iso8583 https://github.com/moov-io/iso8583

Rygian 2y ago

Fun times reviewing the masking logic of credit card data spewed out in system logs, in base64-encoded (or god forbid, EBCDIC-encoded base64-encoded) ISO 8583.

aftbit 2y ago

I wonder if this is the standard that drove Charles Stross slightly insane and led to Accelerando. https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html https://www.antipope.org/charlie/blog-static/fiction/acceler... Actually based on the timing, this is probably the new better standard that replaced the obscure protocols of the 70s.

t0mas88 2y ago

The type of protocol (message type, bitmap to define fields, followed by a set of fixed and variable length values) is pretty normal for the time it was developed in. Many low level things are basically packed C-structs with this type of protocol. It comes with some pitfalls on the receiver side to be careful validating dynamic field length and refusing to read past end of message or allocate an infinite buffer. But all of those are well understood by now. What I find baffling is that this "standard" does not specify how to encode the fields or even the message type. Pick anything, binary, ASCII, BCD, EBCDIC. That doesn't work as a standard, every implementation could send you nearly any random set of bytes with no way for the receiver to make sense of them.

mananaysiempre 2y ago
A bitmap to define field presence doesn’t seem so offensive, as far as serialization formats go. FlatBuffers[1] use a list of offsets instead, but that’s in the context of expecting to see many identically-shaped records. One could argue that Cap’n Proto with zero-packing[2] amounts to the same thing if you squint, just with the bitmap smeared all over the message. I mean, this specific thing sounds like it should have been a fatter but much more unexciting TLV affair instead. But given it’s from 1987, it’d probably have ended up as ASN.1 BER in that case (ETA: ah, and for extensions, it mostly did, what fun), instead of a simpler design like Protobufs or MessagePack/CBOR, so maybe the bitmaps are a blessing in disguise. [1] https://google.github.io/flatbuffers/flatbuffers_internals.html https://google.github.io/flatbuffers/flatbuffers_internals.h... [2] https://capnproto.org/encoding.html#packing https://capnproto.org/encoding.html#packing
- lxgr 2y ago
  I'd trade the field layer of ISO 8583 for some ASN.1 any day! Luckily, there's a bit of everything in the archeological site that is ISO 8583, and field 55 (where EMV chip data goes, and EMV itself is quite ASN.1-heavy, presumably for historical reasons) and some others in fact contain something very close to it :)
- grishka 2y ago
  Telegram's "TL" serialization, that's part of its network protocol, also uses a bitmap for optional fields. It's an okay protocol overall. The only problem is that the official documentation[1] was written by Nikolay Durov who is a mathematician. He just loves to overgeneralize everything to a ridiculous degree and spend two screens worth of outstandingly obtuse text to say what amounts to, for example, "some types have type IDs before them and some don't because the type is obvious form the schema". [1] https://core.telegram.org/mtproto https://core.telegram.org/mtproto
lxgr 2y ago
> Many low level things are basically packed C-structs with this type of protocol. Not really: C structs notably don't have variable-length fields, but ISO 8583 very much does. To add insult to injury, it does not offer a standard way to determine field lengths. This means that in order to ignore a given field, you'll need to be able to parse it (at least at the highest level)! Even ASN.1, not exactly the easiest format to deal with, is one step up from that (in a TLV structure, you can always skip unknown types by just skipping "length" bytes).
- david-gpu 2y ago
  > Not really: C structs notably don't have variable-length fields Feast your eyes: C99 introduced an ~~abomination~~ feature called flexible array members (FAM), which allows the last member of a struct to be a variable length array. If you want to ~~gouge you eyes~~ learn more, see section 6.7.2.1.16 [0]. > To add insult to injury, it does not offer a standard way to determine field lengths That's awful. You can sort of say the same about variable length struts in C, but at least the strict tupe definition usually has a field that tell you the length of the variable length array at the end. [0] https://rgambord.github.io/c99-doc/sections/6/7/2/1/index.html#p16 https://rgambord.github.io/c99-doc/sections/6/7/2/1/index.ht...
  - j16sdiz 2y ago
    FAM was a (not so successful) attempt to standardise some existing usage
  - lxgr 2y ago
    > [...] feature called flexible array members (FAM), which allows the last member of a struct to be a variable length array. Oh, ISO 8583 has these too! Sometimes they're even combined with the "feature" described in the article where there's a variable number of fixed-length elements, except for the last element, which is a variable-length string (or sometimes the last field type repeated n times). That's always "fun" to work with. ISO 8583 really is a living museum of all ideas people had about binary encoding in the last half century or so.
quotemstr 2y ago
As far as I'm concerned, we solved binary formats with ASN.1 and its various encodings. Everything afterwards has been NIH, ignorance, and square wheel reinvention.
- wbl 2y ago
  ASN.1 DER, BER, or OER? Implicit and optional can really break compat in surprising ways. Then there are the machine unfriendly roster of available types. XDR was more tuned for that. Finally free tooling doesn't really exist. The connection to the OSI model also didn't help.
  - quotemstr 2y ago
    > ASN.1 DER, BER, or OER? Or XER or JER! One of the brilliant things about ASN.1 is that it decouples the data model from the serialization format. Of the major successor systems, only protobuf does something similar, and the text proto format barely counts. > Implicit and optional can really break compat in surprising ways Any implementation of any spec can be broken. You could argue that the spec should be simpler and require, e.g., explicit tagging everywhere, like protobuf. Sure. But the complexity enables efficiencies, and it's sometimes worth making a foundational library a bit more complex to enable simplifications and optimizations throughout the ecosystem. > Then there are the machine unfriendly roster of available type Protobuf's variable-length integers are machine-friendly now? :-) We can always come up with better encoding rules without changing the fundamental data structures. > Finally free tooling doesn't really exist. What do you mean? You use ASN.1 to talk to every server talking SNMP, LDAP, or the X.509 PKI. Every programming environment has a way to talk about ASN.1. > The connection to the OSI model also didn't help. Agreed. The legacy string types aren't great either. You can, of course, do ASN.1 better. No technology is perfect. But what we don't need, IMHO, is more investment in "simple" technologies like varlink that end up being too simple and shunting complexity and schema-ness that belongs in a universal foundation layer into every single application using the new "simple" thing.
    - devman0 2y ago
      XML also decouples the data model and serialization with the XML Infoset specification.
immibis 2y ago
Something similar is TLV which is extremely common in binary network protocols, because it's very flexible for compatibility.

TacticalCoder 2y ago

> "ISO 8583: The language of credit cards" "ISO 8583: The language of both debit and credit cards"

lxgr 2y ago
And sometimes even bank transfers (I believe at least FPS in the UK used it, or possibly still does). Also don't forget about prepaid cards, charge cards etc., depending on where they exist in your personal ontology of card funding methods ;)

bokohut 2y ago

A lot of payments chatter on here recently and patio11 throwing out some great content as well. May I ask where this pretty visual explanation website was 25 years ago? ;) Oh the woes of programming ISO8583 as I see another commented on EBCDIC which adds in a whole other level of mind numbing when passing between the endians. It was a fun experience however back in the early 2000s when I worked in isolation with Discover card to get the GUID field added to the ISO8583 specification. We are living in changing times on many fronts and the worlds financial systems is one of those new battlefields. Many are ignorant as to what is occurring but with big tech owning their own payments ecosystems this should be insight for others not aware as we are absolutely certain to see more following their lead. Some of those others following are entire countries, they are just a bigger business after all, as it is already happening for those aware and a small select few are doing i.t. Stay Healthy!

nonrandomstring 2y ago
I learned a lot more about this discussing the PCI/DSS [0] regulation framework here [1]. It's about to change to a new 4.0 in 2025 which means that to use or run any payments system you'll have to meet ever more stringent regulation. This is going to start applying to other pseudo currencies (in game value tokens etc) if they exceed certain value and scale. At present Visa and Mastercard have a big stake in defining this (capturing the regulator). Interestingly local real (non-digital) currencies like the Brixton Pound [2] and other local paper scrip seem to escape this, which seems a boost for paper technologies. [0] https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Sec... [1] https://cybershow.uk/episodes.php?id=36 https://cybershow.uk/episodes.php?id=36 [2] https://brixtonpound.org/ https://brixtonpound.org/
- lxgr 2y ago
  PCI-DSS is an industry standard, not a law. If you don't think it should apply to your domain, complain to your legislators/regulators, not the authors of PCI-DSS or the payment industry covered by it! > Interestingly local real (non-digital) currencies like the Brixton Pound [2] and other local paper scrip seem to escape this And so do countless other digital (non-real?) payment systems across the globe. That's not to say that there aren't any other security regulations, but they're also most certainly not in PCI scope. Arguably, the original sin of the card payments industry in particular, and US American banking in general, is treating account numbers as bearer tokens, i.e. secret information; if you don't do that, it turns out that a lot of things become much easier when it comes to security. (The industry has successfully transitioned of that way of doing things for card-present payments, but for card-absent, i.e. online, card transactions, the efforts weren't nearly as successful yet.)

krab 2y ago

Oh, this format was fun. You could see history unfold when parsing it. The messages I parsed were ISO-8583 with ~EBCDIC~ no, BCD. But one field contained XML. And the XML had an embedded JSON. The inner format matched the fashion trend of the year when someone had to extend the message with extra data. :-)

ekmartin 2y ago
Fascinating, I don't think I've ever seen an XML field! Do you remember which network that was for?
- krab 2y ago
  We were the issuer. So these were probably the payment processor's extensions. But we were issuing MasterCards.
lxgr 2y ago
> The messages I parsed were ISO-8583 with ~EBCDIC~ no, BCD. The "great" thing about most ISO 8583 implementations I've worked with (all mutually incompatible at the basic syntactic level!) is that they usually freely mix EBCDIC, ASCII, BCD, UTF-8, and hexadecimal encoding across fields.

indus 2y ago

(In the holiday spirit) The only language of credit cards is points, cashback, APYs, and hard to read TOS

heywire 2y ago

It has been fun seeing all the different ways companies have come up with to work around the limitations of ISO 8583. One I’ve been seeing a lot lately is making an API call before/after the ISO message (with non-PCI data) to confer additional information outside of the payment transaction. Definitely speeds up time to market, but opens up a whole new array of failure modes to deal with.

ocf 2y ago

Neither Visa nor Mastercard really implement ISO 8583 a standardized way. Which means they each issue many thousands of pages of documentation covering not only which of the standard fields they use and how, but also how they cram their proprietary data into the messages. Most card management/issuance platforms do a decent job of abstracting this away though. Transition to ISO 20022 would be a positive improvement, but I don't think it will ever meet the required ROI threshold (globally) for that to happen.

lxgr 2y ago
The large card networks have so many proprietary behaviors and extensions that I really doubt whether any common standard would even make sense at this point. And if you look at how "modern" ISO 8583 is evolving, almost all changes and extensions are already happening in TLV-like subfields (where a new field unexpectedly appearing doesn't make existing parsers explode spectacularly), and the top-level structure is essentially irrelevant. Of course, it's a significant hurdle to newcomers to get familiar with that outer layer, but I don't get the sense that catering to these is a particular focus by either network. ISO 8583 is also a great moat (one of many in the industry, really) for existing processors, which have no buy-in to switch to a new standard and the networks need to at least somewhat keep happy.
- throwway120385 2y ago
  I thought that chip-in EMV was bad until I saw some of the stuff coming out of Discover cards for contactless EMV. Buying a test card set from somewhere like B2 Systems was very beneficial even just integrating an EMV reader from a hardware device to a payment processor.
  - lxgr 2y ago
    The problem is that the contactless stuff is all custom per network. Some of the implementations are reasonably close to contact EMV; others might as well be a completely different stack and technology.
- BiteCode_dev 2y ago

TuringNYC 2y ago

Unlike Visa and Mastercard, I noticed that AMEX transaction notifications are near-instantaneous. There is something so magical about a notification popping up on my phone/watch literally the second i swipe a card. I always wondered about the layers on the stack which V/MC must have which AMEX doesnt.

reaperducer 2y ago
Unlike Visa and Mastercard, I noticed that AMEX transaction notifications are near-instantaneous. There is something so magical about a notification popping up on my phone/watch literally the second i swipe a card. I always wondered about the layers on the stack which V/MC must have which AMEX doesnt. Must be your bank, because both my Visa and MasterCard ping my phone instantaneously, too.
- TuringNYC 2y ago
  >> Must be your bank, because both my Visa and MasterCard ping my phone instantaneously, too. Well thats sort of the thing...with Visa and MC, there is an extra layer or two of the bank or Fidelity Information Services. With Amex, they own the full stack end to end.
  - akira2501 2y ago
    Your card limit gets checked on every transaction. There doesn't seem to be a technical reason why information flow back to me should be limited in any way. If the extra layer fails to work the transaction fails to pass.
    - mschuster91 2y ago
      > Your card limit gets checked on every transaction. Nope. The merchant can choose the level of verification - in some cases, like copying the card with an imprinter [1] or running phone transactions (yes, that is possible - it's called MOTO [2]), it's obviously impossible to check card limits. Downside of CNP transactions is, the merchant is fully liable for anything from fraud over chargebacks to exceeding limits. And then you got card-present transactions but the network connectivity is down for whatever reasons... been a while since I messed with that, but at least for German cards you could configure the terminal to store the account details for later submission when connectivity was restored. [1] https://en.wikipedia.org/wiki/Credit_card_imprinter https://en.wikipedia.org/wiki/Credit_card_imprinter [2] https://docs.adyen.com/point-of-sale/mail-and-telephone-order-moto/ https://docs.adyen.com/point-of-sale/mail-and-telephone-orde...

rswail 2y ago

Great article that exposes why ISO20022 will replace 8583 over time, especially in areas not dominated by the M/V monopoly networks. Credit cards, with all their nonsense about cash backs and rewards can be imnplemented in the new payment systems with banks offering line of credit accounts that are part of the appropriate "national payment system", like UPI, PromptPay, Osko/PayID, FedNow etc. Instant settlement between accounts, low cost fixed price txns etc.

sirn 2y ago
Fun anecdote: Thailand's entire banking network (including regular wire transfer) was implemented with ISO 8583 (!). Part of the AnyID master plan (later renamed to PromptPay) was to replace the country's (ab)usage of ISO 8583 with ISO 20022. The Ministry of Finance hired a UK-based Vocalink to build this converter, among with other systems MoF hired them for. (AFAIK, the entire stack was written in Erlang.)

Copyrighted 2y ago

I got my last company certified with Visa and Mastercard for authorization and clearing. It's funny how they call it a standard but it's anything but that. There were some similarities but a lot of subtle differences which made the process 10X harder. Mastercard was the worst to deal with.

Copyrighted 2y ago

I really wonder what a future would look like without companies like Visa/Mastercard/Discover/AMEX.

Copyrighted 2y ago

So is ISO20022 the future then? There should* be a standard system that all the networks stick to... instead of the hodgepodge of bullshit there is now.

roordan 2y ago

An interesting side effect of this low-level bit mapping is that various banks authorization logics can be manipulated to increase auth rates by subtle bit flipping across various fields. All the big fintech companies have ML running over changes to identify what results in the highest auth rates on a per bin basis.

mainde 2y ago
I would be very surprised if bit flipping and ML were really used here, do you have any source? While for sure there's a lot of signal and value in monitoring auth rates per BIN per payload, flipping bits can be extremely disruptive and counterproductive. From doing the wrong operation to being fined by the schemes, it's a lot of risk for not a lot of gain when these fields can be tuned ad-hoc for the few card issuers that deviate from the standard/norm.

tibbydudeza 2y ago

Not only payment cards but value added services as well like prepaid airtime and bill payments and ISO8583 1993 or the 1987 version ???.