5 ms·

Show HN: Numscript, a declarative language to model financial transactions

202 points by superzamp 2y ago

Numscript is a simple, declarative language that helps you model financial transactions. You can do quite a few things with it, such as modeling: * Payments involving vouchers and a user's prepaid balance * Complex funds destination scenario where the customer gets cash back * Configurable user credit balance spending transactions The main idea is to take the pain out of describing of a system dealing with money movements should behave in traditional languages such as JS/TS/Go/Ruby etc, landing an expressive way to model these movements of value. It is voluntarily broad in applicability—our customers use it today for use-cases ranging from marketplaces funds orchestration to home-grown loan management systems. Once those transactions are modeled, they are to be picked up and committed to a system-of-record, ledgering system or executed on a set of payments and banking APIs. It was initially a DSL we bundled into our Core Ledger product at Formance (YCS21) but we're giving it more autonomy now and started to make it standalone, with the idea that anyone could eventually bolt Numscript on top of their ledgering system. We're also exploring to make it natively compatible with other backends. As part of this un-bundling, we've just shipping a playground which lets you fiddle with it without installing anything at https://playground.numscript.org/ https://playground.numscript.org/ (it works best on desktop). Happy to chime in on any questions!

OutOfHere 2y ago
Can this or Formance interact with credit card systems, checking accounts, and layer 2 crypto wallets? If not, how is the money even going to come in or out?
robk 2y ago
This seems cool but what's the intention with the USD/2 notation? Is that for fractional sub-cent precision in rounding situations?
- superzamp 2y ago
  It's indeed relative to cents in a sense, the idea is to force you to declare the precision of the monetary amount you're expressing. You can see various interpretation of what "USD" means in the wild, as some APIs will happily parse USD 100 as $1.00 while some others might parse USD 100 as $100.00. So we recommend this explicit [ASSET/SCALE AMOUNT] notation, where SCALE describes the negative power of ten to multiply the AMOUNT with to obtain the decimal value in the given ASSET. It makes subsequent interaction with external systems much easier. You can read a bit more about it here [1]. [1] https://docs.formance.com/stack/unambiguous-monetary-notation https://docs.formance.com/stack/unambiguous-monetary-notatio...
  - flockonus 2y ago
    What about units that cost sub-cent? For examples, I've seen private company stocks being $0.0001 Not sure why you'd need to make a note in the internal representation, vs. leave adapters to handle conversions.
    - bbor 2y ago
      Not to speak for them, but I think you’ve understood the point exactly. You need to be able to support arbitrary precision, but that support needs to be intentional to avoid errors. And you have to record that decision somewhere if adapters are to correctly handle your outputs; why not in the unit name?
    - petesergeant 2y ago
      If I understand their docs correctly, that’s equivalent to [USD/4 1]
  - tway_GdBRwW 2y ago
    Used to work at a payments company. Yes, you need to be *very* explicit in how you model currency amounts and precision. See also earlier post about Canadian rounding rules. Some of the "logic" is regulatory/compliance driven. ref child post about stocks trading for 0.0001. Yes, those are real trades and (probably) fully legal etc, but I'm not sure the Fed recognizes currency amounts less than 1 US cent ($0.01), so the accounting rules and tax rules might not match expectations based on generalized floating point math.
    - euroderf 2y ago
      Doesn't the Fed recognise mills ? They are not extinct.
  - girvo 2y ago
    Just to add to this, having also implemented a production payment system, we did the same thing. One needs to be very explicit about the scale and how it should be rounded and dealt with, and how to operate on two different scale systems. Quite a fun challenge, though I do not miss the edge cases. Our system was a payment system for childcare management software, interfacing with banks and the government directly.
vslira 2y ago
Super interesting and thanks for sharing. If I understood the license page on the repository correctly, the DSL is MIT-licensed since it's not within the enterprise directory, right?
- superzamp 2y ago
  Yes exactly! There's actually two implementations, one tightly knit to our ledger product located at [1] and the new, standalone one (used by the playground) at [2]. In any case, in both implementations, the DSL is indeed MIT. [1] https://github.com/formancehq/stack/tree/main/components/ledger/internal/machine https://github.com/formancehq/stack/tree/main/components/led... [2] https://github.com/formancehq/numscript https://github.com/formancehq/numscript
bbor 2y ago
To this noob, this seems like a problem that was pretty convincingly solved by double entry accounting ledgers, but from your post it sounds like this isn’t a replacement-of but rather an addition-to that model. What’s a situation where an imperative approach would be preferable to the traditional declarative approach? My depth of knowledge pretty much starts and ends with the below document, so apologies if this is obvious to experts! https://beancount.github.io/docs/the_double_entry_counting_method.html#introduction https://beancount.github.io/docs/the_double_entry_counting_m... E: the problem being “tracking transactions”. Yes?
- superzamp 2y ago
  That's a very good question. So the DSL here operates an agnostic source/dest transaction model, which is akin to the credit/debit model sans the semantic baggage. The goal of this model is indeed to be "tracking transactions" in the abstract sense, having the benefit of not forcing accounting decisions too early on when there is (still yet) none. For example, if you create a transaction moving money from "@stripe:main" to "@acct:123" and "@acct:234", you're merely representing the fact that you want this money to be moved. Wether the movement is clearing off a liability or generating revenue is another concern that you (in our model) want to take care of in a separate layer, that will also likely involve some intense intentionality and iterations from your accounting team. In a sense, it's as close to accounting than it is to warehousing money, moving unitary boxes of it from one location to another. These two models have the same amount of information per entry, so they can actually be converted from one to another, enabling you to also represent some accounting-ish transactions with this DSL, e.g. with a send [USD/2 100] from @ar:invoices:1234 to @sales.
  - bbor 2y ago
    Fascinating, thanks for taking the time to educate! Makes sense to me — it seems this tool is purpose built for situations where tracking is complex enough to deserve decoupling from annotating or interpreting, to put it in my own kindergarten terms. I don’t have a need for this personally but I’ll definitely be bouncing this around in my head for a while, both technically (JS ALL the things!) and methodologically. Accounting is pretty trivial when you don’t have any income to track!
abdullahkhalids 2y ago
How would this handle Canadian guidelines for dealing with cents in cash, where you round to the nearest 5c [1]? Amounts ending in 1 cent and 2 cents are rounded down to the nearest 10 cents; Amounts ending in 3 cents and 4 cents are rounded up to the nearest 5 cents; Amounts ending in 6 cents and 7 cents are rounded down to the nearest 5 cents; Amounts ending in 8 cents and 9 cents are rounded up to the nearest 10 cents; Amounts ending in 0 cent and 5 cents remain unchanged. EDIT: I think if you send 12 cents, send [CADCASH/2 12] ( source = @user1 destination = @user2 ) It should result in sending 10 cents. "postings": [{"source": "user1", "destination": "user2", "amount": 10, "asset": "CADCASH/2"}] Can this happen? [1] https://www.canada.ca/en/revenue-agency/programs/about-canada-revenue-agency-cra/phasing-penny.html https://www.canada.ca/en/revenue-agency/programs/about-canad...
- superzamp 2y ago
  Well that's definitely a good puzzle. I've tried to model it for a bit, but it indeed looks like we'd need to add something to the language to make it possible at all! Thanks for bringing this up.
  - fallat 2y ago
    Same thing with income tax brackets. You need conditional logic.
    - superzamp 2y ago
      For this particular case, I would say that tax-brackets sort of logic can be expressed in the destination block with ordered destinations. For example, you could have something like this: send [USD/2 *] ( source = @users:1234 destination = { // first $1000 are taxed at 10% max [USD/2 100000] to { 10% to @taxes remaining kept } // Anything above that is, taxed at 20% remaining to { 20% to @taxes remaining kept } } ) (You can test it on the playground, you'll just want to feed the "users:1234" account with an initial balance in the input section)
kamalamomala 2y ago
[flagged]
jimbokun 2y ago
Are there converters to/from OFX for Numscript?
- superzamp 2y ago
  Not as of today!
henning 2y ago
Ah, what better input format than JSON, a poorly defined, ambiguous format that freely mixes integers and floating points and lacks supports for bigints and bigdecimals.
- darby_nine 2y ago
  It's only ambiguous if you let it be. The rest is just implementation details.
- monero-xmr 2y ago
  In cryptocurrency the decimals are often a range between 8 and 18 decimals, sometimes much larger. The solution is always to use strings, and have separate fields or data models that explain the precision. The amount of languages and libraries, for tradfi or anything, is so wide that actually trying to use something other than strings for passing data between systems and applications would be self destructive
bschmidt1 2y ago
This is mindblowing. It's like the missing piece of blockchain that was needed for dev adoption. I can see it also being used for modern bank/payments apps including those in games and other simulated economies. Now let's see a "ContentScript" for posting content to ledgers :D send [Image "tacos.jpg", Text "taco 2sday!"] ( source = @bschmidt1 destination = @us-west-relay )
gregwebs 2y ago
The CLI is only displaying a "check" command but I did figure out that there is a "run" command as well. go install github.com/formancehq/numscript@latest go: downloading github.com/formancehq/numscript v0.0.8 go: downloading github.com/antlr4-go/antlr/v4 v4.13.1 go: downloading github.com/sourcegraph/jsonrpc2 v0.2.0 go: downloading golang.org/x/exp v0.0.0-20240707233637-46b078467d37 numscript -h Numscript cli Usage: numscript [command] Available Commands: check Check a numscript file help Help about any command Flags: -h, --help help for numscript -v, --version version for numscript Use "numscript [command] --help" for more information about a command.
- ascandone 2y ago
  Hello, I'm one of the Numscript devs. The Numscript CLI does have a `numscript run` command, which is hidden for now but will be released in the following days. It behaves in the same way as the playground, and you can take a look at it with `numscript run --help` command
adius 2y ago
Nice! I tried to achieve a similar design with my plain text accounting tool Transity: https://github.com/ad-si/Transity https://github.com/ad-si/Transity But this seems to have more programming language like features if I understand correctly. What else is different?
- hum3hum3 2y ago
  You might like https://martin.kleppmann.com/2011/03/07/accounting-for-computer-scientists.html https://martin.kleppmann.com/2011/03/07/accounting-for-compu... which shows it as a directed graph.
holloway 2y ago
Large sends seem to crash the playground with a JSON parse error. eg change the first example 'Simple send' to send `10000000000000000000` (`send [USD/2 10000000000000000000]`) and there's a crash
- superzamp 2y ago
  Thanks for the heads up! While the Numscript backend itself uses big.Int, we're still using normal javascript numbers in the playground for now so that's where it's coming from. But we should definitely switch to BigInt in the playground code though and that would solve the issue.
  - hum3hum3 2y ago
    You should never use floats for money. In python Decimal works well. TigerBeatle uses 128 bit accounts and amounts which I thought was interesting although more than needed gor many cases.
neilwilson 2y ago
How do you model a bank making a loan, where the bank creates the money out of nothing and the accounting appears backwards? The equivalent of DR @my_bank:loans_made 10000, CR @my_bank:1234 10000
- superzamp 2y ago
  Yeah that's a good question, you'll want to leverage the `overdraft` functionality for that, enabling the account end balance to go below zero. So you could do something like: send [USD/2 10000] ( source = @my_bank:loans_made allowing unbounded overdraft destination = @my_bank:1234 ) With the post-transaction balance of @my_bank:loans_made becoming [USD/2 -10000]. So that's for creating money out of nothing. There's a little bit more to it if you want to create accounting-perfect entries, but a simple way to map src/dest to cr/dr entries is to say that every credit becomes a destination, and every debit becomes a source. You can then consider debit normal accounts as having their debit balance being equivalent to the sum of entries where the account is source, their credit balance as the sum of entries where the account is destination—and do the opposite for credit normal accounts. We've written a bit more about it here [1]. [1] https://docs.formance.com/ledger/advanced/accounting/credit-debit-to-src-dest https://docs.formance.com/ledger/advanced/accounting/credit-...
  - neilwilson 2y ago
    That’s helpful. How about Nostro and Vostro accounts used as the balancing item with sub-ledgers like agency banks. (In essence memorandum accounts on a control account) How would you handle a three legged agency bank transaction as you get in a BACS file (source, Nostro, destination).
  - neilwilson 2y ago
    That’s helpful. How about Nostro and Vostro accounts used as the balancing item with sub-ledgers like agency banks. (In essence memorandum accounts on a control account) How would you handle a three legged agency bank transaction as you get in a BACS file (source, Nostro, destination)?
throwaway81523 2y ago
This is a pretty neat paper about a Haskell DSL for specifying financial contracts and calculating their values: https://www.microsoft.com/en-us/research/publication/composing-contracts-an-adventure-in-financial-engineering/ https://www.microsoft.com/en-us/research/publication/composi... It is quite powerful and you might want to look at it for possible features to use.
- pjacotg 2y ago
  Nice! I've worked with a few similar real world payoff languages. This paper discusses one of them: https://www.infoq.com/presentations/haskell-barclays/ https://www.infoq.com/presentations/haskell-barclays/

202 points by superzamp 2y ago

OutOfHere 2y ago

Can this or Formance interact with credit card systems, checking accounts, and layer 2 crypto wallets? If not, how is the money even going to come in or out?

robk 2y ago

This seems cool but what's the intention with the USD/2 notation? Is that for fractional sub-cent precision in rounding situations?

superzamp 2y ago
It's indeed relative to cents in a sense, the idea is to force you to declare the precision of the monetary amount you're expressing. You can see various interpretation of what "USD" means in the wild, as some APIs will happily parse USD 100 as $1.00 while some others might parse USD 100 as $100.00. So we recommend this explicit [ASSET/SCALE AMOUNT] notation, where SCALE describes the negative power of ten to multiply the AMOUNT with to obtain the decimal value in the given ASSET. It makes subsequent interaction with external systems much easier. You can read a bit more about it here [1]. [1] https://docs.formance.com/stack/unambiguous-monetary-notation https://docs.formance.com/stack/unambiguous-monetary-notatio...
- flockonus 2y ago
  What about units that cost sub-cent? For examples, I've seen private company stocks being $0.0001 Not sure why you'd need to make a note in the internal representation, vs. leave adapters to handle conversions.
  - bbor 2y ago
    Not to speak for them, but I think you’ve understood the point exactly. You need to be able to support arbitrary precision, but that support needs to be intentional to avoid errors. And you have to record that decision somewhere if adapters are to correctly handle your outputs; why not in the unit name?
  - petesergeant 2y ago
    If I understand their docs correctly, that’s equivalent to [USD/4 1]
- tway_GdBRwW 2y ago
  Used to work at a payments company. Yes, you need to be *very* explicit in how you model currency amounts and precision. See also earlier post about Canadian rounding rules. Some of the "logic" is regulatory/compliance driven. ref child post about stocks trading for 0.0001. Yes, those are real trades and (probably) fully legal etc, but I'm not sure the Fed recognizes currency amounts less than 1 US cent ($0.01), so the accounting rules and tax rules might not match expectations based on generalized floating point math.
  - euroderf 2y ago
    Doesn't the Fed recognise mills ? They are not extinct.
- girvo 2y ago
  Just to add to this, having also implemented a production payment system, we did the same thing. One needs to be very explicit about the scale and how it should be rounded and dealt with, and how to operate on two different scale systems. Quite a fun challenge, though I do not miss the edge cases. Our system was a payment system for childcare management software, interfacing with banks and the government directly.

vslira 2y ago

Super interesting and thanks for sharing. If I understood the license page on the repository correctly, the DSL is MIT-licensed since it's not within the enterprise directory, right?

superzamp 2y ago
Yes exactly! There's actually two implementations, one tightly knit to our ledger product located at [1] and the new, standalone one (used by the playground) at [2]. In any case, in both implementations, the DSL is indeed MIT. [1] https://github.com/formancehq/stack/tree/main/components/ledger/internal/machine https://github.com/formancehq/stack/tree/main/components/led... [2] https://github.com/formancehq/numscript https://github.com/formancehq/numscript

bbor 2y ago

To this noob, this seems like a problem that was pretty convincingly solved by double entry accounting ledgers, but from your post it sounds like this isn’t a replacement-of but rather an addition-to that model. What’s a situation where an imperative approach would be preferable to the traditional declarative approach? My depth of knowledge pretty much starts and ends with the below document, so apologies if this is obvious to experts! https://beancount.github.io/docs/the_double_entry_counting_method.html#introduction https://beancount.github.io/docs/the_double_entry_counting_m... E: the problem being “tracking transactions”. Yes?

superzamp 2y ago
That's a very good question. So the DSL here operates an agnostic source/dest transaction model, which is akin to the credit/debit model sans the semantic baggage. The goal of this model is indeed to be "tracking transactions" in the abstract sense, having the benefit of not forcing accounting decisions too early on when there is (still yet) none. For example, if you create a transaction moving money from "@stripe:main" to "@acct:123" and "@acct:234", you're merely representing the fact that you want this money to be moved. Wether the movement is clearing off a liability or generating revenue is another concern that you (in our model) want to take care of in a separate layer, that will also likely involve some intense intentionality and iterations from your accounting team. In a sense, it's as close to accounting than it is to warehousing money, moving unitary boxes of it from one location to another. These two models have the same amount of information per entry, so they can actually be converted from one to another, enabling you to also represent some accounting-ish transactions with this DSL, e.g. with a send [USD/2 100] from @ar:invoices:1234 to @sales.
- bbor 2y ago
  Fascinating, thanks for taking the time to educate! Makes sense to me — it seems this tool is purpose built for situations where tracking is complex enough to deserve decoupling from annotating or interpreting, to put it in my own kindergarten terms. I don’t have a need for this personally but I’ll definitely be bouncing this around in my head for a while, both technically (JS ALL the things!) and methodologically. Accounting is pretty trivial when you don’t have any income to track!

abdullahkhalids 2y ago

How would this handle Canadian guidelines for dealing with cents in cash, where you round to the nearest 5c [1]? Amounts ending in 1 cent and 2 cents are rounded down to the nearest 10 cents; Amounts ending in 3 cents and 4 cents are rounded up to the nearest 5 cents; Amounts ending in 6 cents and 7 cents are rounded down to the nearest 5 cents; Amounts ending in 8 cents and 9 cents are rounded up to the nearest 10 cents; Amounts ending in 0 cent and 5 cents remain unchanged. EDIT: I think if you send 12 cents, send [CADCASH/2 12] ( source = @user1 destination = @user2 ) It should result in sending 10 cents. "postings": [{"source": "user1", "destination": "user2", "amount": 10, "asset": "CADCASH/2"}] Can this happen? [1] https://www.canada.ca/en/revenue-agency/programs/about-canada-revenue-agency-cra/phasing-penny.html https://www.canada.ca/en/revenue-agency/programs/about-canad...

superzamp 2y ago
Well that's definitely a good puzzle. I've tried to model it for a bit, but it indeed looks like we'd need to add something to the language to make it possible at all! Thanks for bringing this up.
- fallat 2y ago
  Same thing with income tax brackets. You need conditional logic.
  - superzamp 2y ago
    For this particular case, I would say that tax-brackets sort of logic can be expressed in the destination block with ordered destinations. For example, you could have something like this: send [USD/2 *] ( source = @users:1234 destination = { // first $1000 are taxed at 10% max [USD/2 100000] to { 10% to @taxes remaining kept } // Anything above that is, taxed at 20% remaining to { 20% to @taxes remaining kept } } ) (You can test it on the playground, you'll just want to feed the "users:1234" account with an initial balance in the input section)

kamalamomala 2y ago

[flagged]

jimbokun 2y ago

Are there converters to/from OFX for Numscript?

superzamp 2y ago
Not as of today!

henning 2y ago

Ah, what better input format than JSON, a poorly defined, ambiguous format that freely mixes integers and floating points and lacks supports for bigints and bigdecimals.

darby_nine 2y ago
It's only ambiguous if you let it be. The rest is just implementation details.
monero-xmr 2y ago
In cryptocurrency the decimals are often a range between 8 and 18 decimals, sometimes much larger. The solution is always to use strings, and have separate fields or data models that explain the precision. The amount of languages and libraries, for tradfi or anything, is so wide that actually trying to use something other than strings for passing data between systems and applications would be self destructive

bschmidt1 2y ago

This is mindblowing. It's like the missing piece of blockchain that was needed for dev adoption. I can see it also being used for modern bank/payments apps including those in games and other simulated economies. Now let's see a "ContentScript" for posting content to ledgers :D send [Image "tacos.jpg", Text "taco 2sday!"] ( source = @bschmidt1 destination = @us-west-relay )

gregwebs 2y ago

The CLI is only displaying a "check" command but I did figure out that there is a "run" command as well. go install github.com/formancehq/numscript@latest go: downloading github.com/formancehq/numscript v0.0.8 go: downloading github.com/antlr4-go/antlr/v4 v4.13.1 go: downloading github.com/sourcegraph/jsonrpc2 v0.2.0 go: downloading golang.org/x/exp v0.0.0-20240707233637-46b078467d37 numscript -h Numscript cli Usage: numscript [command] Available Commands: check Check a numscript file help Help about any command Flags: -h, --help help for numscript -v, --version version for numscript Use "numscript [command] --help" for more information about a command.

ascandone 2y ago
Hello, I'm one of the Numscript devs. The Numscript CLI does have a `numscript run` command, which is hidden for now but will be released in the following days. It behaves in the same way as the playground, and you can take a look at it with `numscript run --help` command

adius 2y ago

Nice! I tried to achieve a similar design with my plain text accounting tool Transity: https://github.com/ad-si/Transity https://github.com/ad-si/Transity But this seems to have more programming language like features if I understand correctly. What else is different?

hum3hum3 2y ago
You might like https://martin.kleppmann.com/2011/03/07/accounting-for-computer-scientists.html https://martin.kleppmann.com/2011/03/07/accounting-for-compu... which shows it as a directed graph.

holloway 2y ago

Large sends seem to crash the playground with a JSON parse error. eg change the first example 'Simple send' to send `10000000000000000000` (`send [USD/2 10000000000000000000]`) and there's a crash

superzamp 2y ago
Thanks for the heads up! While the Numscript backend itself uses big.Int, we're still using normal javascript numbers in the playground for now so that's where it's coming from. But we should definitely switch to BigInt in the playground code though and that would solve the issue.
- hum3hum3 2y ago
  You should never use floats for money. In python Decimal works well. TigerBeatle uses 128 bit accounts and amounts which I thought was interesting although more than needed gor many cases.

neilwilson 2y ago

How do you model a bank making a loan, where the bank creates the money out of nothing and the accounting appears backwards? The equivalent of DR @my_bank:loans_made 10000, CR @my_bank:1234 10000

superzamp 2y ago
Yeah that's a good question, you'll want to leverage the `overdraft` functionality for that, enabling the account end balance to go below zero. So you could do something like: send [USD/2 10000] ( source = @my_bank:loans_made allowing unbounded overdraft destination = @my_bank:1234 ) With the post-transaction balance of @my_bank:loans_made becoming [USD/2 -10000]. So that's for creating money out of nothing. There's a little bit more to it if you want to create accounting-perfect entries, but a simple way to map src/dest to cr/dr entries is to say that every credit becomes a destination, and every debit becomes a source. You can then consider debit normal accounts as having their debit balance being equivalent to the sum of entries where the account is source, their credit balance as the sum of entries where the account is destination—and do the opposite for credit normal accounts. We've written a bit more about it here [1]. [1] https://docs.formance.com/ledger/advanced/accounting/credit-debit-to-src-dest https://docs.formance.com/ledger/advanced/accounting/credit-...
- neilwilson 2y ago
  That’s helpful. How about Nostro and Vostro accounts used as the balancing item with sub-ledgers like agency banks. (In essence memorandum accounts on a control account) How would you handle a three legged agency bank transaction as you get in a BACS file (source, Nostro, destination).
- neilwilson 2y ago
  That’s helpful. How about Nostro and Vostro accounts used as the balancing item with sub-ledgers like agency banks. (In essence memorandum accounts on a control account) How would you handle a three legged agency bank transaction as you get in a BACS file (source, Nostro, destination)?

throwaway81523 2y ago

This is a pretty neat paper about a Haskell DSL for specifying financial contracts and calculating their values: https://www.microsoft.com/en-us/research/publication/composing-contracts-an-adventure-in-financial-engineering/ https://www.microsoft.com/en-us/research/publication/composi... It is quite powerful and you might want to look at it for possible features to use.

pjacotg 2y ago
Nice! I've worked with a few similar real world payoff languages. This paper discusses one of them: https://www.infoq.com/presentations/haskell-barclays/ https://www.infoq.com/presentations/haskell-barclays/