5 ms·

Show HN: An open-source financial ledger API written in Clojure

125 points by andriosr 6y ago

andriosr 6y ago
Hi everyone, I have been iterating a startup since the start of the year. And I'm using Clojure for everything I do. The first product was a ledger service, but it ended up not fitting well as a managed service. We pivoted to a different idea and decided to open-source the ledger project. It brings years of fintech experience, with the most common problems faced at scale for a ledger solved into the design. I hope this will be helpful to some of the companies that are struggling with this kind of system. Let me know if you have any questions :)
- bmh100 6y ago
  Why did you go with a non-relational database?
  - andriosr 6y ago
    Interesting question. No-SQL doesn't mean non-relational. DynamoDB is a relational database. It's up to you on how you design the table to accommodate your relations. This is a great presentation about this: https://youtu.be/6yqfmXiZTlM?t=1447 https://youtu.be/6yqfmXiZTlM?t=1447
    - nagyf 6y ago
      DynamoDB is a key-value store. Key-value stores are not relational databases [1] The link you posted is a really good resource for someone who wants to start using DynamoDB though. [1] - https://aws.amazon.com/nosql/key-value/ https://aws.amazon.com/nosql/key-value/
      - andriosr 6y ago
        You're right. DynamoDB is not a relational database. But data in DynamoDB can be relational just as in a regular relational database: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-relational-modeling.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
        outworlder 6y ago
        That's really stretching the definition of a 'relational database'. You can create relationships on flat text files. It doesn't mean that you have a relational database now.
    - Scarbutt 6y ago
      Many have fallen for that marketing video stunt and later regret it.
- smcnally 6y ago
  Sequence has interesting applications for longer-term work I'm on, and/but I'm Baader-Meinhoffing [0] re more-immediate small-biz accounting ledger needs: I spent a good chunk of yesterday in gnu-cash and Frappe books [1] working to import/tag/script Chart of Accounts and transaction info from two banks while aiming to avoid (new-to-me) UI, code base, or Generally Accepted Accounting Principles rabbit holes. Sequence is clearly an API and I ought stick with consumer packages @ immediate needs. I'll spin up the container to check it out in general; Please let me know if I'm mistaken / you recommend specifics it'd be helpful for me to check out and feed back on - [0] - https://en.m.wikipedia.org/wiki/List_of_cognitive_biases#Frequency_illusion https://en.m.wikipedia.org/wiki/List_of_cognitive_biases#Fre... [1] - https://github.com/frappe/books https://github.com/frappe/books
  - andriosr 6y ago
    You're right. Sequence is a ledger API. These tools will give you higher abstractions on top of their internal ledgers. I think you are on the right track. If you want something extremely simple maybe the Ledger CLI [0] is worth looking at. Try to leverage these higher level abstractions. You don't need deep accounting understanding to import/tag/script a few transactions. [0] https://www.ledger-cli.org/ https://www.ledger-cli.org/
- elcritch 6y ago
  Nice clean code! Thanks for open sourcing. I was a bit hoping to see how you were calculating amortization/interest charges in Clojure, but it appears to be only a pure ledger. Out of curiosity, were you using hashing to verify transaction chains? I saw code for MD5 hashes, but not where it was used.
  - andriosr 6y ago
    Thanks! I have interesting ideas for amortization/interests, maybe they can get added at some point. Yes, the hashes are the ids used for Avoiding double-spending along with dynamodb conditional put. I’m on the phone now, will follow up with code once I get to the computer
jsmith99 6y ago
Why use nosql for a ledger?
- weitzj 6y ago
  I was wondering as well. How does that work with eventual consistency? Shouldn’t there be double bookkeeping so you don’t generate money out of the air?
  - weitzj 6y ago
    Also it seems like the amount is of type number. Ideally you would calculate in Dollarcents or Eurocents (I.e. solely use integers) like 100 Eurocents equal 1€ So you do not have any rounding involved Also if a request comes in twice there are no requestids involved. Therefore it might be possible to generate another transaction it seems. There are great blog posts from Stripe engineers about these request ids to make requests idempotent with Postgres databases. I highly recommend reading them.
    - kondro 6y ago
      Not sure about how Clojure treats numbers, but in DynamoDB these are treated as Decimals and don’t have floating-point rounding errors.
      - tappio 6y ago
        Does it allow to save values with floating point imprecision such as 1.000000001? I don't think the issue is never in the database but the service that uses the API to send values. If such value is allowed, it will cause headache and errors when the consumers of the API make mistakes. I think the general practice is to store them in integers, just for the sake of being foolproof.
        andriosr 6y ago
        Yep. Integers they are: https://news.ycombinator.com/item?id=24435148 https://news.ycombinator.com/item?id=24435148
mikejulietbravo 6y ago
this is pretty cool, love it
lelo_tp 6y ago
Nice work! Very well done
didiercrunch2 6y ago
Amazing work there. I have looked at your code and it is very nice. Fun thing, I am currently building a similar ledger in clojure. The task is much harder then expected. My goals are very different then yours; I would like to implement a video game around a fictional financial system. Consequently, It is very important to me to implement the concept of "interest rates". My solution is over engineered power 25. Nevertheless, you might be interested into it. It is based on datomic. https://gitlab.com/didiercrunch/skirmish/-/tree/master/src/clj/skirmish/ledger https://gitlab.com/didiercrunch/skirmish/-/tree/master/src/c...
- andriosr 6y ago
  Wow, this is very interesting! Will look into it, I have plans to add interests to Sequence at some point, this will be of great help.
  - didiercrunch2 6y ago
    I have seen your issue about datomic. To be fair, I am bit disappointed about datomic in a real life situation. If I were you, I would try to implement a postgres backend instead of a datomic one.
    - andriosr 6y ago
      Interesting take. I think Datomic is a low hanging fruit. A lot of people get excited about it and is fairly easy to do. That being said, I saw quite a few examples of companies migrating from it. Complexity to operate for simple things being the biggest reason. What is your experience with it?

125 points by andriosr 6y ago

Hi everyone, I have been iterating a startup since the start of the year. And I'm using Clojure for everything I do. The first product was a ledger service, but it ended up not fitting well as a managed service. We pivoted to a different idea and decided to open-source the ledger project. It brings years of fintech experience, with the most common problems faced at scale for a ledger solved into the design. I hope this will be helpful to some of the companies that are struggling with this kind of system. Let me know if you have any questions :)

bmh100 6y ago
Why did you go with a non-relational database?
- andriosr 6y ago
  Interesting question. No-SQL doesn't mean non-relational. DynamoDB is a relational database. It's up to you on how you design the table to accommodate your relations. This is a great presentation about this: https://youtu.be/6yqfmXiZTlM?t=1447 https://youtu.be/6yqfmXiZTlM?t=1447
  - nagyf 6y ago
    DynamoDB is a key-value store. Key-value stores are not relational databases [1] The link you posted is a really good resource for someone who wants to start using DynamoDB though. [1] - https://aws.amazon.com/nosql/key-value/ https://aws.amazon.com/nosql/key-value/
    - andriosr 6y ago
      You're right. DynamoDB is not a relational database. But data in DynamoDB can be relational just as in a regular relational database: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-relational-modeling.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
      - outworlder 6y ago
        That's really stretching the definition of a 'relational database'. You can create relationships on flat text files. It doesn't mean that you have a relational database now.
  - Scarbutt 6y ago
    Many have fallen for that marketing video stunt and later regret it.
smcnally 6y ago
Sequence has interesting applications for longer-term work I'm on, and/but I'm Baader-Meinhoffing [0] re more-immediate small-biz accounting ledger needs: I spent a good chunk of yesterday in gnu-cash and Frappe books [1] working to import/tag/script Chart of Accounts and transaction info from two banks while aiming to avoid (new-to-me) UI, code base, or Generally Accepted Accounting Principles rabbit holes. Sequence is clearly an API and I ought stick with consumer packages @ immediate needs. I'll spin up the container to check it out in general; Please let me know if I'm mistaken / you recommend specifics it'd be helpful for me to check out and feed back on - [0] - https://en.m.wikipedia.org/wiki/List_of_cognitive_biases#Frequency_illusion https://en.m.wikipedia.org/wiki/List_of_cognitive_biases#Fre... [1] - https://github.com/frappe/books https://github.com/frappe/books
- andriosr 6y ago
  You're right. Sequence is a ledger API. These tools will give you higher abstractions on top of their internal ledgers. I think you are on the right track. If you want something extremely simple maybe the Ledger CLI [0] is worth looking at. Try to leverage these higher level abstractions. You don't need deep accounting understanding to import/tag/script a few transactions. [0] https://www.ledger-cli.org/ https://www.ledger-cli.org/
elcritch 6y ago
Nice clean code! Thanks for open sourcing. I was a bit hoping to see how you were calculating amortization/interest charges in Clojure, but it appears to be only a pure ledger. Out of curiosity, were you using hashing to verify transaction chains? I saw code for MD5 hashes, but not where it was used.
- andriosr 6y ago
  Thanks! I have interesting ideas for amortization/interests, maybe they can get added at some point. Yes, the hashes are the ids used for Avoiding double-spending along with dynamodb conditional put. I’m on the phone now, will follow up with code once I get to the computer

jsmith99 6y ago

Why use nosql for a ledger?

weitzj 6y ago
I was wondering as well. How does that work with eventual consistency? Shouldn’t there be double bookkeeping so you don’t generate money out of the air?
- weitzj 6y ago
  Also it seems like the amount is of type number. Ideally you would calculate in Dollarcents or Eurocents (I.e. solely use integers) like 100 Eurocents equal 1€ So you do not have any rounding involved Also if a request comes in twice there are no requestids involved. Therefore it might be possible to generate another transaction it seems. There are great blog posts from Stripe engineers about these request ids to make requests idempotent with Postgres databases. I highly recommend reading them.
  - kondro 6y ago
    Not sure about how Clojure treats numbers, but in DynamoDB these are treated as Decimals and don’t have floating-point rounding errors.
    - tappio 6y ago
      Does it allow to save values with floating point imprecision such as 1.000000001? I don't think the issue is never in the database but the service that uses the API to send values. If such value is allowed, it will cause headache and errors when the consumers of the API make mistakes. I think the general practice is to store them in integers, just for the sake of being foolproof.
      - andriosr 6y ago
        Yep. Integers they are: https://news.ycombinator.com/item?id=24435148 https://news.ycombinator.com/item?id=24435148

mikejulietbravo 6y ago

this is pretty cool, love it

lelo_tp 6y ago

Nice work! Very well done

didiercrunch2 6y ago

Amazing work there. I have looked at your code and it is very nice. Fun thing, I am currently building a similar ledger in clojure. The task is much harder then expected. My goals are very different then yours; I would like to implement a video game around a fictional financial system. Consequently, It is very important to me to implement the concept of "interest rates". My solution is over engineered power 25. Nevertheless, you might be interested into it. It is based on datomic. https://gitlab.com/didiercrunch/skirmish/-/tree/master/src/clj/skirmish/ledger https://gitlab.com/didiercrunch/skirmish/-/tree/master/src/c...

andriosr 6y ago
Wow, this is very interesting! Will look into it, I have plans to add interests to Sequence at some point, this will be of great help.
- didiercrunch2 6y ago
  I have seen your issue about datomic. To be fair, I am bit disappointed about datomic in a real life situation. If I were you, I would try to implement a postgres backend instead of a datomic one.
  - andriosr 6y ago
    Interesting take. I think Datomic is a low hanging fruit. A lot of people get excited about it and is fairly easy to do. That being said, I saw quite a few examples of companies migrating from it. Complexity to operate for simple things being the biggest reason. What is your experience with it?