14 ms·
How you can track your personal finances using Python
- djhworld 5y agoI'm not really sure about the python aspect in this article however I've been using beancount and Fava for nearly 3 years now and it's one of my favourite pieces of software. Outside of installing it I've never had to really think about python. All of my financial history is contained in a text file processed by beancount and long may it continue! One thing I do wonder though is how well it scales, I'm a single man so tracking everything isn't too much effort but if I had a partner it suddenly becomes at least 50% more things to track
- remram 5y agoYou didn't have to write any Python to import transactions from your bank?
- aquark 5y agoI have some custom code to convert CSV files downloaded from my bank to beancounts text format, but it is all dotNet code since that is what I know the text format is easy to work with
- siddhant 5y agoThat's actually a good point. You only need to write Python code if you're using the Beancount importers framework and intend to use bean-identify and/or bean-extract.
- diibv 5y agohledger is another open-source CLI tool for accounting that works amazingly well for me. You can add transactions editing in plain text too and there is a basic web UI on top of it.
- jamesgreenleaf 5y agoAnd hledger is based on the original https://www.ledger-cli.org/ https://www.ledger-cli.org/ which is also a great option for CLI accounting.
- rmesters 5y agoRegarding "1. Download transactions from your bank" - to avoid having to deal with PDFs, it's possible to use account aggregators to fetch your own bank data in Python-friendly formats like JSON. We have this feature at Nordigen (I'm the cofounder), where we allow developers to download their bank statements in JSON format from 1,000+ European banks (free): https://nordigen.com/en/blog/download-your-bank-statement-json-format/ https://nordigen.com/en/blog/download-your-bank-statement-js...
- BugWatch 5y agoThis is great. Looking forward to a few of your "in development" entries of the Croatian banks!
- rmesters 5y agoComing real soon. If you sign up to the platform, you can get updates on our coverage as we're ramping up in Croatia.
- rendx 5y agoFor a data broker that deals with highly sensitive data, your privacy policy is very broad and unspecific (and as such not actually GDPR compliant). I would not trust you with my data that you will (amongst other purposes) use for: "in order to derive statistical models for future data enrichment purposes", "to improve the performance and accuracy of the Services", "to improve Your experience while using the Services" and "to send Users and website visitors relevant informational content regarding Services and personalized offers they subscribed to". "Nordigen may also provide personal data to companies that process personal data on behalf of Nordigen such as marketing service providers." - wonderful, thank you for doing that. not. GDPR requires you to be more specific in terms of "purpose", to actually list your data processing partners, what purpose the data sharing with that particular partner serves, and which data exactly is shared with which partner.
- aeoleonn 5y agoa REST API such as: https://plaid.com/ https://plaid.com/
- commoner 5y agoPlaid has serious privacy and security concerns, so I would be careful: - https://news.ycombinator.com/item?id=28200076 https://news.ycombinator.com/item?id=28200076 - https://news.ycombinator.com/item?id=28389576 https://news.ycombinator.com/item?id=28389576
- westurner 5y agoFrom https://news.ycombinator.com/item?id=28203393 https://news.ycombinator.com/item?id=28203393 : > No, your personal data is not sold or rented or given away or bartered to parties that are not Plaid, your bank, or the connected app. We talk about all of this in our privacy policy, including ways that data could be used — for example, with data processors/service providers (like AWS which hosts our services) for the purposes of running Plaid’s services or for a user’s connected app to provide their services. >> I saw that. Thank you for your patience and persistence in responding to so many pointed questions. >> For any interested, here is a link to relevant section of the referenced privacy policy: https://plaid.com/legal/#consumers https://plaid.com/legal/#consumers >> I am also impressed by the Legal Changelog on the same page that clearly lays out a log of changes made to privacy & other published legal documents.
- commoner 5y agoThe comments in those threads were more negative than positive, and the fact that Plaid paid a $58 million settlement for allegedly sharing personal banking data with third parties without consent is telling enough. I am not going to give my banking usernames and passwords to Plaid in plaintext, when their employee is arguing on HN over what the word "sold" means: https://news.ycombinator.com/item?id=28201649 https://news.ycombinator.com/item?id=28201649
- 5y ago
- emrah 5y agoI'm thinking about a PF app concept where you enter your own transactions manually, but you don't have to keep track of everything down to the cent. At the end of every month, banks send out statements so importing all transactions aren't necessary to keep track of one's overall financial standing. Banks do make mistakes but extremely rarely that it's not worth wanting to import all transactions in the off chance that you'll catch a mistake. Retailers can make mistakes like double charge you but again those are rare and you can take a quick look at your statements to find them usually. Keeping track of individual transactions are important to be able to figure out where the money is going in general but this doesn't require keeping track of 'every' transaction. Just the ones that make up the biggest percentage of the total money spent. They are usually easy to spot on the bank/creditcard statements. One doesn't even have to be perfectly accurate to gain from inspecting statements either. You spent $834.56 for new tires? You can enter that as $850 and it's fine. Sure, there might be cases where the total is split more or less evenly across several small transactions and that can be handled similarly to the above. Knowing where the money is going isn't terribly useful actually unless there are a lot of frivolous spendings. It usually comes down to "you are not making enough money" or "the lifestyle you desire is too expensive for your current income". No app can fix either of these, other than point out the obvious problem. Most personal finance blogs end up leaning towards "how to earn more" because that's more important than keeping track of your transactions or budgeting.
- names_are_hard 5y agoThis is the philosophy behind the FOSS app I developed for my own system. The way it works is that I track the big stuff and ignore the little stuff. Periodically (typically every month but it doesn't have to be) I enter the current balances of my bank accounts. I have a report that does how much money I spent per period that is unaccounted for, that's money that got spend on "miscellaneous". As long as that number is beneath some threshold that I consider acceptable I let it be. If it's high, I might look back and try to determine where my money went and make adjustments. The guiding principle behind this approach is resilience to laziness. It's not an all or nothing system, instead it gives me whatever I give it. If I am busy and don't enter as many transactions, no worries - I'll just have more in the miscellaneous category. If I don't track my balances every month, the period will be longer. But the system always "balances". The system also serves as a log for bigger purchases. When did I buy that fridge? How much did I pay? What is the exact model number of that piece of equipment in case I need to buy it again? I can search my transaction log.
- darcys22 5y agoShameless plug of my open source finance system. https://github.com/darcys22/godbledger https://github.com/darcys22/godbledger I love these command line self hosted accounting software packages. But double entry bookkeeping was invented using ledgerbooks with ruled tables. I feel the plain text dataformats are a regression compared to a sql database. A general ledger just works so well with columns that you can sum. It also saves you from needing a custom tool like bean-query to replicate sqlite-ish queries because it could have been in a database from the start
- deleted 5y ago[deleted]
- xupybd 5y agoI used HLedger for years it was great. I thought I could do better with a paid solution. YNAB was awful and slow. So I switched to Lunch Money. It's amazing fast and intuitive. Only it's not a good fit for me. I'm trying to shoe horn my budgeting style into and it's not working. I use the envelope style. Every dollar is assigned a bucket when it comes in. All spending is done from a bucket and balances always persist. I don't like monthly budget targets. I prefer to put money in a bucket when the money comes in. I might move back to plain text accounting as it is so flexible. I will miss the amazing interface to Lunch Money. I also like supporting a small business like Lunch Money.
- password4321 5y agoI think it's https://onefinance.com https://onefinance.com that provides separate routable account numbers and credit card numbers per envelope/"pocket". A testimonial popped up about two weeks ago: https://news.ycombinator.com/item?id=28247788#28248539 https://news.ycombinator.com/item?id=28247788#28248539.
- chromatin 5y agoI still use Hledger along with a small phalanx of accessory scripts, mostly written in Python, that do everything from parse CSVs to calling smart contract read functions to categorize crypto transactions for tax purposes. Many thanks to Simon Michael for not only that wonderful program but also being so responsive.
- xupybd 5y agoI might go back to HLedger soon. I guess I can always donate monthly to the project to scratch my small business support itch. I also had a good collection of scripts to manage my finances. I ran it all on a VPS so I could access anywhere. I had an ssh client on my phone to make changes on the go. Not as ergonomic as an app but it worked. Agreed Simon is an amazing project owner. He and his documentation are the reasons I like HLedger over the alternatives. I'll admit that GoDBLedger is also tempting.
- dugmartin 5y ago
- vymague 5y agoI don't think double-entry bookkeeping is such a common knowledge. Anyways, there are discussions and links trying to teach "accounting for developers" on https://news.ycombinator.com/item?id=23964513 https://news.ycombinator.com/item?id=23964513 I personally find the topic difficult and comes with a lot of nonsensical jargons.
- partymonious 5y agoSounds interesting, but it’s still a long way to beat the best finances tracking system, good ol’ Excel sheets.
- Waterluvian 5y agoThere’s so many ways to do personal finances but I found most of them to either be too tedious or require doing stupid things like letting third parties log into your bank account. For me I simply had a spreadsheet with rows for each month and columns for each account: chequing, mortgage, RRSP, RESP, etc. I’d spend 10 mins a month logging in and typing out each balance. I’d chart the over-time trend and that’s all I needed to detect any changes in habit or ugly trajectories. 90% of what signal I needed with 10% of the effort. If you don’t need to micromanage all your flavours of spending or other details, consider keeping it dead simple.
- berkut 5y agoYeah, I wrote my own open source app to do basic personal finance accounting 12 years ago after getting annoyed at the propriatary OS X solutions (iBank, in particular), and KMyMoney / GNUCash not supporting split transactions at the time (they do now), and have been using that for the past 12 years. It's not double-entry (I recognise the benefits of that, but don't like it), and it's a manual process, but it works for me, and I can graph stuff easily to provide some indication of where money is going. I ported it to Qt for Linux last year and still use it, but to be honest, I don't really know why I track my finances that religously - although it has helped me a few times with accounting / tax purposes given I'm a contractor now. I suspect one of the reasons there are so many different solutions is everyone wants to do things differently, and integration with the banks for automated transaction import/download is quite complicated.
- Arainach 5y agoI agree that this works at the income level of most of the HN audience. If you're living paycheck to paycheck, you may have more need to closely track exactly what your money is going to (of course, you'll also likely have the least spare time to spend on such tracking). At the income level of most tech workers/the HN audience, however, I agree with your system. My spreadsheet is similar to what you describe - savings/checking accounts, investment accounts, and balance of all credit cards/mortgate/loans that give me a single number of "net worth". So long as its trending significantly positive, I don't care about the details.
- jmschlmrs 5y agoI built https://mygraph.ca https://mygraph.ca as I was tired of Mint failing to sync with Canadian banks (and later dropped Mint anyway because of the security/privacy concerns). It’s not targeting technical users but it would be cool to add a SQL integration like Stripe Sigma, etc at some point.
- candiddevmike 5y agoHey this is pretty cool, I posted an app too and see some opportunities for collaboration here (especially since I'm starting to explore data visualization capabilities). Can you email me? My contact info is in my profile.
- westurner 5y ago> We take the output of the previous step, pipe everything over to our .beancount file, and "balance" transactions. > Recall that the flow of money in double-entry accounting is represented using transactions involving at least two accounts. When you download CSVs from your bank, each line in that CSV represents money that's either incoming or outgoing. That's only one leg of a transaction (credit or debit). It's up to us to provide the other leg. > This act is called "balancing". Balance (accounting) https://en.wikipedia.org/wiki/Balance_(accounting) https://en.wikipedia.org/wiki/Balance_(accounting) Are unique record IDs necessary for this [financial] application? FWICS, https://plaintextaccounting.org/ https://plaintextaccounting.org/ just throws away the (probably per-institution) transaction IDs; like a non-reflexive logic that eschews Law of identity? Just grep and wc? > What does the ledger look like? > I wrote earlier that one of the main things that Beancount provides is a language specification for defining financial transactions in a plain-text format. > What does this format look like? Here's a quick example: option "title" "Alice" option "operating_currency" "EUR" ; Accounts 2021-01-01 open Assets:MyBank:Checking 2021-01-01 open Expenses:Rent 2021-01-01 * "Landlord" "Thanks for the rent" Assets:MyBank:Checking -1000.00 EUR Expenses:Rent 1000.00 EUR What does the `*` do?
- freddie_mercury 5y agoThe star is just an 1-digit field for "flag". I don't think there are any defined semantics for the field but by convention 'star' means something 'no special flags on this transaction'. People use other flags to indicate, for instance, whether a transaction has been reconciled, or cleared their bank, or whatnot.
- SavantIdiot 5y agoIt's funny that a register application is basically a trivial database, but it requires so much subtle UI/UX that it is really hard to make something usable. I've been using Quicken for 30+ years and I've tried other software, including my own and open source, and nothing really comes close the the ease of what they've built.
- candiddevmike 5y agoIf you want an open source self hosted budgeting solution, firefly is the most commonly recommended (https://www.firefly-iii.org/ https://www.firefly-iii.org/). If you are looking for something supported/less DIY and with features beyond budgeting, checkout Homechart (https://about.homechart.app https://about.homechart.app).
- _Anima_ 5y agoThe author of firefly claims to do double entry but it's just not true. I quickly gave up when I realized that I could not input a refund on my credit card account. It doesn't allow positive transactions. The only suggested workaround by the author himself was to edit the original transaction. Firefly is probably useful for many people but to me it was too opinionated and not natural to use. I moved to Gnucash.
- JC5 5y agoThat's still double entry actually, because it refers to the DB structure ;)
- mattbillenstein 5y agoI find Google Sheets pretty hard to beat - you can get stock pricing data via the GOOGLEFINANCE function, and I have a simple script that does imap -> sheets for bank and credit card balances. I'm able to keep an up to date view of my net worth without resorting to Mint/Plaid et al. And you can generate whatever charts and graphs you like fairly easily - have access on your phone, etc. For budgeting, I import csv from my bank/credit cards into google sheets and tag transactions with ~20 different categories - I do this every month or two and it doesn't take that long. It gives me a backwards view of my spending that I can use to influence future spending. I don't run a hard budget really.
- sockfish 5y agoI struggled with the accounting concepts until I stumbled on this blogpost from Martin Kleppmann: https://martin.kleppmann.com/2011/03/07/accounting-for-computer-scientists.html https://martin.kleppmann.com/2011/03/07/accounting-for-compu.... After that I finally 'got' the accounting graph, and I don't care too much about the medium or the software anymore (its a google sheet in my case).
- siddhant 5y agoMartin's blog post is awesome! It helped clear up a bunch of stuff for me too. And on a somewhat unrelated note: his book on building data-intensive applications is another goldmine (on a different topic).
- trepanne 5y ago<shameless plug> If you're using Python, and you want to import transaction/balance data without handing over your passwords to some third party, you can try https://github.com/csingley/ofxtools https://github.com/csingley/ofxtools </shameless plug>
- siddhant 5y agoLooks cool! This will definitely be of interest to folks using Beancount. I just added a link to it on https://awesome-beancount.com/#tools https://awesome-beancount.com/#tools.
- carlivar 5y agoThis is really cool but I feel like it glosses over married accounting, which is a longer session of yelling things like "Honey! What's this PayPal for $117 3 weeks ago?!"