4 ms·

> Furthermore, bundling an 80MB+ SQLite file to our codebase slowed down the entire Github repository and hindered us from considering more robust hosting platf

> Furthermore, bundling an 80MB+ SQLite file to our codebase slowed down the entire Github repository and hindered us from considering more robust hosting platforms. It's... an 80MB database. It couldn't be smaller. There are local apps that have DBs bigger than that. There is no scale issue here. And... it's committed to GitHub instead of just living somewhere. And they switched to Neon. To me, this screams "we don't know backend and we refuse to learn". To their credit, I will say this: They clearly were in a situation like: "we have no backend, we have nowhere to store a DB, but we need to store this data, what do we do?" and someone came up with "store it in git and that way it's deployed and available to the app". That's... clever. Even if terrible.

BHSPitMonkey 2y ago
> It's... an 80MB database. It couldn't be smaller. There are local apps that have DBs bigger than that. There is no scale issue here. It depends. If that 80MB binary file in git is updated/replaced often, you likely have a problem (every 100 changes/replacements might grow the repo by as much as 8GB).
- __float 2y ago
  They said it was stored with Git LFS, so it's just a pointer in the repo, not all 80 MB.
candiddevmike 2y ago
> That's... clever It's tech debt
- mbreese 2y ago
  It’s tech debt, but it’s cheap debt. 80Mb isn’t much to have to worry about. There are many ways to use a DB of that size that will be “good enough”. So long as the queries were returning fast enough and the dev overhead wasn’t too much (LFS was a good choice), I’m not sure I’d have worried much. There is tech debt that will eat away at you, and then there is this… something that you’ll want to replace someday, but it can wait until you are ready to tackle it properly. And you may not even know what data access patterns you need right away. For a new company with a pretty new way of working with insurance, this seems like a good trade off to me. Plus, it seems clear that they weren’t confident in their backend engineering yet, so this gave them time to figure things out.
gfody 2y ago
80mb is nothing, I think it's a no brainer to keep it in git but they should've commited the .dump instead of the .db
agent001 2y ago
Neon actually solved a bunch of problems here. Sure, 80MB might not seem huge, but in a git repo? Also, 80MB is actually large enough to impact git performance according to Github(https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github https://docs.github.com/en/repositories/working-with-files/m...). That can be a pain, especially if you're updating it often. Neon let them ditch the whole 'DB-in-git' approach. No more slowing down the repo or worrying about it ballooning with every update. Plus, it probably made deployment and scaling way smoother. Sometimes the 'clever' solution isn't the best long-term, you know? Kudos to them for recognizing that and making the switch.

BHSPitMonkey 2y ago

> It's... an 80MB database. It couldn't be smaller. There are local apps that have DBs bigger than that. There is no scale issue here. It depends. If that 80MB binary file in git is updated/replaced often, you likely have a problem (every 100 changes/replacements might grow the repo by as much as 8GB).

__float 2y ago
They said it was stored with Git LFS, so it's just a pointer in the repo, not all 80 MB.

candiddevmike 2y ago

> That's... clever It's tech debt

mbreese 2y ago
It’s tech debt, but it’s cheap debt. 80Mb isn’t much to have to worry about. There are many ways to use a DB of that size that will be “good enough”. So long as the queries were returning fast enough and the dev overhead wasn’t too much (LFS was a good choice), I’m not sure I’d have worried much. There is tech debt that will eat away at you, and then there is this… something that you’ll want to replace someday, but it can wait until you are ready to tackle it properly. And you may not even know what data access patterns you need right away. For a new company with a pretty new way of working with insurance, this seems like a good trade off to me. Plus, it seems clear that they weren’t confident in their backend engineering yet, so this gave them time to figure things out.

gfody 2y ago

80mb is nothing, I think it's a no brainer to keep it in git but they should've commited the .dump instead of the .db

agent001 2y ago

Neon actually solved a bunch of problems here. Sure, 80MB might not seem huge, but in a git repo? Also, 80MB is actually large enough to impact git performance according to Github(https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github https://docs.github.com/en/repositories/working-with-files/m...). That can be a pain, especially if you're updating it often. Neon let them ditch the whole 'DB-in-git' approach. No more slowing down the repo or worrying about it ballooning with every update. Plus, it probably made deployment and scaling way smoother. Sometimes the 'clever' solution isn't the best long-term, you know? Kudos to them for recognizing that and making the switch.