4 ms·
One problem is that at a certain point you can’t just lose logs - for example, somebody might pay you per ad impression or click. Losing logs can mean losing ev
by 542458 5y ago
One problem is that at a certain point you can’t just lose logs - for example, somebody might pay you per ad impression or click. Losing logs can mean losing evidence that those impressions/clicks happened. Logs aren’t just a debugging tool, they can be a critical legal CYA.
I’m not saying that people don’t overcomplicate projects (they do) but a lot of simplifying assumptions can break down at scale.
- sateesh 5y agoBut why would one use log as source of truth ? If the info logged is important for the business, probably it should be stored in a DB (say).
- endisneigh 5y agoLogs are stored in a database, just not usually a relational one.
- sateesh 5y agoTrue, logs do needs to be stored somewhere. What l meant was: if the info is so important for the business, that detail should be stored in a system that provides a longer durability than just logging it.
- endisneigh 5y agoThough I agree, I think the parents point is that you don’t necessarily know if it’s important so you have to have some general record keeping with respect to “activity”
- alex_sf 5y ago> just logging it. This is the misunderstanding here. With a sufficiently complex system, they are never 'just logs'.
- Aeolun 5y agoHmm, I think that anything you do not consider ‘just logs’ should be somewhere in your main database instead.
- jeffbee 5y agoThat's backwards. DBs you can lose because they can be reconstructed from the logs.
- sateesh 5y agoSeems we are referring to two different things. What you mention references DB journal logs, and my comment was about application logs (like debug, info, error message etc.)
- likpok 5y agoYou can even handle impressions/clicks/conversions in a lossy system, you just have to bound how much loss you're willing to accept. The counterbalancing factor is how expensive you want your logging infrastructure to be -- the more consistency, the more it costs. The other trade off is availability: what do you do when the logging backend is temporarily down? Return a 503 to the user? Or accept that you're not making money on that impression and move on?
- RhodesianHunter 5y agoStoring everything is fairly simple and cheap. The complex bit is retrieving/aggregating/searching.