4 ms·
> Even tools people joke about a lot like mysql never gave me this sort of data corruption. People rightfully joked about MySQL when they had the non-ACID engi
by jfkebwjsbx 6y ago
> Even tools people joke about a lot like mysql never gave me this sort of data corruption.
People rightfully joked about MySQL when they had the non-ACID engine.
Same for MongoDB. A database that loses data when properly used is a joke.
Yes, there are use cases out there for fast non-guaranteed writes. No, 99% of companies don’t have them.
- Something1234 6y agoCan you name a use case for a fast non guaranteed write?
- Jweb_Guru 6y agoSure. Data that people don't care about enough to be worried about losing--for example, time series data from an unimportant remote sensor. Should this data be recorded at all? Maybe not, but if should then a best-effort recording may be fine. It may even be all that's possible.
- mbreese 6y agoI wouldn’t go as far as to say an “unimportant” remote sensor... but I think you’re correct in spirit. I could think of an instance where you’d like to log data, but the occasional datapoint being missing wouldn’t be terrible. Maybe something like a temperature monitor — you’d like to have a record of the temperature by the minute, but if a few records dropped out, you’d be able to guess the missing values from context. Something like the data monitoring equivalent of UDP vs TCP.
- throwaway744678 6y agoAnalytics: you don't want to slow down your app, and you don't care if you lose a few records in the process.
- zbentley 6y agoImportantly, in analytics workloads, it is very important to know roughly how many writes aren't making it. Otherwise your analytics system sucks.
- rocho 6y agoInteresting. How would one know that?
- zbentley 6y agoGood question. You'd need some accurate-enough data source telling you about failed writes. Which eventually comes back around to needing a consistent database and indications of client disconnects.
- staticautomatic 6y agoWith a huge amount of data (as I've heard analytics is), could you take a sampling approach where you log every n transaction and only check those against the DB?
- why-el 6y agoEven more elementary that sibling comments, this also happens in gaming all the time. You are recording live results, say in Fifa, but if you unplug your device, your results are gone, since they were memory only. The game simply cannot afford to write to disk, the write is "non guaranteed" in the true sense of the word, but it is fast. You then "checkpoint" when the game is over. You might dissent that is not a "non-guaranted" write, because in fact the write did occur, but I simply want to allude to the concept of a "non-secured" write, in that it vanished without an fsync.
- jfkebwjsbx 6y agoThe number of likes in a given post in your favorite $social-network-of-the-year.
- squiggleblaz 6y agoSince that number may well already be approximate if it's recorded in a hyperlog log.
- twic 6y agoCaches. If you lose a write, you just get a cache miss. Periodic snapshots of state held elsewhere. If you lose a write, you just get stale data until the next update. Firm realtime work. If you lose a write, that sucks, but a slow write sucks just as much.
- threeseed 6y agoTelemetry. I work for a telco where we log large amounts of network requests using MongoDB.