3 ms·

This posting was great. Highly recommended reading through. It gets really good when the author hits "Data is a Liability". > An alternate definition of Big Da

This posting was great. Highly recommended reading through. It gets really good when the author hits "Data is a Liability". > An alternate definition of Big Data is “when the cost of keeping data around is less than the cost of figuring out what to throw away.” This is exactly it. It's way too hard to go through and make decisions about what to throw away. In many respects, companies are the ultimate hoarders and can't fathom throwing any data way, Just In Case. Really appreciated the post overall. Very insightful. As an anecdote to this article, when business folks have come up to me and asked about storing their data in a Big Data facility, I have never found the justification to recommend it. Like, if your data can fit into RAM, what exactly are we talking about Big Data for?

disqard 4y ago
> if your data can fit into RAM, what exactly are we talking about Big Data for? That's a fantastic point, and I keep mentioning the COST paper to anyone who cares: https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos...
- DriverDaily 4y ago
  COVID was proof The Right Data is better than Big Data. All those data sources to measure how many sick people we have and it turns out we just need one: Wastewater.
  - bombcar 4y ago
    Or another way to look at it - if we make a data lake that collects everyone’s shit we might find something useful in it!
CliffStoll 4y ago
In a larger sense, it's a challenge to throw away stuff, just as it's difficult to trim big data. As I reach retirement, our attic, bookshelves, and cabinets must be trimmed -- and each item requires attention and a decision. Some things in the attic are obvious liabilities (what to do with a mercury barometer? A radium dial pocket watch? Old electronics?) Disposing of other stuff requires time, insight, and a sense of the future (should we keep those fingerpainted scribbles from when the kids were 3? How about those cheesy trophies from chess club? Computer books from the 1970's? Betamax home movies? Record albums?)
- selestify 4y ago
  Where is this excerpt from? Did you write it yourself?
- roeles 4y ago
  For what it's worth: I throw things away when I haven't touched them in a year.

disqard 4y ago

> if your data can fit into RAM, what exactly are we talking about Big Data for? That's a fantastic point, and I keep mentioning the COST paper to anyone who cares: https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos...

DriverDaily 4y ago
COVID was proof The Right Data is better than Big Data. All those data sources to measure how many sick people we have and it turns out we just need one: Wastewater.
- bombcar 4y ago
  Or another way to look at it - if we make a data lake that collects everyone’s shit we might find something useful in it!

CliffStoll 4y ago

In a larger sense, it's a challenge to throw away stuff, just as it's difficult to trim big data. As I reach retirement, our attic, bookshelves, and cabinets must be trimmed -- and each item requires attention and a decision. Some things in the attic are obvious liabilities (what to do with a mercury barometer? A radium dial pocket watch? Old electronics?) Disposing of other stuff requires time, insight, and a sense of the future (should we keep those fingerpainted scribbles from when the kids were 3? How about those cheesy trophies from chess club? Computer books from the 1970's? Betamax home movies? Record albums?)

selestify 4y ago
Where is this excerpt from? Did you write it yourself?
roeles 4y ago
For what it's worth: I throw things away when I haven't touched them in a year.