4 ms·
In my opinion, knowing what you're planning to do w/the data once it's stored is the important piece to giving you some idea of where to put it.
by garciasn 6y ago
In my opinion, knowing what you're planning to do w/the data once it's stored is the important piece to giving you some idea of where to put it.
- boringg 6y agoAgreed - though I feel like every data use comes after the fact. Original software engineers/developpers rarely have the foresight that the data scientists need the information for (at least imho).
- lostcolony 6y agoTo be fair, the data scientists rarely have the foresight to know what the data scientists need the information for. The only time I've seen a data scientist correctly include all the data they needed (but still be wrong) was when they answered "All of it. We need all of the data".
- boringg 6y agoSo true. Tough to know in advance which data will hold the secrets.
- monkeybutton 6y agoAnd you can't build a time machine to go and get it once you do know. Want X days of historical data for training/backtesting and we just implemented the metric this sprint? Good luck meeting your deadline!
- philippb 6y agoGood point. I updated the post with some more infos
- x0x0 6y agoWhat is your loss tolerance? If a file is gone, who is annoyed: a free user, a $50/year customer, or a $10k/year customer? Are these files WORM?
- canadianfella 6y agoInfo