12 ms·
I put my whole life into a single database
- TutleCpt 7mo agoYeah, we've all put our whole lives into a single database. It's called the United States Government.
- tymscar 7mo agoSo it didnt end up working too well seeing that the latest data is from 4 years ago.
- StefanJVA 7mo agoI wonder how much time you spend daily on tracking things / data entry
- brodo 7mo agoThe takeaways at the very bottom of the page are valuable: > Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project. The whole "quantified self" movement might be more about OCD and perfectionism than anything else. /edit: quantified, not qualified
- behehebd 7mo agoI wouldnt have minded if I kept a simple daily journal with a photo a day. If I had that now with LLMs I could ask it "what year did xyz happen".
- nemo1618 7mo agoyep, I do a simple version of this in Google Sheets. Very useful to be able to "Ctrl-F" your life, especially when combined with Google Maps location history.
- stevekemp 7mo agoI had a similar epiphany a few years back when I started wearing a step-tracker/sleep monitor. It was kinda interesting to see how many times I woke up, or track hours, but to be honest I realised after a few months that when my tracker said "You had good sleep", or "You had bad sleep" I was already aware - I woke up smiling, or grumpy depending on how I'd done. I didn't ever look at the data and think "I want to go to bed now to catch up on the four hours I missed yesterday". I continued to have mostly consistent hours, but if I was doing something interesting I'd stay awake, and if I was tired I'd go to bed earlier naturally. The graphs and data wasn't providing anything of value, or encouraging me to change my behaviour in any significant way.
- oldandboring 7mo agoSame. I had a Garmin for about 6 months and I eventually just stopped wearing it and sold it. Knowing how many steps I took today, checking it several times a day to see if I was meeting my goal, knowing how many vertical feet I skied.. none of this data ended up meaning anything to me.
- piva00 7mo agoThe best use I got from my Apple Watch was to use the companion app of my gym routine tracker (to track current loads and personal best) and play music so I didn't have to bring the phone to the gym. That was it, I got extremely annoyed by notifications so over time just disabled them. Also for some reason the heart rate monitor glitched a couple times, got alerts about my BPM at 180+ while I was sitting on the couch. Eventually I just stopped using it and now sits in some drawer.
- ismailmaj 7mo agoIn my experience, tracking objective things like "nutrition" and "sleep hours" is immensely useful to reflect on what went wrong, and tracking subjective things like "mood" or "stress" is useless given hedonic adaptation or heavy swings that make problems obvious, and not need tracking. What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier. I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.
- nicbou 7mo agoI think it's good to track mood swings, because it makes you notice them. After a while it makes you call out your own BS.
- Noaidi 7mo agoAs someone with Schizoaffective Disorder Bipolar Type, if you are not diagnosed with a mood disorder, tracking "swings in you mood" when you have no clinical disorder seems like a disorder of its own. I have had people tell me they were "manic". Then I showed them videos I took when I was manic and they see what I mean when I tell them they are not manic. We have come to a place where we do not want even normal fluctuation in mood, and that is a illness of its own, but it is a cultural illness.
- Ajedi32 7mo agoMaybe for some it's a lot more extreme than for others, but even if it's not so dramatic as to be categorized as a mental illness wouldn't you want to know if, say, there were a direct correlation between whether you went for your morning run and your mood later in the day?
- Noaidi 7mo ago
- lokimedes 7mo agoTaking “Know thyself” to a whole new level. I’d love to have these stats on me, if it could be done by inference, rather than conscious effort.
- deleted 7mo ago[deleted]
- cafkafk 7mo agoI get that everyone wants to be cynical about this, but you really can't deny that both the visualization and sheer scale of data is impressive. The way the "my life in weeks" is done is also very cool, I'll be stealing that for myself.
- pwndByDeath 7mo agoAn interesting experiment, I think I'm too uncomfortable leaking data I don't yet know why someone would curate to me free of charge until I knew. If there was a FOSS suite like Home Assistant that would do a few of those things I might try it out, especially the weather (I would add air quality) correlation to mood and other subjective states.
- __mharrison__ 7mo agoThis might sound harsh but for someone who is keen on investing time to track so many things, you should invest some time in learning how to make better visualizations. A few tweaks here and there would really improve what you have.
- BirAdam 7mo agoThis was far more interesting than I first thought it would be when clicking the link. In particular, the place/time and life events and such being presented this way told a story and was fun.
- tomovo 7mo agoStop bragging, mine fits into a csv file.
- nottorp 7mo agoHope they made backups :)
- egeres 7mo agoRelated to this, I highly recommend anyone to install github.com/ActivityWatch/activitywatch, it's an amazing tool to keep track of your computer use completely locally. I think there are lots of possibilities with data analysis/AI aimed to improved one self's life
- cobertos 7mo agoA shame the Android app hasn't been maintained in a while
- leke 7mo agoI had an idea similar to this where you could add information about yourself and answer daily questions and get paid by companies who access this data. This could be an ethical way to share information benefitting all parties.
- alexpadula 7mo agoI thought you created a database from scratch, got me excited! (I’m a db guy)
- quesera 7mo agoHe did! The database is the data, collected into a coherent structure. The database management system is (optionally) software. :)
- morissette 7mo ago[dead]
- lejalv 7mo agoA simple back of the envelope calculation shows that Felix causes between 70 and 110 tonnes of CO2 emissions per year just from flying. Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.
- glemmaPaul 7mo agoyeah America treats flying as like taking a bus, so I am not at all surprised
- andsoitis 7mo ago> yeah America treats flying as like taking a bus, so I am not at all surprised What does this have to do with Felix?
- deleted 7mo ago[deleted]
- drstewart 7mo agoExactly, you'd never see any airlines get popular in Europe on the back of fares costing less than a bus ticket Unrelated link: https://xcancel.com/Ryanair/status/776292730179682304 https://xcancel.com/Ryanair/status/776292730179682304
- alex_duf 7mo agoExactly, I stopped reading when I saw the flying stats. There are people who still haven't clocked where our climate is headed. I get that you may have to see family abroad or maybe indulge for a holiday, but this is "I'm using an airplane to commute" kind of level. And here I am trying to book my train tickets to go to London instead of flying even though it costs three times as much just to avoid a few kg of CO2 (among other things), it's making me angry.
- criddell 7mo ago> And here I am ... Reminds me of the soggy straw memes floating around now. I've been having those why bother? thoughts as well.
- Tacite 7mo agoI'd like to make the same but with Owntracks instead of Swarm and ActivityWatch instead of RescueTime.
- minkeymaniac 7mo agoThe step count in NYC stands out like One World Trade Center compared to the rest of the building when looking at the skyline ;-)
- sbcorvus 7mo agoThis looks like it requires a heavy amount of discipline to track everything consistently over time. How do you build that into your daily routine?
- slfnflctd 7mo agoCapturing data was the main bottleneck I ran into years ago when I tried something similar with a little MS Access database. I spent lots of time making it pretty and defining fields and data types, but actually typing stuff in? I lost interest quickly. It's possible the friction could be reduced here by having some kind of Generative AI try to help capture data, but then you'd have to verify that it was being done correctly... honestly, I think it's simply not practical for most people to do this. Real life is messy. How much time you want to spend on recordkeeping to make it seem less messy or make you feel like you have more control is up to you. But sometimes it's better to embrace the mess and let go of control, in my opinion. Chances are, no one's going to care about whatever you do here in 100 years. YMMV as always.
- tap-snap-or-nap 7mo agoNo need to spy on him, he publishes everything.
- terabyterex 7mo ago> Back in 2019, I started collecting all kinds of metrics about my life. Every single day for the last 3 years I tracked over 100 different data types - ranging from fitness & nutrition to social life, computer usage and weather. I know this is the type of person i would not like.
- WarcrimeActual 7mo agoYou'll get crap here for it, but I agree. This is someone that has so little to worry about that he has to look for things. Then you look at his travel habits and jobs and you realize he's just one more rich bored guy showing off as best as possible.
- hamasho 7mo agoWhy don’t you just query Palantir DB by your human ID? It shows your entire life data and much more.
- faeyanpiraat 7mo agoIt's all nice and all, but I'm just sitting here thinking "How can one afford all this flying around"?
- duskdozer 7mo agoForget that, I couldn't even fly that much if I was paid to
- WarcrimeActual 7mo agoSpoiler alert. You're reading a rich guys humblebrag site.
- ej31 7mo agoThe value rarely shows up where you expected it to. I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment. Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.
- jirigalis 7mo agoWow, this sounds amazing! I am a statistics and chart lover, I love tracking various types of data, but this? I can't even imagine how much time it must have taken to input all the data. Huge respect to you! Keep going.
- owlcompliance 7mo agoBro queried his entire database, added a LIMIT 10 and it only produced 2 rows. Just a little humor in good faith.
- WarcrimeActual 7mo agoHe's rich. You don't owe him good faith.
- owlcompliance 7mo agoUnderstood - good faith rescinded, bad faith extended.
- WarcrimeActual 7mo agoI don't know if you're joking, but this is exactly how it should work.
- Noumenon72 7mo agoThis might be true if you said it about most people but clearly not him.
- deleted 7mo ago[deleted]
- rodolphoarruda 7mo ago"GitHub Open Source Contributions" aside, everything else falls into the consumption/intake/internal categories; thin data on production/output/external ones.
- vadelfe 7mo agoThis is fascinating. One question I always have with large personal datasets like this: at what point did you start getting genuinely surprising insights versus confirming things you already suspected? 380k datapoints sounds incredible but I imagine the real challenge is turning that into decisions that actually change behavior
- edward 7mo agoI store my travel in YAML files in git. https://edwardbetts.com/agenda/trip/past https://edwardbetts.com/agenda/trip/past https://edwardbetts.com/agenda/trip/stats https://edwardbetts.com/agenda/trip/stats
- ajstars 7mo agoThe hardest part of this kind of personal data system is retrieval not storage. At some point you have more data than fits in a prompt, so you need to decide what's relevant per query. Did you build any ranking or filtering logic, or do you query specific tables directly?
- bede 7mo ago> Probably obvious for many, but I didn't realize ACs don't transport any air into the room, but just moves it around I had the same epiphany as you days after acquiring a CO2 monitor. Most people notice poor indoor air quality from proxies such as humidity and temperature. AC (without ventilation) eliminates these and tricks our senses very effectively, giving us cool and fresh feeling indoor spaces full of CO2 and devoid of oxygen.
- antonyh 7mo agoAs someone who dislikes being tracked, I find it disconcerting to have this level of data stored in 3rd party services, but joining up multiple services to give one coherent picture is cool. How cool is questionable, trying to correlate health data with location data is going to give a strange picture, and I question how relating health data to weather is useful. (Do I have lower blood pressure on rainy days?) Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.
- danesparza 7mo agoDude. Backup that database.
- deleted 7mo ago[deleted]
- spaam 7mo agoHow do you sync the steps data from iOS to your own server automatically?
- bronco21016 7mo agoFlying stats dashboards always amuse me. I get it, for the non-pilot it's kind of like a status thing, "Well I traveled X times in 2025!". As a pilot though, I have gobs of stats I could put up there but from flying for 15 years I realize there's not really anything meaningful in there other than "Gee whiz, I flew a little more/less than last year." I know some other professional pilots do track some of their stats a little closer as they try to optimize for hours flown:hours paid but I've never cared to hyper optimize my schedule in that way.
- Bridged7756 7mo agoI can see the value of doing this but I think that this is a case of the 80-20 rule. Just the low effort 80, which could be keeping a journal where you reflect on how you felt a given day, why that might be, and what steps to take to avoid it from happening (or keep it happening) should get you pretty far. Most of the things that really hinder my productiveness and happiness are the same things that everyone tells you about; sleep, diet, sunlight, not being stuck indoors all day, socializing. And most of these can be improved by making changes to my habits. Other than that, is that I think that it's important to get into the habit of self reflection. Having a feedback loop on my own life has helped me find out what works and what doesn't. It's too easy to just go through the days slogging through and not making any changes because you don't even realize what's going on.
- fidorka 7mo agoTried the manual tracking a few times and it does not tend to stick. Passive capture is the approach that ever worked for me. Having different apps and devices with permissions to track you and good APIs that enable integrating the data seems to be the way to go. Did you think of building some proactive AI tools to make use of all this centralized data?
- ge96 7mo agoTangent, I've always had notes and have this longing to store things but at the same time I like "moving on" too. For ex. I used to have windows phones and saw I still have an accessible One Note from 10 years ago... tempted to read through it but also I've changed as a person. Another tangent, recently bought a paper shredder, started shredding through boxes of mail I have kept since they have personal info like cc statements but on the same idea of moving on/reducing stuff I'm hoarding whether it's data or physical.
- JCharante 7mo ago> I walk more than twice as much when I'm in New York, compared to any other city This is why I moved to Tokyo. Even if I want to avoid exercising I still take many steps
- patchnull 7mo ago[flagged]
- romulussilvia 7mo ago[dead]
- priowise 7mo ago[dead]
- ericwebb 7mo agoBravo. This is my dream and also my nightmare. I was super into the quantified self movement a while back, before Apple Watch, writing Withings Scale API wrappers in Ruby and Fitbit days. Every time I try to seriously track metrics of my life, the excitement of the insight gets worn away by the friction of recording and managing. I expect LLMs can help reduce the cost of this by an order of magnitude but then, as you mention, the question is, what do you do / change / learn because of the data? I recently started tracking nutrition macros with an iOS app MacroFactor which I really like. This is the first time taking my weight doesn't feel like a IDK SHRUG moment and I can actually map my food intake to my weight. Finances is probably the other highly actionable data source that is such high friction to manage (downloading CSVs, OFXs, monthly...) that it has always been a false start for me. I finally wrote a service to talk to Plaid directly and I successfully used it to categorize my business expenses at tax time. I finally have programmatic access to my bank account data! You conclusion is definitely a cautionary take: > the main conclusion is that it is not worth building your own solution, and investing this much time. But, perhaps a subset of that data you find useful.
- mym1990 7mo agoFor the few times that I have tracked myself doing various activities, I have found that I didn't even need to do anything with the data, I was actively changing habits whilst I knew I was being monitored. For example, I wore a GCM for a few weeks and I found it difficult to live my "normal" day to day lifestyle. I wanted to do the things I felt were "right" and "healthy". Whether or not that is a good thing or not is a different topic, but just my 2c.
- ericwebb 7mo agoTotally makes sense. Eventually being confronted with the measurement is its own motivator. Some sort of “future self” awareness or something.
- bigbuppo 7mo agoMmm... yes. It definitely looks like he's in the market for a new mattress.
- esquire_900 7mo agoAnd here I am, feeling slightly ashamed at taking 1 or 2 flights a year. These "commute" statistics are staggering.
- lostlogin 7mo agoNew Zealand almost made the world map.
- davidanekstein 7mo agoIf anyone is interested in doing this sort of thing themselves, I make an iPhone app called Reflect meant for this exact purpose https://apps.apple.com/us/app/reflect-track-anything/id6463800032 https://apps.apple.com/us/app/reflect-track-anything/id64638...
- wlowenfeld 7mo agoThat last takeaway "it's not worth it" hits hard. I went through a similar process and came to the same conclusion, although with a slightly different twist. The project became the point and I almost stopped noticing the data towards the end.
- deleted 7mo ago[deleted]
- RovaAI 7mo ago[flagged]
- themafia 7mo agoYou and Stephen Wolfram. Ironically when I ask Mathematica "what is the mass of Stephen Wolfram" it cannot tell me. I'm honestly not trying to pick on him but I wanted to calculate his average density. What a rip off.
- andai 7mo ago"Unsurprisingly, I'm exposed to less solar energy in Winter" Never heard it phrased like this, and wondering if this phrasing explains the rest of the project as well :)
- themadturk 7mo agoThis reminds me of someone I knew slightly back in the late 70s: Robert Shields, who kept the world's longest diary [0]. He was eccentric but very nice...but he wrote 33 million words, filling 90+ boxes, of every detail of his life for decades, only quitting when a stroke rendered him unable to type about ten years before his death. I doubt he tried to reach any conclusions based on his journaling, though. [0] https://en.wikipedia.org/wiki/Robert_Shields_%28diarist%29 https://en.wikipedia.org/wiki/Robert_Shields_%28diarist%29
- agnishom 7mo agoI strongly recommend reading "The Right to Oblivion" for a different perspective. Great book too.
- aryehof 7mo ago> 2025 Update: I’ve stopped collecting data and working on this project, but I will keep this page alive. I wonder why? Also, perhaps worthwhile putting this at the very top of the page, rather than the very bottom.
- vswaroop04 7mo agoowning your data is powerful