4 ms·

What tools do you use to collect this data? Looks great, by the way.

by arscan 14y ago

jehiah 14y ago
a lot of it is just simple python scripts to download/convert data to csv files, and do some simple post processing. for the computer activity i took a snapshot every 30seconds of what program was active, and how long my laptop was active.
- nphase 14y ago
  Are these tools open source?
  - jehiah 14y ago
    The script I used for extracting sms/iMessages is here https://gist.github.com/4437883 https://gist.github.com/4437883 Some of the other scripts just download data from foursquare's API, and post processing transaction download from Mint. Hopefully I'll have time to clean and post those scripts.
    - mikeevans 14y ago
      I'd be interested in seeing the one(s) that keep track of application usage!
      - jehiah 14y ago
        I've dumped the cron job i use for data collection here https://gist.github.com/4477372 https://gist.github.com/4477372 My rough approach to post processing this data was to group by 6 minute intervals, and to drop any records where the computer was inactive for more than 120 seconds.
        sbouafif 14y ago
        Many thanks for the script :)
        lucaspiller 14y ago
        I've tweaked it a bit here https://gist.github.com/4478297 https://gist.github.com/4478297 - Logs active website from Chrome - Stores time including local timezone (interesting to see what timezones I'm in) - File name includes hostname
- arscan 14y ago
  So you parse your credit card statements (or equivalent) for the coffee & travel costs? I thought the travel was by distance (not cost) at first, which is what I was curious about because it would require a more active role in logging on your part. Is this purely for kicks, or are you trying to use it to drive behavioral changes (e.g. spend less on coffee)? I ask because I always think about doing this, but I get a stuck on the "why".
  - jehiah 14y ago
    coffee data is more driven from foursquare checkins, and only partially credit card transactions. The motivation for me is a way to learn about my year (not directly to change my actions), and part a fun exercise to get better at data processing, javascript, and UI Interaction/charting. I like the hard deadline that a personal annual report creates naturally (ie: it's no good if i release it in june).
enobrev 14y ago
For collecting application usage data, I used to use an application called Timesnapper ( http://www.timesnapper.com/ http://www.timesnapper.com/ ), which takes a screenshot every x seconds and keeps track of the application title / etc, with the added benefit of allowing you to automatically categorize usage "Work", "Play", etc. Found it really useful to be able to literally play back my day in 10 second intervals. Unfortunately it's windows-only. One of the very few apps I truly miss since moving to Linux almost 2 years ago. -- I have absolutely no affiliation with TimeSnapper besides being a happily paying customer a couple years ago.
- perssontm 14y ago
  I think there is some package for ubuntu that tracked everything written on the keyboard, like a personal keylogger. It was mentioned on HN during december, cant find it now though.
  - josyw 14y ago
    Selfspy? (http://news.ycombinator.com/item?id=4590987 http://news.ycombinator.com/item?id=4590987)
- gwern 14y ago
  You could use arbtt (http://www.joachim-breitner.de/projects#arbtt http://www.joachim-breitner.de/projects#arbtt) and then just toss your favorite screenshot utility like ImageMagick into crontab set to fire every 10 seconds.
  - enobrev 14y ago
    Definitely interesting, and I'll probably try it at some point should I find both the time and the inclination to do so. Admittedly, I paid for that application because I didn't want to implement it myself. But I'm glad there are some building blocks in place that would make rolling my own a relatively easy task. Thanks for the suggestion!
neetij 14y ago
Checkout LifeSlice by Stan James (https://github.com/wanderingstan/Lifeslice https://github.com/wanderingstan/Lifeslice) for OSX - the script takes a screenshot, picture of the user and tracks web browser usage. The sample report leaves much to be desired from a design aesthetic but the data is available.

What tools do you use to collect this data? Looks great, by the way.

by arscan 14y ago

What tools do you use to collect this data? Looks great, by the way.

a lot of it is just simple python scripts to download/convert data to csv files, and do some simple post processing. for the computer activity i took a snapshot every 30seconds of what program was active, and how long my laptop was active.

nphase 14y ago
Are these tools open source?
- jehiah 14y ago
  The script I used for extracting sms/iMessages is here https://gist.github.com/4437883 https://gist.github.com/4437883 Some of the other scripts just download data from foursquare's API, and post processing transaction download from Mint. Hopefully I'll have time to clean and post those scripts.
  - mikeevans 14y ago
    I'd be interested in seeing the one(s) that keep track of application usage!
    - jehiah 14y ago
      I've dumped the cron job i use for data collection here https://gist.github.com/4477372 https://gist.github.com/4477372 My rough approach to post processing this data was to group by 6 minute intervals, and to drop any records where the computer was inactive for more than 120 seconds.
      - sbouafif 14y ago
        Many thanks for the script :)
      - lucaspiller 14y ago
        I've tweaked it a bit here https://gist.github.com/4478297 https://gist.github.com/4478297 - Logs active website from Chrome - Stores time including local timezone (interesting to see what timezones I'm in) - File name includes hostname
arscan 14y ago
So you parse your credit card statements (or equivalent) for the coffee & travel costs? I thought the travel was by distance (not cost) at first, which is what I was curious about because it would require a more active role in logging on your part. Is this purely for kicks, or are you trying to use it to drive behavioral changes (e.g. spend less on coffee)? I ask because I always think about doing this, but I get a stuck on the "why".
- jehiah 14y ago
  coffee data is more driven from foursquare checkins, and only partially credit card transactions. The motivation for me is a way to learn about my year (not directly to change my actions), and part a fun exercise to get better at data processing, javascript, and UI Interaction/charting. I like the hard deadline that a personal annual report creates naturally (ie: it's no good if i release it in june).

enobrev 14y ago

For collecting application usage data, I used to use an application called Timesnapper ( http://www.timesnapper.com/ http://www.timesnapper.com/ ), which takes a screenshot every x seconds and keeps track of the application title / etc, with the added benefit of allowing you to automatically categorize usage "Work", "Play", etc. Found it really useful to be able to literally play back my day in 10 second intervals. Unfortunately it's windows-only. One of the very few apps I truly miss since moving to Linux almost 2 years ago. -- I have absolutely no affiliation with TimeSnapper besides being a happily paying customer a couple years ago.

perssontm 14y ago
I think there is some package for ubuntu that tracked everything written on the keyboard, like a personal keylogger. It was mentioned on HN during december, cant find it now though.
- josyw 14y ago
  Selfspy? (http://news.ycombinator.com/item?id=4590987 http://news.ycombinator.com/item?id=4590987)
gwern 14y ago
You could use arbtt (http://www.joachim-breitner.de/projects#arbtt http://www.joachim-breitner.de/projects#arbtt) and then just toss your favorite screenshot utility like ImageMagick into crontab set to fire every 10 seconds.
- enobrev 14y ago
  Definitely interesting, and I'll probably try it at some point should I find both the time and the inclination to do so. Admittedly, I paid for that application because I didn't want to implement it myself. But I'm glad there are some building blocks in place that would make rolling my own a relatively easy task. Thanks for the suggestion!

neetij 14y ago

Checkout LifeSlice by Stan James (https://github.com/wanderingstan/Lifeslice https://github.com/wanderingstan/Lifeslice) for OSX - the script takes a screenshot, picture of the user and tracks web browser usage. The sample report leaves much to be desired from a design aesthetic but the data is available.