9 ms·

Show HN: iMessage Wrapped – your year in texts

62 points by lxor677 6y ago

lucolsthoorn 6y ago
I made a similar one for facebook messenger messages https://luc-olsthoorn.github.io/FacebookMessengerViz/ https://luc-olsthoorn.github.io/FacebookMessengerViz/
lxor677 6y ago
I got inspired by Spotify Wrapped to put together this - an iMessage Wrapped with data and visualizations on who/where/what/how/when people texted in 2020. Stack is Flask, Jinja, chart.js, leaflet.js and bootstrap. The app runs locally, and accesses the chat and contact SQLite DBs on OS machines. I've been programming for about 18 months, but this was is the first side project I've released into the wild, so it's been an awesome learning experience. Link to the repo is here https://github.com/michael-danello/iMessageWrapped https://github.com/michael-danello/iMessageWrapped
- oefrha 6y ago
  The contact map seems to be based on area code: https://github.com/michael-danello/iMessageWrapped/blob/cef20564e5744fb9a69ec0483ff1e124490b134e/imessage2020wrapped/app.py#L275-L290 https://github.com/michael-danello/iMessageWrapped/blob/cef2... Unfortunately area code of a cell phone number isn't a great indicator of where the owner lives (though to be fair you can't really tell otherwise). Relevant XKCD entry: https://xkcd.com/1129/ https://xkcd.com/1129/ In my case my area code (650) is where I went to college, and many if not most of my friends and colleagues seem to be in a similar position.
  - lxor677 6y ago
    Haha, that XKCD is great. Yeah, this was definitely a recognized constraint for the where section. What I will say is that I recently graduated from college, so my data was actually fairly descriptive - friends' hometowns (where a number of them have moved back) + a whole lot of international students in my college town who got US numbers.
- pvg 6y ago
  I got the impression from the code that this operates directly on whatever sqlite db iMessage actually uses. Is this right? If so, it's probably worth considering just making a copy, or including a script that makes the copy, when sudo'd. That way you don't need the permanent permission and there's no chance of accidentally messing up an important bit of user data. And paranoid people will probably feel better about not having a live local webserver that's poking around their personal messages.
  - lxor677 6y ago
    Totally fair. I'll look into that for a v2. Since the operations are RO, I don't think there's much risk of messing up the data, but I think it makes sense from a context standpoint. The application certainly doesn't need permanent access. Although tbh, one of the reasons I had it was because I tried to orient the setup towards a non-technical audience, so I wanted a 'clickable' solution, like giving full disk access.
    - 67868018 6y ago
      SQLite has write locking, it's not going to happen. And you're only reading anyway...
- gnicholas 6y ago
  Fun, thanks! BTW, the word "represents" is misspelled in the WHO section.
- Zanni 6y ago
  This is very cool. Some minor nits, since you seem to be open to feedback here: 1) Received (not Recieved) 2) represents (not repersents) 3) I have a random late night buddy because I sent no texts between 12 and 3am. 4) The "Good morning sunshine" block tells me my first morning text was sent at 13:39 AM on average, which is not a real time (should be just 13:39 or 1:39 PM) and not in the morning--which further suggests you're taking my overall average text time and not my average morning text time. This might be clearer as "You sent your first text at" (skipping "morning"). 5) I got no days of the week data and no map data. Once again, these are minor nits and didn't detract from my enjoyment at all. Just thought you might appreciate the chance to fix some easy errors.
  - lxor677 6y ago
    Thank you! Appreciate the feedback and glad you enjoyed. Spelling errors are my bane - I'll make sure to fix those. And yep, I think fixing some of those time errors is where I'll start. I feel that pandas is better optimized for handling dates than times, which explains some of my wonky work arounds. Days of the week is also interesting - did you have a full year of imessages on your computer or do you auto-delete after a certain period? The map is puzzling and I'm working to debug. It seems that it works occasionally on reload and that it works consistently if all fade-in animations are removed from page. I put this together in the last two weeks, so it's definitely been rushed. Thanks again for the feedback!
andrethegiant 6y ago
Love it, and would love to see more analysis based on chat.db. A while ago I made a script to select a conversation from a person and output it as a styled PDF, and gave it to them as a gift. It was a hit :-) still need to publish that code, but it was for a previous version of iOS when it used to be sms.db.
- lxor677 6y ago
  Thanks! There's a lot more I would want to include in a V2 - statistics like average contact response time, top phrases, and conversation analysis (did you like to double text? did you have conversations in bursts or consistently?). That said, it's been cool sharing with friends and seeing the results they share with me. Seeing some interest dips/spikes in text frequency around the start of the pandemic depending on the person.
  - andrethegiant 6y ago
    It would be cool to tie this data into a personal CRM so you could e.g. get a reminder when it's been longer than 6 months without texting a person
    - lxor677 6y ago
      I like that! It bugs me endlessly that you can't schedule/get nudged by imessages
TheCount21 6y ago
Cool app, love the design and presentation
- lxor677 6y ago
  Thank you :)
vimy 6y ago
It won't start on my computer. 500 internal server error. ---- [nltk_data] Error loading averaged_perceptron_tagger: <urlopen error [nltk_data] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify [nltk_data] failed: unable to get local issuer certificate [nltk_data] (_ssl.c:1108)> And a nitpick: "Once you’ve done this, you can open the iMessageWrapped.exe file." .exe is windows. .app is macOS.
- lxor677 6y ago
  Thanks vimy - I'll look into that error and change the README
  - vimy 6y ago
    Sorry, that was the wrong error. "[2021-01-01 21:06:27,366] ERROR in app: Exception on / [GET] Traceback (most recent call last): File "flask/app.py", line 2447, in wsgi_app File "flask/app.py", line 1952, in full_dispatch_request File "flask/app.py", line 1821, in handle_user_exception File "flask/_compat.py", line 39, in reraise File "flask/app.py", line 1950, in full_dispatch_request File "flask/app.py", line 1936, in dispatch_request File "imessage2020wrapped/app.py", line 360, in index File "imessage2020wrapped/app.py", line 194, in get_when_data File "imessage2020wrapped/app.py", line 155, in avg_datetime "
    - lxor677 6y ago
      Huh ok. I had a pretty hacky way of calculating average time, so I'll take a look. Thanks for flagging this.
      - ibrarani 6y ago
        I got the same error as well
  - mrala 6y ago
    It seems like the error is related to nltk: https://stackoverflow.com/questions/38916452/nltk-download-ssl-certificate-verify-failed https://stackoverflow.com/questions/38916452/nltk-download-s...
raunakdag 6y ago
I know the answer will be no, but is there any similar way to accomplish this on iPhone without jailbreak?
- deleted 6y ago
  [deleted]
- throwawaysea 6y ago
  This tool relies on having a Mac desktop or laptop, since they can also show iMessages. It doesn’t require a jailbroken phone.
- lights0123 6y ago
  Find a vulnerability in the Messages app like the Safari one a few months ago that allowed you to share its SQLite databases :D?
ibrarani 6y ago
I keep getting Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
zajn 6y ago
Really cool analysis! Weirdly I get a total of 0 messages sent and received, but the `span` has a style that seems to have the correct number. My top 10 data was all there and correct. Nice job!
- adeelk93 6y ago
  Might be a safari/chrome issue
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
firekvz 6y ago
As a side note, not american and not iphone user. the last time i got an sms from a real person was mid 2018 and it was my mom telling me that she had no wifi thus could not answer my whatsapp messages after that, i have received around 2k SMSs, all of them are just notifications from bunch of services (banking, airlines, etc) and 2FA from another bunch of services. Has SMS really died as a form of communication for most of the people? Also, I saw some news about RCS in android, I saw my pixel 4 got it, i set it up but i had noone to use it with
- icelancer 6y ago
  In America for people in their 30s it's the primary method of communication for personal use. I try to push Signal but haven't had much success. It's Slack (or Discord) and iMessage/SMS for the most part. I do communicate internationally though and that's nearly exclusively through WhatsApp. Younger generation seems to be more on Facebook Messenger, Snapchat, and Instagram DMs.
- BiteCode_dev 6y ago
  I keep using a lot of text messages: I have many people with different type of chat apps, or no chat apps at all (old gen), so the only way to reach everybody is doing that instead of juggling with different services. I don't use whatsapp, and don't have an iPhone, so no imessage. Most people don't use Telegram or Signal which I do. Not to mention a lot of geeks have their favorite way of being contacted (slack, wire, etc). And of course all that will have change in a few years. Before you had to have facebook. When in Asia, it was lime or something else. As soon as you have a lot diversity in your life, text messages are the only thing that works all the time, plus they are free in my country. In fact, iMessage is basically sending text messages without the user knowing they do if the other hand is not an Apple device. In the end, I suspect text messages will still be here in 10 years, and all people around me asking to subscribe to yet another chat service will have change 3 times (and lose their contact list in the process).
- swiley 6y ago
  Apple killed internet chat by effectively disallowing community maintained clients on the app store. So your choices are convince all your friends to use a proprietary app/service (with all the downsides) or put up with mms group chats which are free in the US anyway and don't require setting up accounts. I've struggled with this for years and until apple un___ks iOS I think it's going to stay the same.
swiley 6y ago
IME installing OSX and logging into iCloud alone won't give you access to iMessage archives even though it allows you to send and receive them.
tahabroach 6y ago
I just want to make sure that by OSX, you meant macOS, right? Or is it only the older versions?

62 points by lxor677 6y ago

I made a similar one for facebook messenger messages https://luc-olsthoorn.github.io/FacebookMessengerViz/ https://luc-olsthoorn.github.io/FacebookMessengerViz/

lxor677 6y ago

I got inspired by Spotify Wrapped to put together this - an iMessage Wrapped with data and visualizations on who/where/what/how/when people texted in 2020. Stack is Flask, Jinja, chart.js, leaflet.js and bootstrap. The app runs locally, and accesses the chat and contact SQLite DBs on OS machines. I've been programming for about 18 months, but this was is the first side project I've released into the wild, so it's been an awesome learning experience. Link to the repo is here https://github.com/michael-danello/iMessageWrapped https://github.com/michael-danello/iMessageWrapped

oefrha 6y ago
The contact map seems to be based on area code: https://github.com/michael-danello/iMessageWrapped/blob/cef20564e5744fb9a69ec0483ff1e124490b134e/imessage2020wrapped/app.py#L275-L290 https://github.com/michael-danello/iMessageWrapped/blob/cef2... Unfortunately area code of a cell phone number isn't a great indicator of where the owner lives (though to be fair you can't really tell otherwise). Relevant XKCD entry: https://xkcd.com/1129/ https://xkcd.com/1129/ In my case my area code (650) is where I went to college, and many if not most of my friends and colleagues seem to be in a similar position.
- lxor677 6y ago
  Haha, that XKCD is great. Yeah, this was definitely a recognized constraint for the where section. What I will say is that I recently graduated from college, so my data was actually fairly descriptive - friends' hometowns (where a number of them have moved back) + a whole lot of international students in my college town who got US numbers.
pvg 6y ago
I got the impression from the code that this operates directly on whatever sqlite db iMessage actually uses. Is this right? If so, it's probably worth considering just making a copy, or including a script that makes the copy, when sudo'd. That way you don't need the permanent permission and there's no chance of accidentally messing up an important bit of user data. And paranoid people will probably feel better about not having a live local webserver that's poking around their personal messages.
- lxor677 6y ago
  Totally fair. I'll look into that for a v2. Since the operations are RO, I don't think there's much risk of messing up the data, but I think it makes sense from a context standpoint. The application certainly doesn't need permanent access. Although tbh, one of the reasons I had it was because I tried to orient the setup towards a non-technical audience, so I wanted a 'clickable' solution, like giving full disk access.
  - 67868018 6y ago
    SQLite has write locking, it's not going to happen. And you're only reading anyway...
gnicholas 6y ago
Fun, thanks! BTW, the word "represents" is misspelled in the WHO section.
Zanni 6y ago
This is very cool. Some minor nits, since you seem to be open to feedback here: 1) Received (not Recieved) 2) represents (not repersents) 3) I have a random late night buddy because I sent no texts between 12 and 3am. 4) The "Good morning sunshine" block tells me my first morning text was sent at 13:39 AM on average, which is not a real time (should be just 13:39 or 1:39 PM) and not in the morning--which further suggests you're taking my overall average text time and not my average morning text time. This might be clearer as "You sent your first text at" (skipping "morning"). 5) I got no days of the week data and no map data. Once again, these are minor nits and didn't detract from my enjoyment at all. Just thought you might appreciate the chance to fix some easy errors.
- lxor677 6y ago
  Thank you! Appreciate the feedback and glad you enjoyed. Spelling errors are my bane - I'll make sure to fix those. And yep, I think fixing some of those time errors is where I'll start. I feel that pandas is better optimized for handling dates than times, which explains some of my wonky work arounds. Days of the week is also interesting - did you have a full year of imessages on your computer or do you auto-delete after a certain period? The map is puzzling and I'm working to debug. It seems that it works occasionally on reload and that it works consistently if all fade-in animations are removed from page. I put this together in the last two weeks, so it's definitely been rushed. Thanks again for the feedback!

andrethegiant 6y ago

Love it, and would love to see more analysis based on chat.db. A while ago I made a script to select a conversation from a person and output it as a styled PDF, and gave it to them as a gift. It was a hit :-) still need to publish that code, but it was for a previous version of iOS when it used to be sms.db.

lxor677 6y ago
Thanks! There's a lot more I would want to include in a V2 - statistics like average contact response time, top phrases, and conversation analysis (did you like to double text? did you have conversations in bursts or consistently?). That said, it's been cool sharing with friends and seeing the results they share with me. Seeing some interest dips/spikes in text frequency around the start of the pandemic depending on the person.
- andrethegiant 6y ago
  It would be cool to tie this data into a personal CRM so you could e.g. get a reminder when it's been longer than 6 months without texting a person
  - lxor677 6y ago
    I like that! It bugs me endlessly that you can't schedule/get nudged by imessages

TheCount21 6y ago

Cool app, love the design and presentation

lxor677 6y ago
Thank you :)

vimy 6y ago

It won't start on my computer. 500 internal server error. ---- [nltk_data] Error loading averaged_perceptron_tagger: <urlopen error [nltk_data] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify [nltk_data] failed: unable to get local issuer certificate [nltk_data] (_ssl.c:1108)> And a nitpick: "Once you’ve done this, you can open the iMessageWrapped.exe file." .exe is windows. .app is macOS.

lxor677 6y ago
Thanks vimy - I'll look into that error and change the README
- vimy 6y ago
  Sorry, that was the wrong error. "[2021-01-01 21:06:27,366] ERROR in app: Exception on / [GET] Traceback (most recent call last): File "flask/app.py", line 2447, in wsgi_app File "flask/app.py", line 1952, in full_dispatch_request File "flask/app.py", line 1821, in handle_user_exception File "flask/_compat.py", line 39, in reraise File "flask/app.py", line 1950, in full_dispatch_request File "flask/app.py", line 1936, in dispatch_request File "imessage2020wrapped/app.py", line 360, in index File "imessage2020wrapped/app.py", line 194, in get_when_data File "imessage2020wrapped/app.py", line 155, in avg_datetime "
  - lxor677 6y ago
    Huh ok. I had a pretty hacky way of calculating average time, so I'll take a look. Thanks for flagging this.
    - ibrarani 6y ago
      I got the same error as well
- mrala 6y ago
  It seems like the error is related to nltk: https://stackoverflow.com/questions/38916452/nltk-download-ssl-certificate-verify-failed https://stackoverflow.com/questions/38916452/nltk-download-s...

raunakdag 6y ago

I know the answer will be no, but is there any similar way to accomplish this on iPhone without jailbreak?

deleted 6y ago
[deleted]
throwawaysea 6y ago
This tool relies on having a Mac desktop or laptop, since they can also show iMessages. It doesn’t require a jailbroken phone.
lights0123 6y ago
Find a vulnerability in the Messages app like the Safari one a few months ago that allowed you to share its SQLite databases :D?

ibrarani 6y ago

I keep getting Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

zajn 6y ago

Really cool analysis! Weirdly I get a total of 0 messages sent and received, but the `span` has a style that seems to have the correct number. My top 10 data was all there and correct. Nice job!

adeelk93 6y ago
Might be a safari/chrome issue

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

firekvz 6y ago

As a side note, not american and not iphone user. the last time i got an sms from a real person was mid 2018 and it was my mom telling me that she had no wifi thus could not answer my whatsapp messages after that, i have received around 2k SMSs, all of them are just notifications from bunch of services (banking, airlines, etc) and 2FA from another bunch of services. Has SMS really died as a form of communication for most of the people? Also, I saw some news about RCS in android, I saw my pixel 4 got it, i set it up but i had noone to use it with

icelancer 6y ago
In America for people in their 30s it's the primary method of communication for personal use. I try to push Signal but haven't had much success. It's Slack (or Discord) and iMessage/SMS for the most part. I do communicate internationally though and that's nearly exclusively through WhatsApp. Younger generation seems to be more on Facebook Messenger, Snapchat, and Instagram DMs.
BiteCode_dev 6y ago
I keep using a lot of text messages: I have many people with different type of chat apps, or no chat apps at all (old gen), so the only way to reach everybody is doing that instead of juggling with different services. I don't use whatsapp, and don't have an iPhone, so no imessage. Most people don't use Telegram or Signal which I do. Not to mention a lot of geeks have their favorite way of being contacted (slack, wire, etc). And of course all that will have change in a few years. Before you had to have facebook. When in Asia, it was lime or something else. As soon as you have a lot diversity in your life, text messages are the only thing that works all the time, plus they are free in my country. In fact, iMessage is basically sending text messages without the user knowing they do if the other hand is not an Apple device. In the end, I suspect text messages will still be here in 10 years, and all people around me asking to subscribe to yet another chat service will have change 3 times (and lose their contact list in the process).
swiley 6y ago
Apple killed internet chat by effectively disallowing community maintained clients on the app store. So your choices are convince all your friends to use a proprietary app/service (with all the downsides) or put up with mms group chats which are free in the US anyway and don't require setting up accounts. I've struggled with this for years and until apple un___ks iOS I think it's going to stay the same.

swiley 6y ago

IME installing OSX and logging into iCloud alone won't give you access to iMessage archives even though it allows you to send and receive them.

tahabroach 6y ago

I just want to make sure that by OSX, you meant macOS, right? Or is it only the older versions?