5 ms·

IoT in general has this problem. I live a highly automated life and my database has when I woke up, where I drive, when I get home, what TV shows I watch, how

by throwaway2016a 10y ago

IoT in general has this problem. I live a highly automated life and my database has when I woke up, where I drive, when I get home, what TV shows I watch, how long I spend on Hacker News and when, when I enter the house it is recorded... and then more benign stuff like how long my air conditioner has been on. And and I do this voluntarily. Overall it's a gig of data a day. The weird thing is my wife has access to this data too and I have access to her's (which is less extensive but still there... for example, if her code is used to disarm the alarm, it is stored that way). I also might be crazy, most people would be disturbed with this level of data. As a side note, I store the database on my own server and I am very comfortable with the security on said server. Firewalls, tripwires, multi-factor auth, encryption, the works. Even I wouldn't trust that database with a third party.

TeMPOraL 10y ago
I wouldn't be disturbed with this level of data - I would be delighted. Being disturbed signals lack of touch with how reality works. That said, I absolutely, positively don't want this data to be owned by third parties. It should be my data, for my perusal.
- throwaway2016a 10y ago
  I agree completely. And the data certainly delights me.[1] I don't think most Hacker News users would be disturbed by it. I was generalizing (without evidence, mind you) the larger population. [1] One nice thing about owning the database is if I want to, say, stick it in an R program and graph some stuff / make some models I can. Which as a software engineer is fun for days and has allowed me to see and correct and/or automate patterns. For example, my bank accounts and credit cards are also in this database and it has allowed me to see spending patterns that I needed to fix.
- k-mcgrady 10y ago
  >> "Being disturbed signals lack of touch with how reality works." Could you explain what you mean by this? That level of data collection disturbs me for privacy reasons. I don't see how I lack touch with reality. It's interesting and I would like to have all that data on myself too but the privacy risks make it not worth it for me.
  - TeMPOraL 10y ago
    I was talking about being disturbed by existence/collectability of such data - not by collection by third parties, which is disturbing for privacy reasons. You radiate tons of data every second of your life. Your every interaction with every thing is recorded in the fabric of reality, and progress of technology gives us ways to access more and more of that information. My point is - existence of such data should not be disturbing. How much of it is being collected by third parties - that's another story.
cdnsteve 10y ago
But why?
- throwaway2016a 10y ago
  Which part? Why do I do it? To make myself more efficient. Automate things I have to do but that don't make me happy. Improve my physical health (I'm motivated by numbers... they get me to the gym). Make more informed decisions. Basically I run my life like a business. I track KPIs[1] and use them to plan and act. For example, right now my app is telling me I'm spending more than my average amount of time on Hacker News. And based on that, I can make the decision I should probably get some work done :) Also, if I ever get murdered, unlikely for sure, the cops will have an easy job ;) (joking, this isn't a primary motivation but, certainly a side effect). Edit: plus I think it is fun, it's a hobby. Everyone needs a hobby :) [1] Key performance indicators
  - serg_chernata 10y ago
    Do you have a writeup of the tools you use? I'd love to see something like that.
    - throwaway2016a 10y ago
      No writeup and I haven't open sourced (yet) but the gist of it is: For my software: - Node.js - MongoDB - Ubuntu Server - Docker - Java for my mobile app - Electron for the desktop app For hardware: - Galaxy Note 5 using the accelerometer and GPS - Macbook Pro running custom monitoring software - Fitbit smart scale - A lot of Aurdinos - A few Raspberry Pis - A few Spark Photons - A Vera for home automation and integration with household sensors - A whole slew of Zwave and Wifi house sensors and switches - The cameras and alarm are through ADT Pulse but to be honest I wish I had used a more open solution. Everything runs through their servers, I can't talk directly to the cameras. - Some more home automation stuff that is not really quantified self (more IoT) so I'm not including (Nest thermostats, myQ garage door opener, etc) For protocols (you'd be surprised at how many devices you can get data from knowing these three protocols): - UPnP - Bonjour - HTTP - IMAP For APIs: - Fitbit - Facebook - Twitter - IMAP for email (not an API in the sense people think of APIs but still...) - Nest - A bunch of stuff that I had to reverse engineer because they don't offer public APIs For analysis: - R mostly For reverse engineering hardware devices that don't open their APIs: - Charles for a proxy to sniff traffic (if they have an app) - Wireshark for devices that don't obey proxies I'm sure I forgot something. As you can see, it is a LOT of work. But it's a hobby so I enjoy it. I want to open source it one day but as you can imagine... none of this is unit tested or meets any of the criteria I would use for an open source project. It is, in effect, a "hack"... but it works.
FilterSweep 10y ago
Question for you: Have you done any analysis on the data? I'm wondering if you can find out if you're coming down with an illness before your body even exhibits the symptoms. Here's the implication of why I say this: IoT companies who sell this data to third parties will then have this bit of leverage over you. The next time you browse Amazon/FB/News your browser, sans-adblock, will be just as infested with ads as your body is with the cold.
- throwaway2016a 10y ago
  I often wonder if there is enough data to do that but I don't have a medical background so I wouldn't know how. Part of the dataset is my output from 23 and Me, as well as my steps, heart rate, weight, BMI, and caloric intake. I'd love to gather more but blood pressure, blood sugar, body temperature, and the like are semi-invasive to gather and I avoid gathering data I need to do daily manual work for. The only exceptions I make are weight lifted and repetitions when working out and only because I haven't found a good way to do it and it... I've been playing with Arduinos and accelerometers and measurements (waist, chest, thighs, etc) which I only do once a month. So in theory it is possible. But in any case, per your motivation behind the question. I avoid devices that report data to a central server so advertisers using it would be hard pressed, but not all consumers are as careful. I can't avoid it entirely without making my own hardware. For example my Fitbit data goes to Fitbit, my treadmill data goes to iFit, and 23 and Me has my DNA sequence. But I think you'd need all the pieces to it all together and there is no third party that has them all.
  - j_s 10y ago
    These guys claim to count your reps: https://www.atlaswearables.com/ https://www.atlaswearables.com/
    - throwaway2016a 10y ago
      Thank you, I've seen it but forgot about it. At the time I couldn't convince my wife to let me spend $200 on it so I've been trying to make my own :) Maybe she'll get one for my birthday.
jbott 10y ago
What does this consist of? Is it an open source project? I would love to set up something like this, but I constantly find myself not having the time.
- throwaway2016a 10y ago
  Home grown. Node.js running on a server in my basement. I do plan to open source it someday. It was a lot of work. Some devices I had to reverse engineer.[1] [1] A little more detail on that because I'm sure they might find it interesting: - If there is a phone, Linux, or Mac app I use Charles to try to intercept the traffic - If that fails I use Wireshark - If that fails I decompile their app - If none of those work, I have to ditch the hardware and try new hardware.
  - flatulent1 10y ago
    Don't forget the possibility of data being repeated by your electric meter even if you don't provide WiFi. They're generally dual band, operating at WiFi frequencies for local interaction and with more power at a lower frequency for the networked traffic. While the protocols are not intended to be WiFi compatible, the already-proven-hackable devices are just a patch away from unexpected functionality. The local link is intended for things like telling your air conditioner not to run part of the time when the grid supply doesn't have much excess capacity. What other data is sitting around? How about raster images of whatever your "copier" printer fax has handled? Even if you're using USB, if there's WiFi capability in there, assume it can be awakened. A link that's not compatible with your sniffer is a possibility too. Drag out a spectrum analyzer and directional antenna. While you're at it, check those new LED streetlights for bursts of r.f. too.
softawre 10y ago
Here's an (older but interesting) interview with the "world's most quantified man": http://hanselminutes.com/399/chris-dancy-the-worlds-most-quantified-man-explains-the-quantified-self http://hanselminutes.com/399/chris-dancy-the-worlds-most-qua...
- throwaway2016a 10y ago
  I'd be lying if I said this exact article wasn't behind some of the ideas I've used to build my own system. This was one of the first times I've heard the term Quantified Self. Now I see "myself" as just a thing in the Internet of Things that I can track metrics on (and tell to do things) just like any other thing. Don't take me philosophically or spiritually with that statement. As a human I am trained to think of myself as more than "just a thing" and to think otherwise would be a bit morbid :) but from an architectural standpoint I don't treat my own metrics different than the rest of the 200+ devices in my IoT network.
XorNot 10y ago
How have you implemented multi-factor? Near as I can tell there's no good open-source ways of managing it (i.e. something where I get Google-style app-passwords available via LDAP to connected applications + a UI for managing them).
- throwaway2016a 10y ago
  Google Authenticator uses an open time-based algorithm that doesn't even require a server. Both sides just need to have the same seed and the same time on their clock (you can configure server-side how much clock drift to allow). Authy supports the algorithm as well, they just don't advertise it because they want you to use their's. It's actually an RFC https://tools.ietf.org/html/rfc6238 https://tools.ietf.org/html/rfc6238 and there are implementations for almost any language.

TeMPOraL 10y ago

I wouldn't be disturbed with this level of data - I would be delighted. Being disturbed signals lack of touch with how reality works. That said, I absolutely, positively don't want this data to be owned by third parties. It should be my data, for my perusal.

throwaway2016a 10y ago
I agree completely. And the data certainly delights me.[1] I don't think most Hacker News users would be disturbed by it. I was generalizing (without evidence, mind you) the larger population. [1] One nice thing about owning the database is if I want to, say, stick it in an R program and graph some stuff / make some models I can. Which as a software engineer is fun for days and has allowed me to see and correct and/or automate patterns. For example, my bank accounts and credit cards are also in this database and it has allowed me to see spending patterns that I needed to fix.
k-mcgrady 10y ago
>> "Being disturbed signals lack of touch with how reality works." Could you explain what you mean by this? That level of data collection disturbs me for privacy reasons. I don't see how I lack touch with reality. It's interesting and I would like to have all that data on myself too but the privacy risks make it not worth it for me.
- TeMPOraL 10y ago
  I was talking about being disturbed by existence/collectability of such data - not by collection by third parties, which is disturbing for privacy reasons. You radiate tons of data every second of your life. Your every interaction with every thing is recorded in the fabric of reality, and progress of technology gives us ways to access more and more of that information. My point is - existence of such data should not be disturbing. How much of it is being collected by third parties - that's another story.

cdnsteve 10y ago

But why?

throwaway2016a 10y ago
Which part? Why do I do it? To make myself more efficient. Automate things I have to do but that don't make me happy. Improve my physical health (I'm motivated by numbers... they get me to the gym). Make more informed decisions. Basically I run my life like a business. I track KPIs[1] and use them to plan and act. For example, right now my app is telling me I'm spending more than my average amount of time on Hacker News. And based on that, I can make the decision I should probably get some work done :) Also, if I ever get murdered, unlikely for sure, the cops will have an easy job ;) (joking, this isn't a primary motivation but, certainly a side effect). Edit: plus I think it is fun, it's a hobby. Everyone needs a hobby :) [1] Key performance indicators
- serg_chernata 10y ago
  Do you have a writeup of the tools you use? I'd love to see something like that.
  - throwaway2016a 10y ago
    No writeup and I haven't open sourced (yet) but the gist of it is: For my software: - Node.js - MongoDB - Ubuntu Server - Docker - Java for my mobile app - Electron for the desktop app For hardware: - Galaxy Note 5 using the accelerometer and GPS - Macbook Pro running custom monitoring software - Fitbit smart scale - A lot of Aurdinos - A few Raspberry Pis - A few Spark Photons - A Vera for home automation and integration with household sensors - A whole slew of Zwave and Wifi house sensors and switches - The cameras and alarm are through ADT Pulse but to be honest I wish I had used a more open solution. Everything runs through their servers, I can't talk directly to the cameras. - Some more home automation stuff that is not really quantified self (more IoT) so I'm not including (Nest thermostats, myQ garage door opener, etc) For protocols (you'd be surprised at how many devices you can get data from knowing these three protocols): - UPnP - Bonjour - HTTP - IMAP For APIs: - Fitbit - Facebook - Twitter - IMAP for email (not an API in the sense people think of APIs but still...) - Nest - A bunch of stuff that I had to reverse engineer because they don't offer public APIs For analysis: - R mostly For reverse engineering hardware devices that don't open their APIs: - Charles for a proxy to sniff traffic (if they have an app) - Wireshark for devices that don't obey proxies I'm sure I forgot something. As you can see, it is a LOT of work. But it's a hobby so I enjoy it. I want to open source it one day but as you can imagine... none of this is unit tested or meets any of the criteria I would use for an open source project. It is, in effect, a "hack"... but it works.

FilterSweep 10y ago

Question for you: Have you done any analysis on the data? I'm wondering if you can find out if you're coming down with an illness before your body even exhibits the symptoms. Here's the implication of why I say this: IoT companies who sell this data to third parties will then have this bit of leverage over you. The next time you browse Amazon/FB/News your browser, sans-adblock, will be just as infested with ads as your body is with the cold.

throwaway2016a 10y ago
I often wonder if there is enough data to do that but I don't have a medical background so I wouldn't know how. Part of the dataset is my output from 23 and Me, as well as my steps, heart rate, weight, BMI, and caloric intake. I'd love to gather more but blood pressure, blood sugar, body temperature, and the like are semi-invasive to gather and I avoid gathering data I need to do daily manual work for. The only exceptions I make are weight lifted and repetitions when working out and only because I haven't found a good way to do it and it... I've been playing with Arduinos and accelerometers and measurements (waist, chest, thighs, etc) which I only do once a month. So in theory it is possible. But in any case, per your motivation behind the question. I avoid devices that report data to a central server so advertisers using it would be hard pressed, but not all consumers are as careful. I can't avoid it entirely without making my own hardware. For example my Fitbit data goes to Fitbit, my treadmill data goes to iFit, and 23 and Me has my DNA sequence. But I think you'd need all the pieces to it all together and there is no third party that has them all.
- j_s 10y ago
  These guys claim to count your reps: https://www.atlaswearables.com/ https://www.atlaswearables.com/
  - throwaway2016a 10y ago
    Thank you, I've seen it but forgot about it. At the time I couldn't convince my wife to let me spend $200 on it so I've been trying to make my own :) Maybe she'll get one for my birthday.

jbott 10y ago

What does this consist of? Is it an open source project? I would love to set up something like this, but I constantly find myself not having the time.

throwaway2016a 10y ago
Home grown. Node.js running on a server in my basement. I do plan to open source it someday. It was a lot of work. Some devices I had to reverse engineer.[1] [1] A little more detail on that because I'm sure they might find it interesting: - If there is a phone, Linux, or Mac app I use Charles to try to intercept the traffic - If that fails I use Wireshark - If that fails I decompile their app - If none of those work, I have to ditch the hardware and try new hardware.
- flatulent1 10y ago
  Don't forget the possibility of data being repeated by your electric meter even if you don't provide WiFi. They're generally dual band, operating at WiFi frequencies for local interaction and with more power at a lower frequency for the networked traffic. While the protocols are not intended to be WiFi compatible, the already-proven-hackable devices are just a patch away from unexpected functionality. The local link is intended for things like telling your air conditioner not to run part of the time when the grid supply doesn't have much excess capacity. What other data is sitting around? How about raster images of whatever your "copier" printer fax has handled? Even if you're using USB, if there's WiFi capability in there, assume it can be awakened. A link that's not compatible with your sniffer is a possibility too. Drag out a spectrum analyzer and directional antenna. While you're at it, check those new LED streetlights for bursts of r.f. too.

softawre 10y ago

Here's an (older but interesting) interview with the "world's most quantified man": http://hanselminutes.com/399/chris-dancy-the-worlds-most-quantified-man-explains-the-quantified-self http://hanselminutes.com/399/chris-dancy-the-worlds-most-qua...

throwaway2016a 10y ago
I'd be lying if I said this exact article wasn't behind some of the ideas I've used to build my own system. This was one of the first times I've heard the term Quantified Self. Now I see "myself" as just a thing in the Internet of Things that I can track metrics on (and tell to do things) just like any other thing. Don't take me philosophically or spiritually with that statement. As a human I am trained to think of myself as more than "just a thing" and to think otherwise would be a bit morbid :) but from an architectural standpoint I don't treat my own metrics different than the rest of the 200+ devices in my IoT network.

XorNot 10y ago

How have you implemented multi-factor? Near as I can tell there's no good open-source ways of managing it (i.e. something where I get Google-style app-passwords available via LDAP to connected applications + a UI for managing them).

throwaway2016a 10y ago
Google Authenticator uses an open time-based algorithm that doesn't even require a server. Both sides just need to have the same seed and the same time on their clock (you can configure server-side how much clock drift to allow). Authy supports the algorithm as well, they just don't advertise it because they want you to use their's. It's actually an RFC https://tools.ietf.org/html/rfc6238 https://tools.ietf.org/html/rfc6238 and there are implementations for almost any language.