6 ms·

I'm curious. How else would you implement a cloud-based recording service with image recognition? (EDIT: full disclosure, I work for Nest through the Dropcam ac

I'm curious. How else would you implement a cloud-based recording service with image recognition? (EDIT: full disclosure, I work for Nest through the Dropcam acquisition)

afandian 11y ago
That's kind of like saying "how would you make a car without wheels?". It could be that inherent in the idea of making object X are privacy concerns. The fact that the privacy concerns are intrinsic to the object just means that you have to call the whole object X into question when discussing them. Your question seems to suggest a sentiment like "there isn't any other way to do this, can you think of one?" but there is, and it's not to do it at all. (I'm not taking a position on the Nest device, but this comment just seemed a little like a cognitive bias similar to anchoring)
- dguaraglia 11y ago
  Exactly! And I'm OK with people taking the stance "if it's not 100% private and my video needs to be sent over the internet, then I don't want the camera". You can either have a camera that keeps a safe copy of your video in the cloud and detects unexpected activity happening in front of your camera, or you can have one that is 100% private and doesn't upload video to the cloud. You can't have both. At least not without having a huge setup in a secured room inside your home. I understand the privacy concerns, I just argue that you can't have the ideal service without getting over them. At least not with current technology, and probably not without the right economic incentives to build a stand-alone system.
  - bloomingfractal 11y ago
    What about doing all processing locally and storing only end-to-end encrypted data in servers?
    - dguaraglia 11y ago
      Not really an option right now. The current generation of cameras out there are basically a Raspberry-Pi level computer with a better camera and a hardware h264 encoder. The moment you start doing something fancy, like running any non-trivial motion-detection algorithm, you are bound to run into performance or thermal (read: overheating) issues. Let's not even talk about machine learning. Just think how much money Nest would save in server time with such a setup :)
      - bloomingfractal 11y ago
        I see, Thanks for the insightful answer! Regarding machine learning it would be nice to be able do the training in your desktop/laptop when it's idle or something like that. But it's very good to know this is a technological issue (as opposed to a business issue). Well, hope you smart folks solve this. Meanwhile, I'll keep tinkering with my raspberry pi and raspberry pi camera :)
      - lazaroclapp 11y ago
        My intuition says we are not anywhere near there yet, but do you know if any video processing algorithms exist that can reasonably be executed on encrypted data? Basically, I know fully-homomorphic encryption is ridiculously inefficient in the general case. At the same time, I know of specialized homomorphic encryption algorithms that can operate on encrypted data of specific formats. There are efficient-ish algorithms for encrypted (social-network-type) graphs, and encrypted vote ballots. I was wondering if you or anyone in your team has come across any work on privacy-preserving encrypted audio/image/video processing? I assume this is a very hard problem, but I imagine someone has tried looking into it.
        dguaraglia 11y ago
        I'm not really the person to answer that (I'm a lowly software engineer keeping the cogs greased m'lord!) I know at some point Larry (https://github.com/lwneal https://github.com/lwneal) was looking into that, at least cursorily. I'll refer to him as the authority on anything encryption-related at Dropcam (or anything, in general. Brilliant guy!)
bhauer 11y ago
> a cloud-based recording service Well, there's the rub, right? I think what those of us who consider themselves self-hosting partisans would say is that we'd prefer a device that allows us to send its signals to a server we own and operate. The recording and image recognition would then occur on that server. In my ideal world, Dropcam (and later Nest) would have provided software I could install on my own server (located either on my home network or at a data center). I know that sounds like a far-fetched dream, but that's the ideal I want, and I will pay twice as much or more for devices that recognize the emergent demand for self-management. I have a Dropcam, and I enjoyed using it for a while. But I stopped using my Dropcam about a year ago because I grew increasingly unhappy with the idea of its video stream being sent to an untrusted third-party (Dropcam and later Google/Nest). Since there is no "self-service" mode for Dropcams, it has become a decoration in my house.
- dguaraglia 11y ago
  I don't know why the comment is being downvoted. There is a bunch of people that want what you want, it's just that, as you are saying, is a bit of a pipe dream right now. The way technology is progressing, it might be feasible in a few years.
  - arthur_pryor 11y ago
    why is it a technology issue? not that there's never gray area, but this looks like more of a business issue than a technology issue. what's desired there (sending video to a server instance the user controls, instead of a third party vendor's servers) seems entirely feasible with current technology, though not necessarily profitable (or perceived as profitable) by the people willing to sell the equipment.
    - nitrogen 11y ago
      It's not just feasible, it's existed for decades and is used by security cameras that record to (usually on-premises) DVRs. There have been open source apps that do basically the same thing (detect motion, save video around events, etc.). With modern mobile GPUs you have ample processing power for running over-the-top machine learning algorithms. There is no technical need to violate privacy by default.
pfarnsworth 11y ago
As a Nest employee can you confirm that Dropcam has been essentially abandoned by Nest? In the 2+ years since the acquisition, nothing significant has been released, and the Nest App is far inferior to the Dropcam app. As well, the "customer suggestion" part of the Nest community board is a graveyard of suggestions and goodwill. It's literally a waste of time because nothing has been addressed, like simple things like access to year-over-year data instead of 10 days back which is worthless. Signed, An unhappy owner of 3 Dropcams
- dguaraglia 11y ago
  I can't really talk publicly about our roadmap (I'd love to, believe me.) I'll try to engage some of the marketing folks and point them to this thread.
  - pfarnsworth 11y ago
    Thank you. Please let them know that the "Product Suggestion" portion of the Nest Community site is an absolute insult to their customers. They should just shut it down instead of collecting years of requests and completely ignoring them.
  - banana_giraffe 11y ago
    Please do. My interaction with support has been laughably bad. I have a Dropcam that sends me a monthly warning that my account that's paid up for a year is due. I also have a Nest Protect that can no longer see my network because I have two APs broadcasting the same SSID. For the first one, customer service told me "ignore the emails". For the second one, customer service told me "permanently remove one of your access points" Both responses are enough that I no longer recommend the Nest ecosystem for anyone. They clearly don't care about support, which is troubling for a company that's in the services business.
    - dguaraglia 11y ago
      Please e-mail me on the account listed in my profile. I should be able to help with the first one (at least forward it to the right people.) I'm really sorry about the bad experience.
xg15 11y ago
According to the Nest Cam website [1], the camera (without Nest Aware) offers the following features for customers: - 24/7, continuous 1080p HD streaming - Motion alerts - Sound alerts - Night Vision - Talk & Listen - Clear Zoom - Quick, easy setup - Nest App - Software updates over Wi‑Fi With exception of the HD streaming, Talk & Listen and the app, all of those are pretty basic features which I imagine could be run on the camera hardware itself. Streaming and Talk & Listen could be implemented such that the video stream is directly routed to the app and the cloud only brokers the connection. So, I don't understand, why operating a Nest Cam needs a cloud-based service with image recognition at all. Nest Aware adds a recording service, which makes more sense in a cloud setting. But again, it's not clear to me why this is the only option and there is no way to store the stream, e.g. on a NAS instead. As far as I can see, the page doesn't mention image recognition. Nest Aware supports face detection, but as far as I know, that's a feature that middle-class smartphones and digicams can do on local hardware. I don't see why you'd need a cloud service for that. [1] https://nest.com/camera/install-and-explore/ https://nest.com/camera/install-and-explore/
- dguaraglia 11y ago
  Mea culpa: I used the term "image recognition" very haphazardly. What I meant to say was "the service provides a lot of features that are only possible through heavy analysis of the video." Have you considered the engineering effort that'd go in providing two different behaviors for the camera? Let's say we decide to support the simplest case: FTP to your local NAS. How do you format the video for it to be consumable by a normal user? Filling your hard drive with thousands of small video files is not a great experience, so that's a problem. We could buffer as much as we can in-camera and then FTP a biggish (200mb) video file. Now that introduces a delay, because you can't really watch the video until you are done buffering, so no real-timish stream. So you have to come up with a third alternative that provides every other feature the online camera has, because users will complain if their offline experience is different. Then again, your local NAS might be stolen by the same burglars that broke into your house, and the whole exercise was a waste of everyone's time. Multiply that for each other possible combination ("I want Dropbox support!", "I want my files to be in an AVI container instead of MP4!", "I want the camera to have a SIM slot so it can send me SMS messages when there's a notification!") and you have a recipe for disaster. You get featuritis, lots of technical effort, and no actual financial incentive to do any of this work. Unless we sell a $700 camera, and nobody wants that. As a small company, Dropcam had to draw the line somewhere. We didn't want to be the product that provided a solution for every requirement that came our way (and believe me, we've seen all of these before), but rather a good product for people that were happy with: 1) a product that worked as advertised (one day, a group of us were trying to setup a competitor's product: it took 4 engineers over half an hour before we gave up. This was supposed to be a "5 minute setup") 2) a product that evolved over time: running features in the cloud allows you to give new value to device owners without requiring a new device 3) trusting the company to have strict internal processes to avoid unintentional or malicious access to anyone's video (this was not treated lightly: a single case of anyone snooping on someone's stream could completely ruin the trust the users put on us.)
Animats 11y ago
Recognition needs to be local (possibly in a box that plugs into a wall outlet and has one of the better ARM chips). Recording can be remote, but should be encrypted, with the key unknown to the cloud service provider. You set the encryption key by having your phone generate it, and your phone displays a QR code, which is shown to the camera once. You now have a shared private key, and can view video from your phone. You can only view video from your phone, or from other phones which have been paired in this way.

I'm curious. How else would you implement a cloud-based recording service with image recognition? (EDIT: full disclosure, I work for Nest through the Dropcam acquisition)

afandian 11y ago

That's kind of like saying "how would you make a car without wheels?". It could be that inherent in the idea of making object X are privacy concerns. The fact that the privacy concerns are intrinsic to the object just means that you have to call the whole object X into question when discussing them. Your question seems to suggest a sentiment like "there isn't any other way to do this, can you think of one?" but there is, and it's not to do it at all. (I'm not taking a position on the Nest device, but this comment just seemed a little like a cognitive bias similar to anchoring)

dguaraglia 11y ago
Exactly! And I'm OK with people taking the stance "if it's not 100% private and my video needs to be sent over the internet, then I don't want the camera". You can either have a camera that keeps a safe copy of your video in the cloud and detects unexpected activity happening in front of your camera, or you can have one that is 100% private and doesn't upload video to the cloud. You can't have both. At least not without having a huge setup in a secured room inside your home. I understand the privacy concerns, I just argue that you can't have the ideal service without getting over them. At least not with current technology, and probably not without the right economic incentives to build a stand-alone system.
- bloomingfractal 11y ago
  What about doing all processing locally and storing only end-to-end encrypted data in servers?
  - dguaraglia 11y ago
    Not really an option right now. The current generation of cameras out there are basically a Raspberry-Pi level computer with a better camera and a hardware h264 encoder. The moment you start doing something fancy, like running any non-trivial motion-detection algorithm, you are bound to run into performance or thermal (read: overheating) issues. Let's not even talk about machine learning. Just think how much money Nest would save in server time with such a setup :)
    - bloomingfractal 11y ago
      I see, Thanks for the insightful answer! Regarding machine learning it would be nice to be able do the training in your desktop/laptop when it's idle or something like that. But it's very good to know this is a technological issue (as opposed to a business issue). Well, hope you smart folks solve this. Meanwhile, I'll keep tinkering with my raspberry pi and raspberry pi camera :)
    - lazaroclapp 11y ago
      My intuition says we are not anywhere near there yet, but do you know if any video processing algorithms exist that can reasonably be executed on encrypted data? Basically, I know fully-homomorphic encryption is ridiculously inefficient in the general case. At the same time, I know of specialized homomorphic encryption algorithms that can operate on encrypted data of specific formats. There are efficient-ish algorithms for encrypted (social-network-type) graphs, and encrypted vote ballots. I was wondering if you or anyone in your team has come across any work on privacy-preserving encrypted audio/image/video processing? I assume this is a very hard problem, but I imagine someone has tried looking into it.
      - dguaraglia 11y ago
        I'm not really the person to answer that (I'm a lowly software engineer keeping the cogs greased m'lord!) I know at some point Larry (https://github.com/lwneal https://github.com/lwneal) was looking into that, at least cursorily. I'll refer to him as the authority on anything encryption-related at Dropcam (or anything, in general. Brilliant guy!)

bhauer 11y ago

> a cloud-based recording service Well, there's the rub, right? I think what those of us who consider themselves self-hosting partisans would say is that we'd prefer a device that allows us to send its signals to a server we own and operate. The recording and image recognition would then occur on that server. In my ideal world, Dropcam (and later Nest) would have provided software I could install on my own server (located either on my home network or at a data center). I know that sounds like a far-fetched dream, but that's the ideal I want, and I will pay twice as much or more for devices that recognize the emergent demand for self-management. I have a Dropcam, and I enjoyed using it for a while. But I stopped using my Dropcam about a year ago because I grew increasingly unhappy with the idea of its video stream being sent to an untrusted third-party (Dropcam and later Google/Nest). Since there is no "self-service" mode for Dropcams, it has become a decoration in my house.

dguaraglia 11y ago
I don't know why the comment is being downvoted. There is a bunch of people that want what you want, it's just that, as you are saying, is a bit of a pipe dream right now. The way technology is progressing, it might be feasible in a few years.
- arthur_pryor 11y ago
  why is it a technology issue? not that there's never gray area, but this looks like more of a business issue than a technology issue. what's desired there (sending video to a server instance the user controls, instead of a third party vendor's servers) seems entirely feasible with current technology, though not necessarily profitable (or perceived as profitable) by the people willing to sell the equipment.
  - nitrogen 11y ago
    It's not just feasible, it's existed for decades and is used by security cameras that record to (usually on-premises) DVRs. There have been open source apps that do basically the same thing (detect motion, save video around events, etc.). With modern mobile GPUs you have ample processing power for running over-the-top machine learning algorithms. There is no technical need to violate privacy by default.

pfarnsworth 11y ago

As a Nest employee can you confirm that Dropcam has been essentially abandoned by Nest? In the 2+ years since the acquisition, nothing significant has been released, and the Nest App is far inferior to the Dropcam app. As well, the "customer suggestion" part of the Nest community board is a graveyard of suggestions and goodwill. It's literally a waste of time because nothing has been addressed, like simple things like access to year-over-year data instead of 10 days back which is worthless. Signed, An unhappy owner of 3 Dropcams

dguaraglia 11y ago
I can't really talk publicly about our roadmap (I'd love to, believe me.) I'll try to engage some of the marketing folks and point them to this thread.
- pfarnsworth 11y ago
  Thank you. Please let them know that the "Product Suggestion" portion of the Nest Community site is an absolute insult to their customers. They should just shut it down instead of collecting years of requests and completely ignoring them.
- banana_giraffe 11y ago
  Please do. My interaction with support has been laughably bad. I have a Dropcam that sends me a monthly warning that my account that's paid up for a year is due. I also have a Nest Protect that can no longer see my network because I have two APs broadcasting the same SSID. For the first one, customer service told me "ignore the emails". For the second one, customer service told me "permanently remove one of your access points" Both responses are enough that I no longer recommend the Nest ecosystem for anyone. They clearly don't care about support, which is troubling for a company that's in the services business.
  - dguaraglia 11y ago
    Please e-mail me on the account listed in my profile. I should be able to help with the first one (at least forward it to the right people.) I'm really sorry about the bad experience.

xg15 11y ago

According to the Nest Cam website [1], the camera (without Nest Aware) offers the following features for customers: - 24/7, continuous 1080p HD streaming - Motion alerts - Sound alerts - Night Vision - Talk & Listen - Clear Zoom - Quick, easy setup - Nest App - Software updates over Wi‑Fi With exception of the HD streaming, Talk & Listen and the app, all of those are pretty basic features which I imagine could be run on the camera hardware itself. Streaming and Talk & Listen could be implemented such that the video stream is directly routed to the app and the cloud only brokers the connection. So, I don't understand, why operating a Nest Cam needs a cloud-based service with image recognition at all. Nest Aware adds a recording service, which makes more sense in a cloud setting. But again, it's not clear to me why this is the only option and there is no way to store the stream, e.g. on a NAS instead. As far as I can see, the page doesn't mention image recognition. Nest Aware supports face detection, but as far as I know, that's a feature that middle-class smartphones and digicams can do on local hardware. I don't see why you'd need a cloud service for that. [1] https://nest.com/camera/install-and-explore/ https://nest.com/camera/install-and-explore/

dguaraglia 11y ago
Mea culpa: I used the term "image recognition" very haphazardly. What I meant to say was "the service provides a lot of features that are only possible through heavy analysis of the video." Have you considered the engineering effort that'd go in providing two different behaviors for the camera? Let's say we decide to support the simplest case: FTP to your local NAS. How do you format the video for it to be consumable by a normal user? Filling your hard drive with thousands of small video files is not a great experience, so that's a problem. We could buffer as much as we can in-camera and then FTP a biggish (200mb) video file. Now that introduces a delay, because you can't really watch the video until you are done buffering, so no real-timish stream. So you have to come up with a third alternative that provides every other feature the online camera has, because users will complain if their offline experience is different. Then again, your local NAS might be stolen by the same burglars that broke into your house, and the whole exercise was a waste of everyone's time. Multiply that for each other possible combination ("I want Dropbox support!", "I want my files to be in an AVI container instead of MP4!", "I want the camera to have a SIM slot so it can send me SMS messages when there's a notification!") and you have a recipe for disaster. You get featuritis, lots of technical effort, and no actual financial incentive to do any of this work. Unless we sell a $700 camera, and nobody wants that. As a small company, Dropcam had to draw the line somewhere. We didn't want to be the product that provided a solution for every requirement that came our way (and believe me, we've seen all of these before), but rather a good product for people that were happy with: 1) a product that worked as advertised (one day, a group of us were trying to setup a competitor's product: it took 4 engineers over half an hour before we gave up. This was supposed to be a "5 minute setup") 2) a product that evolved over time: running features in the cloud allows you to give new value to device owners without requiring a new device 3) trusting the company to have strict internal processes to avoid unintentional or malicious access to anyone's video (this was not treated lightly: a single case of anyone snooping on someone's stream could completely ruin the trust the users put on us.)

Animats 11y ago

Recognition needs to be local (possibly in a box that plugs into a wall outlet and has one of the better ARM chips). Recording can be remote, but should be encrypted, with the key unknown to the cloud service provider. You set the encryption key by having your phone generate it, and your phone displays a QR code, which is shown to the camera once. You now have a shared private key, and can view video from your phone. You can only view video from your phone, or from other phones which have been paired in this way.