5 ms·

Giving Windows total recall of everything a user does is a privacy minefield

50 points by srameshc 2y ago

DocChi77 2y ago
Honestly, I like the idea behind the feature. Sure, I could (and probably should!) be more proactive with my bookmarks, or otherwise saving content I find interesting. But the fact is, I don't. If something like this (with negligible enough compute/storage impacts) could be implemented locally, I would probably buy into it. I'm sure it would be "better" with a larger model, but as a jumping-off point for finding something I saw or read about prior, I think a locally available model would suffice. Trusting Microsoft (or Apple, OpenAI, etc.) with this information for a "better" AI model is not a trade-off I'd personally find acceptable.
- wilsonnb3 2y ago
  > If something like this (with negligible enough compute/storage impacts) could be implemented locally, I would probably buy into it It is implemented locally
  - kingforaday 2y ago
    > It is implemented locally I am afraid nothing Microsoft works on is local anymore.
    - pitaj 2y ago
      Not trusting Microsoft when they something runs locally probably means you shouldn't be using Windows, period.
  - ekianjo 2y ago
    oh like the windows login?
    - Terr_ 2y ago
      No, like browser choice :p https://www.theverge.com/23935029/microsoft-edge-forced-windows-10-google-chrome-fight https://www.theverge.com/23935029/microsoft-edge-forced-wind...
- Terr_ 2y ago
  I think it's ultimately about autonomy and control, and data storage is only one part of that. If it was open source software that I could fully administer that only changed when I wanted it to change, things would be very different.
labrador 2y ago
I said this yesterday but I'll say it again in case Microsoft is listening: recall could come in handy as long as it has a "start record" and "stop record" button with an indicator like a red light when it is recording. Otherwise I won't use it.
- ekianjo 2y ago
  will you even have a choice to turn it off? they always make it opt out at first until updates silently turn things on again
  - labrador 2y ago
    I'm suggesting they treat it like a screen recorder, which it is
  - m463 2y ago
    happened to me once with icloud something or other. All of a sudden ALL my photos started uploading.
- runsonrum 2y ago
  A red light not turning on will only give you the impression that nothing is being recorded.
  - labrador 2y ago
    Microsoft could continuously reading all my private files but a) I doubt they'd do that b) I'm not very interesting c) It'd be a massive lawsuit if they were d) privacy hackers have motivation to find out if they are - see c
    - b112 2y ago
      I wish people wouldn't use/state b, ever. It doesn't add to the conversation, and further, people you know could be interesting. And interesting in ways, that you don't know about.
SavageBeast 2y ago
Some days I think the Microsoft personal computing team is just a division of the Apple marketing department where their main charge is to help sell more Macs. About 1x/year I get frustrated with the depth of my lock-in to the Apple Ecosystem and start looking for an alternative (thats not a science project). After I go through the Windows offerings and do some reading I'm generally happy to go back to dealing with whatever bullshit Apple want's to cram down my throat this release. Now sure iCloud knows the depths of my soul but at least I voluntarily put things there. Thats a far cry from some AI screen capping me in the background and running some AI over it (to show me more relevant ads in the Start Menu).
- angulardragon03 2y ago
  I use a Mac every day, for work and personal stuff. I have a Windows desktop PC, that I use almost exclusively for playing PC games with friends. I already had to use some obscure method to bypass signing in with my Microsoft account during installation. I boot it anywhere between once a month and a few times a week, and about 50% of the time some random configuration changes and I can’t do the one thing that I set out to do (play a game) without updating or downgrading some arbitrary driver. Oh, and it nags me almost every boot to sign in with my Microsoft account for one reason or another. And this is an operating system with an MSRP of $200! There are loads of things to critique about my Mac, but I can use it daily just fine, without the OS trying to trick me into signing into some service or other. The daily experience of using it feels much more consensual, as you describe.
  - weikju 2y ago
    hah we need to come up with a campaign - "Consensual Computing".
Grimeton 2y ago
The first issue I have with this feature is: It's opt-out and not opt-in. The second problem I have is that they tell me that neither the snapshots are send to Microsoft nor are they analyzed on Microsoft's servers, but I do NOT know what happens with the data once stuff got analyzed on an operating system that is phoning home all kinds of information all the time w/o me being able to tell what it is. Not even the enterprise version can disable all the "phone home features" there are and Microsoft doesn't make everything public that they transfer. The third issue is that stuff is stored on my local drive with bitlocker encryption that can be easily circumvented even if they store the keys in a TPM. There is this universal, not well known, only documented in a single file, backdoor to Windows [1] that allows you to gain full control over the system even with secure boot enabled. The possibilities of your boss spying on you will become endless. Not that they aren't already, but this opens a whole new can of worms. [1] - https://download.microsoft.com/download/8/a/2/8a2fb72d-9b96-4e2d-a559-4a27cf905a80/windows-platform-binary-table.docx https://download.microsoft.com/download/8/a/2/8a2fb72d-9b96-...
- SoftTalker 2y ago
  Sounds like Windows should only be used on an air-gapped network.
  - Grimeton 2y ago
    Yeah. This [1] is also a nice read. Even if the page is in dutch, the documents are available as PDFs in english. [1] - https://www.rijksoverheid.nl/documenten/rapporten/2019/06/11/data-protection-impact-assessment-windows-10-enterprise https://www.rijksoverheid.nl/documenten/rapporten/2019/06/11...
- Eisenstein 2y ago
  What is in that docx file? What is this backdoor that will crack a personal Bitlocker drive encrypted with a PIN? There is (another) 'backdoor' that I know about which is only a backdoor because people don't generally know it exists, but if you use your Windows account to log in with Windows and decrypt the Bitlocker drive, it stores your key in the cloud so that you can unlock it by using the MS account as a 'password reset' feature. I am sure this exists because people don't understand that the question 'I encrypted my drive and forget the password, how do I fix it' is 'you can't', unless you make the encryption worthless by having the key somewhere other people can access. But MS really doesn't care because it also allows businesses to unlock their employee's machines, and a business's concerns are the only concerns MS cares about, and only partially and only sometimes.
joebob42 2y ago
Presumably the real feature here is for admins to query the machine and ask it "how many hours of work did joebob actually do this week", and "has joebob complied with the will of his manager at all times this week".
- m463 2y ago
  You are thinking too tamely. "senior citizen, searching dementia symptoms, wants to invest life savings" "verizon employee, money trouble, access to sim reprogramming application" "young woman, height proportional to weight, poor decision making skills"
greatgib 2y ago
And wait a little bit for "Recall" in windows for corporate. Providing all the power of this recording to watch employees. If anyone thinks that it is too extreme to be serious, you have to know that Microsoft offered (and is maybe still offering) to company creepy spy stats other employee activities through Office 365, like how long is spent on emails, active on the computer, chatting with other employees,...
deleted 2y ago
[deleted]
walterbell 2y ago
Related threads: "AI PCs are the final nail in the coffin of open computing", 52 comments, https://news.ycombinator.com/item?id=40436975 https://news.ycombinator.com/item?id=40436975 "How the new Microsoft Recall feature fundamentally undermines Windows security", 48 comments, https://news.ycombinator.com/item?id=40433884 https://news.ycombinator.com/item?id=40433884 "Windows Recall sounds like a privacy nightmare", 293 comments, https://news.ycombinator.com/item?id=40443682 https://news.ycombinator.com/item?id=40443682
upon_drumhead 2y ago
This is going to be such a benefit to law enforcement. Being able to just scan the history of the machine to see if a suspect had a illegal file and then deleted it, or what they were doing before they wiped their browser history is going to cause a ton more successful court cases. That said, it makes going to random websites from comment chains or user submitted stories much more dangerous. If they lead to illegal content, it's now stored ready to be used against you if law enforcement ever wants to take a close look at you.
renegat0x0 2y ago
If AI gets really good we may be on trouble. If such features become a necessity, to boost productivity we may realy forget about privacy. We may be forced to choose between productivity and privacy, and I suspect what many of us will choose
wbshaw 2y ago
Calling it a minefield doesn't go far enough. Atrocity seems closer. Even if they're promising protections now, how long do you think it will take before they realize that profits are greater than privacy for this feature too?

50 points by srameshc 2y ago

Honestly, I like the idea behind the feature. Sure, I could (and probably should!) be more proactive with my bookmarks, or otherwise saving content I find interesting. But the fact is, I don't. If something like this (with negligible enough compute/storage impacts) could be implemented locally, I would probably buy into it. I'm sure it would be "better" with a larger model, but as a jumping-off point for finding something I saw or read about prior, I think a locally available model would suffice. Trusting Microsoft (or Apple, OpenAI, etc.) with this information for a "better" AI model is not a trade-off I'd personally find acceptable.

wilsonnb3 2y ago
> If something like this (with negligible enough compute/storage impacts) could be implemented locally, I would probably buy into it It is implemented locally
- kingforaday 2y ago
  > It is implemented locally I am afraid nothing Microsoft works on is local anymore.
  - pitaj 2y ago
    Not trusting Microsoft when they something runs locally probably means you shouldn't be using Windows, period.
- ekianjo 2y ago
  oh like the windows login?
  - Terr_ 2y ago
    No, like browser choice :p https://www.theverge.com/23935029/microsoft-edge-forced-windows-10-google-chrome-fight https://www.theverge.com/23935029/microsoft-edge-forced-wind...
Terr_ 2y ago
I think it's ultimately about autonomy and control, and data storage is only one part of that. If it was open source software that I could fully administer that only changed when I wanted it to change, things would be very different.

labrador 2y ago

I said this yesterday but I'll say it again in case Microsoft is listening: recall could come in handy as long as it has a "start record" and "stop record" button with an indicator like a red light when it is recording. Otherwise I won't use it.

ekianjo 2y ago
will you even have a choice to turn it off? they always make it opt out at first until updates silently turn things on again
- labrador 2y ago
  I'm suggesting they treat it like a screen recorder, which it is
- m463 2y ago
  happened to me once with icloud something or other. All of a sudden ALL my photos started uploading.
runsonrum 2y ago
A red light not turning on will only give you the impression that nothing is being recorded.
- labrador 2y ago
  Microsoft could continuously reading all my private files but a) I doubt they'd do that b) I'm not very interesting c) It'd be a massive lawsuit if they were d) privacy hackers have motivation to find out if they are - see c
  - b112 2y ago
    I wish people wouldn't use/state b, ever. It doesn't add to the conversation, and further, people you know could be interesting. And interesting in ways, that you don't know about.

SavageBeast 2y ago

Some days I think the Microsoft personal computing team is just a division of the Apple marketing department where their main charge is to help sell more Macs. About 1x/year I get frustrated with the depth of my lock-in to the Apple Ecosystem and start looking for an alternative (thats not a science project). After I go through the Windows offerings and do some reading I'm generally happy to go back to dealing with whatever bullshit Apple want's to cram down my throat this release. Now sure iCloud knows the depths of my soul but at least I voluntarily put things there. Thats a far cry from some AI screen capping me in the background and running some AI over it (to show me more relevant ads in the Start Menu).

angulardragon03 2y ago
I use a Mac every day, for work and personal stuff. I have a Windows desktop PC, that I use almost exclusively for playing PC games with friends. I already had to use some obscure method to bypass signing in with my Microsoft account during installation. I boot it anywhere between once a month and a few times a week, and about 50% of the time some random configuration changes and I can’t do the one thing that I set out to do (play a game) without updating or downgrading some arbitrary driver. Oh, and it nags me almost every boot to sign in with my Microsoft account for one reason or another. And this is an operating system with an MSRP of $200! There are loads of things to critique about my Mac, but I can use it daily just fine, without the OS trying to trick me into signing into some service or other. The daily experience of using it feels much more consensual, as you describe.
- weikju 2y ago
  hah we need to come up with a campaign - "Consensual Computing".

Grimeton 2y ago

The first issue I have with this feature is: It's opt-out and not opt-in. The second problem I have is that they tell me that neither the snapshots are send to Microsoft nor are they analyzed on Microsoft's servers, but I do NOT know what happens with the data once stuff got analyzed on an operating system that is phoning home all kinds of information all the time w/o me being able to tell what it is. Not even the enterprise version can disable all the "phone home features" there are and Microsoft doesn't make everything public that they transfer. The third issue is that stuff is stored on my local drive with bitlocker encryption that can be easily circumvented even if they store the keys in a TPM. There is this universal, not well known, only documented in a single file, backdoor to Windows [1] that allows you to gain full control over the system even with secure boot enabled. The possibilities of your boss spying on you will become endless. Not that they aren't already, but this opens a whole new can of worms. [1] - https://download.microsoft.com/download/8/a/2/8a2fb72d-9b96-4e2d-a559-4a27cf905a80/windows-platform-binary-table.docx https://download.microsoft.com/download/8/a/2/8a2fb72d-9b96-...

SoftTalker 2y ago
Sounds like Windows should only be used on an air-gapped network.
- Grimeton 2y ago
  Yeah. This [1] is also a nice read. Even if the page is in dutch, the documents are available as PDFs in english. [1] - https://www.rijksoverheid.nl/documenten/rapporten/2019/06/11/data-protection-impact-assessment-windows-10-enterprise https://www.rijksoverheid.nl/documenten/rapporten/2019/06/11...
Eisenstein 2y ago
What is in that docx file? What is this backdoor that will crack a personal Bitlocker drive encrypted with a PIN? There is (another) 'backdoor' that I know about which is only a backdoor because people don't generally know it exists, but if you use your Windows account to log in with Windows and decrypt the Bitlocker drive, it stores your key in the cloud so that you can unlock it by using the MS account as a 'password reset' feature. I am sure this exists because people don't understand that the question 'I encrypted my drive and forget the password, how do I fix it' is 'you can't', unless you make the encryption worthless by having the key somewhere other people can access. But MS really doesn't care because it also allows businesses to unlock their employee's machines, and a business's concerns are the only concerns MS cares about, and only partially and only sometimes.

joebob42 2y ago

Presumably the real feature here is for admins to query the machine and ask it "how many hours of work did joebob actually do this week", and "has joebob complied with the will of his manager at all times this week".

m463 2y ago
You are thinking too tamely. "senior citizen, searching dementia symptoms, wants to invest life savings" "verizon employee, money trouble, access to sim reprogramming application" "young woman, height proportional to weight, poor decision making skills"

greatgib 2y ago

And wait a little bit for "Recall" in windows for corporate. Providing all the power of this recording to watch employees. If anyone thinks that it is too extreme to be serious, you have to know that Microsoft offered (and is maybe still offering) to company creepy spy stats other employee activities through Office 365, like how long is spent on emails, active on the computer, chatting with other employees,...

deleted 2y ago

[deleted]

walterbell 2y ago

Related threads: "AI PCs are the final nail in the coffin of open computing", 52 comments, https://news.ycombinator.com/item?id=40436975 https://news.ycombinator.com/item?id=40436975 "How the new Microsoft Recall feature fundamentally undermines Windows security", 48 comments, https://news.ycombinator.com/item?id=40433884 https://news.ycombinator.com/item?id=40433884 "Windows Recall sounds like a privacy nightmare", 293 comments, https://news.ycombinator.com/item?id=40443682 https://news.ycombinator.com/item?id=40443682

upon_drumhead 2y ago

This is going to be such a benefit to law enforcement. Being able to just scan the history of the machine to see if a suspect had a illegal file and then deleted it, or what they were doing before they wiped their browser history is going to cause a ton more successful court cases. That said, it makes going to random websites from comment chains or user submitted stories much more dangerous. If they lead to illegal content, it's now stored ready to be used against you if law enforcement ever wants to take a close look at you.

renegat0x0 2y ago

If AI gets really good we may be on trouble. If such features become a necessity, to boost productivity we may realy forget about privacy. We may be forced to choose between productivity and privacy, and I suspect what many of us will choose

wbshaw 2y ago

Calling it a minefield doesn't go far enough. Atrocity seems closer. Even if they're promising protections now, how long do you think it will take before they realize that profits are greater than privacy for this feature too?