8 ms·

New Windows AI feature records everything you've done on your PC

106 points by quantisan 2y ago

snowwrestler 2y ago
I continue to believe that the essential challenge in AI is trust, not just capability. Obviously if you can give an AI system lots of data about your life, it can learn ways to automate and improve it. But do we trust AI companies with that data? Not really, not now anyway. Ultimately there may be a technical solution to this challenge, kind of like how E2E encryption addresses many trust concerns around messaging. But if companies pursue capability without addressing trust, they might find a harder time in the marketplace than they expect.
onemoresoop 2y ago
Windows users will be Microsoft's lunch. I will warn whoever I know to avoid windows 11 at all costs. Whoever thinks this is a great idea, it's on them I guess.
bluetidepro 2y ago
I can’t imagine they launch this feature without a way to disable it. Or if they are that crazy, I imagine someone will write some script that blocks it.
- rightbyte 2y ago
  > I imagine someone will write some script that blocks it. Why? We are not in such a position that we need to put up with antagonistic OS vendors. Workarounds are just making it worse by keeping technologists on Windows.
lemondice 2y ago
Why is anyone surprised? Everything you do recorded and you have no way to see or delete the data. Just like that company that ripped off the Beatles record company name has been doing for years.
- kalupa 2y ago
  oh, yes, thier microsoft album was _so good_
- instagib 2y ago
  I remember some post about video drivers recording screen captures for usage information product improvement etc. I believe it was related to the forums for Nvidia clean install to remove driver telemetry. https://www.techpowerup.com/forums/forums/nvcleanstall.91/ https://www.techpowerup.com/forums/forums/nvcleanstall.91/
dc_ist 2y ago
Anyone beginning to think that maybe owning a desktop/laptop PC doesn't make sense anymore?
- jononor 2y ago
  You can use Linux to have a desktop/laptop PC that respects you r privacy. Btw, this shit is likely to go into Android as well. Maybe iOS will stay clear for a bit longer...
afinlayson 2y ago
This is concerning and a huge opportunity for Security thieves to learn what you did before they connected to your computer. Then make it easier to have async calls when you repeat the action they want to hack. Ohh I see you bank at Bank of America, I'll create a script to reroute through my server and give me full access to your account instead.
deafpolygon 2y ago
> threaten the lives of journalists or perceived enemies of the state Whew. You had me going there. Journalists or perceived enemies of the state won't be using Microsoft products, anyways.
jononor 2y ago
It is getting more and more like the Matrix. Just that we are being harvested for our data (behavioural patterns), not the energy.
zomg 2y ago
i remember when we used to call this malware... :x
guilhas 2y ago
I quite liked the abandoned linux zeitgeist
chris-buck 2y ago
Who asked for this? What problem does this solve?
- autoexec 2y ago
  Microsoft, who is likely collecting data from this feature somehow even if the raw video/images never leave the PC. I imagine law enforcement and three letter agencies love it too since now they'll just have to be able to access someone's windows account (MS will help them) to see everything a suspect ever done using their PC. They can even just chat with the AI about what it saw. That windows user had better hope it doesn't start hallucinating and invent a story about something that sounds criminal.
- halJordan 2y ago
  People have been asking for this since computers were invented. Dont act like just asking the computer and getting the needle from the haystack hasnt been chased since Kernighan wrote the first line of grep. The anti-genAI luddite nonsense has zero place on such a prestigious pro-technology platform.
  - squigz 2y ago
    Oh dear you were doing good up until the last sentence.
    - fortyseven 2y ago
      I mean, shit, I love seeing all the various AI advancements, but even I wouldn't want this crap built into my OS. At least not unless it's something I can self-host, and have full control over. My days of letting a corporation get it's data hooks into my personal information are long gone, as long as I have any say in the matter. I don't trust them, nor how absolutely drunk they are on shoving AI into every possible hole right now.
      - klyrs 2y ago
        I'm pretty sure we warned you that capitalists were behind this. But no, you loved the shiny. I'd say to flee to Linux, but Mozilla's already on the AI train. Computers are going to shit faster than we can react. Is this what the singularity feels like?
miles 2y ago
This HN post from last month generated a lot of interest (525 points, 166 comments) and recommendations for similar tools: I made an open source Windows app to rewind and search everything on screen https://news.ycombinator.com/item?id=40105371 https://news.ycombinator.com/item?id=40105371
lispm 2y ago
what can go wrong?
bhyahoo 2y ago
This is Microsoft collecting lots of data to shove into AI training while selling the data collection to users as a feature for their benefit. Same as Google adding feature to android to warn you on live calls if they detect it's a scam call. Requires them collecting, uploading and analyzing the phone call live in process. You are the product, and so is your data.
- oriel 2y ago
  This comment is the product. Most recently I experienced it with the slack LLM announcement with a lack of real opt-out. I've long since moved to federated services and minimized my footprint on centralized for-profit services and sites as a result. Makes me wonder if there is (or should be created) a list of companies and their data offenses akin to layoffs.fyi
- chung8123 2y ago
  This is encrypted with local storage. How is this data being used to train and sell for them?
  - Eggpants 2y ago
    Except the user license agreement states Microsoft can upload/download anything, run anything, and ignore any user setting on “your” computer.
    - throwaway5959 2y ago
      Can you link me to that? I’d like to read it.
      - deleted 2y ago
        [deleted]
      - Eggpants 2y ago
        You have to navigate all the side links of this page https://www.microsoft.com/en-us/servicesagreement/ https://www.microsoft.com/en-us/servicesagreement/ and the separated privacy policy, But here’s a good starting point in Paragraph 2.b It’s been modified slightly since I last captured this snippet: [...] you grant to Microsoft a worldwide and royalty-free intellectual property license to use Your Content, for example, to make copies of, retain, transmit, reformat, display, and distribute via communication tools Your Content on the Services [...]
htrp 2y ago
basically rewind ai on mac
- ncr100 2y ago
  They are pivoting to limitless.ai, and evolving their product according to customer feedback. Creating sort of a meeting advisor for pre-meeting and post meeting quality. Action items leading in and action items coming out.
abrichr 2y ago
https://openadapt.ai https://openadapt.ai has similar functionality, except: - it's open source - it only records when you explicitly tell it to - it has multiple PII/PHI scrubbing providers built in (see https://github.com/OpenAdaptAI/OpenAdapt?tab=readme-ov-file#industry-leading-privacy-piiphi-scrubbing-via-aws-comprehend-microsoft-presidio-and-private-ai https://github.com/OpenAdaptAI/OpenAdapt?tab=readme-ov-file#...) - the reason for recording is to automate tasks in desktop apps - it's cross platform (Mac and Windows) Full disclosure: I'm the primary author. Feedback welcome!
floppiplopp 2y ago
I'll just continue to laugh at every windows user... only harder. But I guess the technically illiterate managerial caste still will throw dumb money at the hype that the desktop monopolist has a lot of investment wasted in.
- arathis 2y ago
  Yeah righto

106 points by quantisan 2y ago

I continue to believe that the essential challenge in AI is trust, not just capability. Obviously if you can give an AI system lots of data about your life, it can learn ways to automate and improve it. But do we trust AI companies with that data? Not really, not now anyway. Ultimately there may be a technical solution to this challenge, kind of like how E2E encryption addresses many trust concerns around messaging. But if companies pursue capability without addressing trust, they might find a harder time in the marketplace than they expect.

onemoresoop 2y ago

Windows users will be Microsoft's lunch. I will warn whoever I know to avoid windows 11 at all costs. Whoever thinks this is a great idea, it's on them I guess.

bluetidepro 2y ago

I can’t imagine they launch this feature without a way to disable it. Or if they are that crazy, I imagine someone will write some script that blocks it.

rightbyte 2y ago
> I imagine someone will write some script that blocks it. Why? We are not in such a position that we need to put up with antagonistic OS vendors. Workarounds are just making it worse by keeping technologists on Windows.

lemondice 2y ago

Why is anyone surprised? Everything you do recorded and you have no way to see or delete the data. Just like that company that ripped off the Beatles record company name has been doing for years.

kalupa 2y ago
oh, yes, thier microsoft album was _so good_
instagib 2y ago
I remember some post about video drivers recording screen captures for usage information product improvement etc. I believe it was related to the forums for Nvidia clean install to remove driver telemetry. https://www.techpowerup.com/forums/forums/nvcleanstall.91/ https://www.techpowerup.com/forums/forums/nvcleanstall.91/

dc_ist 2y ago

Anyone beginning to think that maybe owning a desktop/laptop PC doesn't make sense anymore?

jononor 2y ago
You can use Linux to have a desktop/laptop PC that respects you r privacy. Btw, this shit is likely to go into Android as well. Maybe iOS will stay clear for a bit longer...

afinlayson 2y ago

This is concerning and a huge opportunity for Security thieves to learn what you did before they connected to your computer. Then make it easier to have async calls when you repeat the action they want to hack. Ohh I see you bank at Bank of America, I'll create a script to reroute through my server and give me full access to your account instead.

deafpolygon 2y ago

> threaten the lives of journalists or perceived enemies of the state Whew. You had me going there. Journalists or perceived enemies of the state won't be using Microsoft products, anyways.

jononor 2y ago

It is getting more and more like the Matrix. Just that we are being harvested for our data (behavioural patterns), not the energy.

zomg 2y ago

i remember when we used to call this malware... :x

guilhas 2y ago

I quite liked the abandoned linux zeitgeist

chris-buck 2y ago

Who asked for this? What problem does this solve?

autoexec 2y ago
Microsoft, who is likely collecting data from this feature somehow even if the raw video/images never leave the PC. I imagine law enforcement and three letter agencies love it too since now they'll just have to be able to access someone's windows account (MS will help them) to see everything a suspect ever done using their PC. They can even just chat with the AI about what it saw. That windows user had better hope it doesn't start hallucinating and invent a story about something that sounds criminal.
halJordan 2y ago
People have been asking for this since computers were invented. Dont act like just asking the computer and getting the needle from the haystack hasnt been chased since Kernighan wrote the first line of grep. The anti-genAI luddite nonsense has zero place on such a prestigious pro-technology platform.
- squigz 2y ago
  Oh dear you were doing good up until the last sentence.
  - fortyseven 2y ago
    I mean, shit, I love seeing all the various AI advancements, but even I wouldn't want this crap built into my OS. At least not unless it's something I can self-host, and have full control over. My days of letting a corporation get it's data hooks into my personal information are long gone, as long as I have any say in the matter. I don't trust them, nor how absolutely drunk they are on shoving AI into every possible hole right now.
    - klyrs 2y ago
      I'm pretty sure we warned you that capitalists were behind this. But no, you loved the shiny. I'd say to flee to Linux, but Mozilla's already on the AI train. Computers are going to shit faster than we can react. Is this what the singularity feels like?

miles 2y ago

This HN post from last month generated a lot of interest (525 points, 166 comments) and recommendations for similar tools: I made an open source Windows app to rewind and search everything on screen https://news.ycombinator.com/item?id=40105371 https://news.ycombinator.com/item?id=40105371

lispm 2y ago

what can go wrong?

bhyahoo 2y ago

This is Microsoft collecting lots of data to shove into AI training while selling the data collection to users as a feature for their benefit. Same as Google adding feature to android to warn you on live calls if they detect it's a scam call. Requires them collecting, uploading and analyzing the phone call live in process. You are the product, and so is your data.

oriel 2y ago
This comment is the product. Most recently I experienced it with the slack LLM announcement with a lack of real opt-out. I've long since moved to federated services and minimized my footprint on centralized for-profit services and sites as a result. Makes me wonder if there is (or should be created) a list of companies and their data offenses akin to layoffs.fyi
chung8123 2y ago
This is encrypted with local storage. How is this data being used to train and sell for them?
- Eggpants 2y ago
  Except the user license agreement states Microsoft can upload/download anything, run anything, and ignore any user setting on “your” computer.
  - throwaway5959 2y ago
    Can you link me to that? I’d like to read it.
    - deleted 2y ago
      [deleted]
    - Eggpants 2y ago
      You have to navigate all the side links of this page https://www.microsoft.com/en-us/servicesagreement/ https://www.microsoft.com/en-us/servicesagreement/ and the separated privacy policy, But here’s a good starting point in Paragraph 2.b It’s been modified slightly since I last captured this snippet: [...] you grant to Microsoft a worldwide and royalty-free intellectual property license to use Your Content, for example, to make copies of, retain, transmit, reformat, display, and distribute via communication tools Your Content on the Services [...]

htrp 2y ago

basically rewind ai on mac

ncr100 2y ago
They are pivoting to limitless.ai, and evolving their product according to customer feedback. Creating sort of a meeting advisor for pre-meeting and post meeting quality. Action items leading in and action items coming out.

abrichr 2y ago

https://openadapt.ai https://openadapt.ai has similar functionality, except: - it's open source - it only records when you explicitly tell it to - it has multiple PII/PHI scrubbing providers built in (see https://github.com/OpenAdaptAI/OpenAdapt?tab=readme-ov-file#industry-leading-privacy-piiphi-scrubbing-via-aws-comprehend-microsoft-presidio-and-private-ai https://github.com/OpenAdaptAI/OpenAdapt?tab=readme-ov-file#...) - the reason for recording is to automate tasks in desktop apps - it's cross platform (Mac and Windows) Full disclosure: I'm the primary author. Feedback welcome!

floppiplopp 2y ago

I'll just continue to laugh at every windows user... only harder. But I guess the technically illiterate managerial caste still will throw dumb money at the hype that the desktop monopolist has a lot of investment wasted in.

arathis 2y ago
Yeah righto