5 ms·
What LLMs Know About Their Users
- careful_ai 1y agoPlainspoken and unnerving. Schneier nails it: LLMs become mirrors that reflect, amplify, and exploit our own signals. What resonated with me: every prompt, every correction, every hesitation feeds a profile the model refines over time. It’s not just personalization—it’s psychoanalysis-by-proxy. This matter needs more than opt-out buttons. We need transparency and circuit breakers. Give me AI that explains why it suggests a certain answer—not just adapts forever without oversight.
- wcarss 1y agoThe link should probably be: https://www.schneier.com/blog/archives/2025/06/what-llms-know-about-their-users.html https://www.schneier.com/blog/archives/2025/06/what-llms-kno...
- tveita 1y agoOr the post it quotes https://simonwillison.net/2025/May/21/chatgpt-new-memory/ https://simonwillison.net/2025/May/21/chatgpt-new-memory/
- jazzyjackson 1y ago"6": "User is active 2 days in the last 1 day, 8 days in the last 7 days, and 11 days in the last 30 days." lol
- vavooom 1y agoI applied this query with 4o and yes, quite a thorough historical recounting. Really weaves together all those "random" questions one asks an LLM into a surprisingly (and somewhat scary) encompassing of one's self.
- john61 1y agoI just asked Mistral.ai, using it without an account: what do you know about me? 2:50pm I don't know anything about you unless you share information with me. How can I assist you today? 2:50pm you do not know my IP and its location? 2:50pm That's correct, I don't have access to your IP address or your location. If you need assistance with something specific, feel free to let me know!
- BoiledCabbage 1y agoNot to complain, but that test would be more interesting if you ran it with an account.
- amelius 1y agoOf course that doesn't prove much. Perhaps the information is only available to administrators.
- noident 1y agoYou might as well have asked the pen on your desk how much it knows about you. Whether your personal data is in the current LLM context window or not has nothing to do with whether it was collected from you by Mistral.
- wongarsu 1y agoRather than ask it if it knows anything about you, ask it where the nearest McDonalds is Mistral seems honest in its assessment (as should be expected from a French company operating under GDPR). But e.g. ChatGPT will deny knowing your location, but still show a McDonalds that suggests it knows your IP geolocation
- john61 1y agowhere is the nearest McDonald's? 18:51 I'm sorry, but I don't have the capability to access real-time location data to find the nearest McDonald's for you. However, you can easily find this information by using a maps application on your phone or computer.
- bgwalter 1y ago> User’s account is 237 weeks old. ChatGPT was released to the public on November 30, 2022. Otherwise, this reads like an FBI file: https://digitalcommons.unl.edu/usjusticematls/10/ https://digitalcommons.unl.edu/usjusticematls/10/
- simonw 1y agoI created an account with OpenAI back in 2020 to try out GPT-3, which would explain that account age. (I just confirmed that by searching my email for anything mentioning OpenAI in 2020 and found an invite to create an account in October 2020.)
- jruohonen 1y agoSame old but there has been surprisingly little attention paid to it in the present context.
- nonelog 1y agoThe solution is to use www.duck.ai. Full privacy, no data collection, no User profiling, no surveillance, no training with your data. Plus, you can chose your preferred AI model: Claude, GPT, etc. And it's even free of charge.
- hardlianotion 1y agoI still don't know what integrous means.
- sydney6 1y ago"Integrous" is a rarely used adjective meaning "having or characterized by integrity". It's derived from the Latin word "integer," meaning whole or complete, and the suffix "-ous" indicating fullness or possession. While technically correct, it's more common to use synonyms like "honest," "decent," or "virtuous" to convey the concept of integrity. Now, take a guess where this comes from..
- simianwords 1y agoLLM's knowing things about us is genuinely useful - what's the alternative here?
- Diti 1y agoThe alternative is that LLMs don’t know anything about us, of course. That’s what Kagi’s Assistants do.
- GMoromisato 1y agoI don’t care what chatGPT knows about me. But I do care what OpenAI knows about me and how it will use that knowledge. The alternative is transparency and verification of who has that data.
- john61 1y agoThat the LLM is in control of us.
- xarope 1y agofor a start, the ability to partition that knowledge? e.g. what is being researched or worked on for project A, might be vastly different or require different domain of knowledge for project B. Or worse still, you ended up reusing code across different projects for different clients, and one day client O(#1) decides to sue client G(#2) and finds code similarities? [1] as example, perhaps client O has a name that rhymes with "Horrible" [2] and potentially client G's name rhymes with "Flugel"
- BurningFrog 1y agoFWIW, Grok answers this when asked "what do you know about me?" (I do have an account): I don’t have any personal information about you, as I don’t have access to user data unless it’s explicitly shared in the conversation. If you’ve posted publicly on X, I could potentially look up your profile or posts to learn more, but I’d need your X handle to do that. Alternatively, if you want to share something about yourself—like your interests, location, or what you’re curious about—I can tailor my responses based on that. What would you like me to know or explore about you?
- msgodel 1y agoI know someone else who uses Grok and has the "learn from my responses" feature turned on, he claims it generates memories (like chatgpt did for a while) although he might just not understand how the dialog engine works.
- EGreg 1y agoWhen I speak to ChatGPT and other LLMs, I am mindful of the profile they build on me and what I ask about. And even how I ask it. It is the same as I have been doing for 15 years on public social media.
- neom 1y agoI really dislike the new memory, find myself using temp chats on ChatGPT a lot these days, they claim it adds nothing to mine or the collective knowledge, although to be honest at this point my trust in OpenAI is incredibly low.
- deleted 1y ago[deleted]
- nullc 1y agoPeople's privacy expactations are miscalculated in a world with LLMs. Before LLMs there was little reason to care if you were leaving breadcrumbs, because no one with access was going to investigate them and draw conclusions from them. The fact that the LLMs are prone to hallucinations and fanciful conclusion jumping probably make things worse, because the only way to be safe from it wrongfully concluding things about you is to just be invisible to it.
- rcruzeiro 1y agoIsn’t this just outputting the same information that you can find on my “administre memories” section of ChatGPT settings?
- simonw 1y agoNo. That's the old memory feature, where you could view and delete individual memories. This is the new memory feature where it automatically summarizes notes from your previous conversations. I wrote more about the difference between the two here: https://simonwillison.net/2025/May/21/chatgpt-new-memory/#we-re-losing-control-of-the-context https://simonwillison.net/2025/May/21/chatgpt-new-memory/#we...
- marcusstorm 1y agoThis is like the prompts which ask the LLM to act as a spy / FBI agent, to expose what it knows about the user, right?
- b0a04gl 1y ago[dead]
- drillsteps5 1y agoAll: There's no valid use cases for local LLM, unless you're doing something illegal or unethical, you creepy pervert and/or criminal Also all: How come LLM (and the company that runs it and provides a way for me to access and use it) knows so much about me? That's creepy. And there's just no way anybody could predict that would happen