4 ms·
For a dash of dystopia: Imagine a company starts using that same LLM fuzzy-matching [0] against what you intend to be normal search activity, to detect "bad" qu
by Terr_ 4mo ago
For a dash of dystopia: Imagine a company starts using that same LLM fuzzy-matching [0] against what you intend to be normal search activity, to detect "bad" queries and "bad" users who will be blocked. Maybe they'll delete your SSO/email/videos/photos too, who knows.
I can easily imagine it happening, especially after some point where they start using the same systems to "enhance" your query.
[0] To be specific, your searches will be placed into a narrative document template, where a character Mr. Safety Bot is about to speak a verdict, and then the LLM story-generator decides whether it "fits" for Mr. Safety Bot to declare you banned.
- genxy 4mo agoThere is prior art with Tuttle vs Buttle.
- Terr_ 4mo agoAhh, but with Naruto v. Slater, animals can't create or hold copyright, so the art posthumously created by the fly would be in public domain. :p
- sillysaurusx 4mo agoThat is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.
- SXX 4mo agoIt has nothing to do with human decisions. Bans always come later because it's a best way to make sure most people who get banned would never know what are they banned for exactly and what threshold there is, The same tactics used in game development against cheaters. If it would ban you right after prompt you'll know how to avoid getting banned. Obviously that didn't worked for you because you wasn't doing multiple attempts to bypass filters like if you were jailbreaking it by repeatedly trying different stuff.
- hobo123 4mo agoLike rat poison. If it works too fast, the rats learn to avoid it.
- charcircuit 4mo agoExcept in the case of video games it in practice means that cheaters get to terrorize your playerbase without being banned. This tactic is the kind of decision that is made by people staring at metrics all day without considering what's going on in reality.
- DaSHacka 4mo agoIt's pretty effective actually, this is why they do ban waves. One of my friends in high school used to cheat on a popular video game. The fact ban waves would occur about once a week to once a month meant whenever his accounts got banned, he never knew why exactly and wasn't able to stop it the next time. Of course, if ban waves are too long apart then yeah you're just letting a known cheater wreak havoc on the playerbase.
- charcircuit 4mo agoBut the price of a new account is low. So worst case cheaters can cheat on the playerbase for a couple weeks get banned and then switch to another account and continue cheating for another couple weeks.
- DaSHacka 4mo agoThat's the problem with many older games, I don't know of a good solution other than either age-gating accounts or keeping the price of the game ~$60
- butlike 4mo agoBut isn't the whole point to avoid getting banned? Like the system doesn't want to ban you, and you don't want to get banned, so what's the issue with knowing exactly how you get banned?
- userbinator 4mo agoOrwell's idea of "wrongthink" is more relevant than ever.
- Terr_ 4mo agoMuch like with 1984, I think the promising/terrifying part is that it's not an issue with technology per se, but about a society that has somehow decided to allow Terrible Things to happen. (Often via too much power in too few hands with no accountability or alternatives.) For example, imagine that there are 20 great search-engines around the world (who don't collude), and it hits rather differently.
- Cthulhu_ 4mo agoI think (armchair analysis) it's a combination of two things, one is that it's by choice - people enjoy using LLMs, they find it valuable and convenient. As for social media, companies and society itself successfully managed to gamify people sharing their personal lives on the internet, to voluntarily reduce their own privacy. Also because they still believe they have a choice in the matter, by e.g. not posting or sharing things. The other thing is the boiling frog analogy, it wasn't a sudden "we're at war now so you get a camera in your house" moment (iirc 1984 skips over the transition to an authoritative state though), it's a slow, gradual progress. People got used to taking selfies, then applying a filter, then using facial scans for identification, then a cool app that puts your face on movie scenes and now the company behind faceapp and co has detailed facial scans of millions of people. Europe tried to limit it via legislation, but that's a lot of after-the-fact policing and that's just Europe.
- brookst 4mo agoOrwell imagined it 80 years ago, not sure it’s more relevant just because someone else imagines it today.
- pona-a 4mo agoOrwell also imagined a future where Speakwrites made everyone unable to write by hand, making their thoughts dull and tanged as a long unprepared monologue tends to be. I don't think he's anywhere on your side.
- Tenoke 4mo agoIf you are imagining that, you could imagine it with search doing the same 10 years ago, which would have more thoroughly prevented you from researching things.
- jval43 4mo agoThis is one of the reasons I currently use Gemini for daily use and research. Google has lots of experience with search history, and presumably handles this better than new companies.
- throwuxiytayq 4mo agoThis is also the main reason I currently use Gemini. I hate my gmail account, my Android phone is annoying, and I spend too much time on Youtube, so hopefully they take my access away to all of these soon.
- KaiserPro 4mo agowhat you mean like searching for porn? emailing client lists to your personal email?