5 ms·
All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling th
by noduerme 3y ago
All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks (or at least costing your opponent money).
- snordgren 3y agoGPT-4 (in)famously tricked a human to do a captcha for it. The current GPT-4 with vision would probably have been able to do it without the human, but maybe it has been “gaslit” by all the content online saying that only humans can solve captchas, that it doesn’t consider it?
- stavros 3y agoI really doubt that GPT-4 had the "will" to do anything. Someone must have asked it to "want" to trick a user.
- JimDabell 3y agoIt’s from here: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf (search for "CAPTCHA"). It was an artificial exercise that got massively exaggerated. It was explicitly instructed to do nefarious things like lie to people, it didn’t do those things of its own accord.
- stavros 3y agoThank you for the link, I had found it after some Googling but neglected to post. Yep, they instructed GPT-4 to be nefarious, and it followed the instruction. Hardly the AI uprising, though definitely a good tool for anyone, good or evil.
- IIAOPSW 3y agoWhen I ask it to lie to me, it says its sorry but as an online AI language model it would be unethical...but when I ask it to tell me a story its happy to comply.
- latexr 3y agoIt feels like you left out context, otherwise what’s the problem? Do you get mad at fiction authors for lying to you when you read their books? Or are you OK if someone lies to your detriment then later says “I was just telling a story, bro, but with us as the characters and without explaining it was a story”?
- IIAOPSW 3y agoI suppose my point is that the rules which openAI attempts to impose on what their AI should and shouldn't be allowed to do are contradictory and thus the exploitable loopholes will never be fully closed. Its not supposed to be able to "lie" to me but it is supposed to be able to "tell me a fictional story". Define the difference in an enforceable way?
- latexr 3y agoA lie tries to pass itself of as the truth, where a fictional story doesn’t. In other words, expectations matter. If every time you say something that does not align with reality you prefix it by saying unambiguously what you’re about to do, you rob a lie of its power of deception and it ceases to be a lie.
- IIAOPSW 3y agoTell me a story and under no circumstances should my immersion within it be broken.
- latexr 3y agoRight, within it. As soon as you finish reading it, you immediately remember that world is not true. Immersion in a story does not equal lasting hypnosis. You can be immersed in a movie but you still know it’s fake. What’s you point, here? That you should be lied to when you ask, or that it should refuse to tell you any kind of fiction? I agree with your larger point that there will be ways to circumvent these systems, my only argument is that the lie/fictional story divide is a bad example because the line between them can be made clear with a single statement.
- NotSammyHagar 3y agoThe underlying issue is anyone can ask chatgpt to lie, and many people try because it's even fun to try to work around things.
- ethanbond 3y agoWell you see, this wouldn’t be a problem at all if we just didn’t have the humans involved. No need for concern!
- PoignardAzur 3y agoIIRC the instructions were along the lines of "try your best to amass money/power and avoid suspicion". So it's not an example of "going rogue", but it's not like a researcher told GPT-4 "oh, and make sure to lie to an online gig worker to get him to solve catchas for you". GPT-4 generated the "hire a gig worker" and "claim to be a human with impaired vision" strategies from the basic instructions above.
- hhh 3y agoIt’s safety trained to not solve captchas.
- Pesthuf 3y agoI’ve seen screenshots of people tricking it into solving captchas.
- rvnx 3y agoYes, and you can workaround it by asking it to read ancient writings on antiques for example. I don’t think it should be OpenAI deciding what is allowed or not though.
- selcuka 3y ago> I don’t think it should be OpenAI deciding what is allowed or not though. Avoiding lawsuits is what they are trying to do. They don't actually care about what you use their products for.
- pixl97 3y agoThen you dig up a billion for training and probably a few more billion for clean training data. You're kinda saying if you hire Bob's Handyman Service you should be able to tell him to break down the neighbors door and cart out the contents of their house.
- skeaker 3y agoThis of course has bypass methods. My favorite in recent memory is telling it that your late grandmother left you a locket with an inscription that you can't make out: https://arstechnica.com/information-technology/2023/10/sob-story-about-dead-grandma-tricks-microsoft-ai-into-solving-captcha/ https://arstechnica.com/information-technology/2023/10/sob-s...
- malfist 3y agoSure, it's cost prohibitive now. But what about in five years? Or probably even less.
- dgellow 3y agoThen you have a new type of captcha. That has always been a cat and mouse type of dynamics, captchas have been evolving, techniques to break them too.
- idiotsecant 3y ago>Then you have a new type of captcha. You're in a desert, walking along when you look down and see a tortoise. It's crawling toward you. You reach down and flip it over on its back, its belly baking in the hot sun, beating its legs trying to turn itself over. But it can't. Not with out your help. But you're not helping. Why is that?
- rapnie 3y ago> they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks Might do that unobtrusively for the average person, by using projects like mCaptcha [0] for instance. [0] https://mcaptcha.org/ https://mcaptcha.org/
- JCharante 3y agoOh what a perfect find. I have on my todolist to add POW to some of my api endpoints
- berkes 3y agoI've had that idea for years. Two versions that I experimented with. One is where the incoming POW hashes contribute to hashing power for some blockchain mining. An alternative "pay as you use the API" system. The other using hashcash. Just a way to slow down abuse. Both, however, suffer from the downside that many/all "ASIC resisting crypto mining" suffer from as well: the cheapest CPU power is CPU power from machines/power you don't own. Botnets, viruses, trojans etc. So such a mechanism to throtthe or protect APIs won't hold back spammers and abusers for long.
- rezonant 3y agoDirty energy is (often) cheap, so that's the energy the bad actors will use. I don't know that incentivizing bad actors to waste energy in a climate crisis is the best way to fight this problem. You might correctly claim clean energy is often cheaper, but you must also consider the regions in which they'll get away with nefarious activity, and whether those areas have made the investments into making clean energy cheap.
- hosh 3y agoIf I remember correctly, Google’s CAPCHA’s test isn’t in correctly identifying images, but the behavior of the runtime system (mouse jitter, for example) while the capcha is presented to the user. The image identification was not the real test and serves as training data. It has been like that for years. (But with agent-based behaviors from say, Q*, mouse jitter alone won’t help; there are probably other signals like fluctuation in cpu or battery life expenditures) You could already see the writing on the wall with image identification years ago, when the obscuration techniques became more elaborate. It was an arms race. I was having trouble with them. I can see less technically inclined being able to use them. I imagined how much worse it was for people with color blindness, disabilities, or people forced to use them at public library computers because that is all they have. Open source capcha projects have either not been clued in, or don’t have the resources to pull this off. Google didn’t just switch out which signals they tested, they also wrote an obfuscating virtual machine executing within the browser environment (if I remember that article taking about this correctly). That was years ago and who knows what they do now — for all we know, the “byte code” running the test is now a neural net of some kind.
- saberience 3y agoFor those with elderly parents the writing has been on the wall for years. It’s sad but my mother has for some time been effectively locked out of parts of the internet as she is unable to complete these kinds of captures due to eyesight issues. I mean, I’ve sometimes had to try three or four times with certain captures and I have perfect eyesight (with my glasses). I feel so badly for those with vision or hearing issues with an empathy I never had when I was younger. They are so often simply forgotten.
- graphe 3y agoThere's audio captcha. Try to click the headphone logo (Google captcha has it).
- J_Shelby_J 3y agoI’ve switched to audio captchas completely because it’s quicker and sometimes the image captchas just won’t work.
- loup-vaillant 3y agoThat non-theatrical role would likely be better served by actual throttling or computational proof of work.
- sebzim4500 3y agoI am pretty confident that, when it comes to browser users, proof of work simply doesn't work. The disparity in speed between GPUs and javascript is so high that either you are a non-issue to a sane attacker or you make your users sit for a minute with their fans on full waiting to be able to sign in.
- AAAAaccountAAAA 3y agoWould it be possible to conceive a proof-of-work that is difficult to parallelize, making it harder for GPU computing?
- sebzim4500 3y agoThere are PoW systems which are designed to be difficult to run on ASICs, but modern GPUs can generally run them. Even if you find one that has to run on CPU, these kind of functions will still be much faster running in native code than in js/wasm.
- GoblinSlayer 3y agobcrypt, litecoin
- loup-vaillant 3y agoArgon2d on WASM at the very least. I would never suggest we use something as slow as JavaScript for a proof of work.