12 ms·

"We need a pony. And the moon on a stick. By next Thursday."

208 points by cstross 12y ago

tormeh 12y ago
On the other hand, handing out huge wads of cash in the hope that the recipient will do the impossible may not be likely to be directly successful, but it often has interesting results. I wonder what they will discover. I just hope that it won't be classified.
- danielweber 12y ago
  Think of the spinoff technologies! (That was, btw, sarcasm. Normally I don't like it, but it seemed appropriate here.)
- arethuza 12y ago
  I can certainly imagine automated disinformation techniques - or as Neal Stephenson described it in Anathem: "Artificial Inanity".
  - logfromblammo 12y ago
    What book was it that posited an automated system for the retroactive logical justification of predetermined conclusions back to reasonable-sounding premises? That was Adams's _Dirk Gently's Holistic Detective Agency_, right? A program called Anthem? That would also be a plausible spinoff.
phkahler 12y ago
My daughter has been identifying sarcasm since she was 5 or 6 years old. At 8 she's really good at it. While non-trivial for machines, this problem is not really as advanced for people as it may seem.
- Torn 12y ago
  Non-trivial for machines? That's a massive understatement. (Or were you being sarcastic?)
  - tomswartz07 12y ago
    Suure he was...
  - j03w 12y ago
    I think he's Aussie so that's normal.
- sanderjd 12y ago
  Not sure whether or not this is sarcasm... Too bad we don't have that detector yet!
  - scj 12y ago
    Ambiguity implies guilt! I'm pretty sure the secret service could be convinced to proceed on that maxim.
- deleted 12y ago
  [deleted]
- Houshalter 12y ago
  We also have clues about the tone it was said in, which is more difficult in text.
- VLM 12y ago
  Her world is probably extremely small. So sarcastic commentary about adult political matters is highly likely to fly over her head. This might be an interesting way to detect / work around / operationally neutralize the theorized sarcasm detector, as it should be easy to make a google-verifiable sarcastic comment thats ridiculously out of line for most mouth breathing humans WRT CS or quantum mechanics or math in general. What does this mean to the hypothetical algo: "If P=NP then I'm killing Heisenbergs cat". (edited to add, I hope the algo tries to prove P=NP or crashes when it thinks about it) Even more fun way to hack with the hypothetical AI, tag something obscure in wikipedia with your site as an external source, then use that obscure reference in a theoretical threat against the prez and log the accesses to your site and research your site accesses accordingly. Perhaps by careful, fast, automated wikipedia edits if you can make changes faster than the generic spider can figure out what to look at, you could theoretically SWAT people using the sarcasm detector. So if you posted in 2011, "Obama's going to get Wisconsin in 2012" as in electoral college votes, by proper automated wikipedia editing you could turn that into a SWATting incident by editing "Wisconsin" to be a military grade nerve gas before the sarcasm detector accesses it. Who cares if your automated edit gets reverted in 5 minutes if the men in black are already carrying M16s in the van on the way to the SWATting victim's house by the time its reverted...
- SoftwareMaven 12y ago
  Kids are amazingly bad at detecting sarcasm, due to the lack of shared context. Instead, they rely on vocal cues to identify it[1]. My sarcasm is very dry, and my 15 year old daughter still has trouble recognizing it. But, with the NSA databases, any government solution will have that shared context, right? (I can't tell if I'm being sarcastic...) 1. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8624.1990.tb03568.x/abstract http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8624.1990....
- perlgeek 12y ago
  A 5 or 6 year old can also speak one or several natural languages very fluently, drive a bicycle, and do lots of other stuff that computers (and robots) are very bad at. So that's actually a pretty high bar in those areas that humans have traditionally much better at than computers.
cafard 12y ago
Almost 40 years ago, some kid jumped up in the front row of an audience and squirted Ronald Reagan with a squirt gun--water only, no injury inflicted. After the Secret Service tackled him, cuffed him, and took him off for psychiatric evaluation, they started to inquire among persons who had previously known him. One was an acquaintance of mine, who had lived down the hall from the kid in a college dorm. As he recounted it, the conversation went in part like this: Secret Service guys: Did he seem like the sort of person who would do something like this? Acquaintance: No, really pretty serious. Now, I might think it was funny to do something like that. [Smiles, chuckles.] SSg: [Silence, no smiles, no chuckles] Let's face it, they aren't paid to have a sense of humor.
- keithpeter 12y ago
  Ford survived two assassination attempts. In both cases, the assailants were able to get close, but no bullets fired. Regan survived an actual attack - shots fired and one bullet stopped. The assailant was seen in the waiting guests visibly nervous, abrupt and aggressive. The 'special agent' (the ones with guns who travel with the president) got in the way to block further shots. I suspect it is a bit edgy being the one just behind the president...
JacobAldridge 12y ago
Funnily enough, I once sponsored a Booth at a convention for presidential assassins.
andyjohnson0 12y ago
According to this [1] BBC News article from last year, a French company called Spotter [2] offers an analytics tool that they claim can identify sarcasm in web and social media comments. Supports multiple languages to 80% accuracy. I've no idea if it works and their site is rather light on detail. [1] http://www.bbc.co.uk/news/technology-23160583 http://www.bbc.co.uk/news/technology-23160583 [2] http://spotter.com/ http://spotter.com/
- vog 12y ago
  It would be interesting to know how this algorithm behaved on fundamentalist texts. (see Poe's law, https://en.wikipedia.org/wiki/Poe%27s_law https://en.wikipedia.org/wiki/Poe%27s_law)
- opium_tea 12y ago
  This video suggests their early prototypes required some further calibration: https://www.youtube.com/watch?v=mSy5mEcmgwU https://www.youtube.com/watch?v=mSy5mEcmgwU
- Create 12y ago
  DARPA / IBM http://www.research.ibm.com/cognitive-computing/index.shtml http://www.research.ibm.com/cognitive-computing/index.shtml
- rainforest 12y ago
  This paper appears to claim up to 91% accuracy (precision) [1] by looking at hashtags, with a description of the approach they use (and a link to the software). The approaches probably aren't directly comparable since one depends on hashtags while the other doesn't though. [1] : https://gate.ac.uk/sale/lrec2014/arcomem/sarcasm.pdf https://gate.ac.uk/sale/lrec2014/arcomem/sarcasm.pdf
- eli_gottlieb 12y ago
  Yeah, really. It's creating a whole human mind that's hard. Particular, isolated tasks or features about particular things in particular cultures are not actually that hard to learn/classify in themselves, as your data corpus will be more uniform than if you were trying to perform some more general cognition across a wider variety of contexts. More uniformity makes it easier to learn.
digitalengineer 12y ago
Or.. force a public ID (or FaceBook) on every Internet-user and arrest/scare the hell out of a few teens being "sarcastic". Send the bill for the SWAT team or something like that. That will stop most people from behaving "sarcastic" on the social media. Did you ever try sarcasm during a conversation with the police/TSA? Doesn't end well.
- sbarre 12y ago
  Are you being sarcastic?
  - digitalengineer 12y ago
    Ha ha! No, I was just trying to see the world through their eyes... In their view, wanting these functions isn't the problem. We are the problem.
rquantz 12y ago
And lest we forget, reliably identifying sarcasm on the internet is pretty hard even for humans who have all the requisite cultural knowledge. Unless the secret service can turn on the twitterer's webcam and observe their facial expression while they type, this really is a no go. And we all know that could never happen.
- smoyer 12y ago
  Sarcasm doesn't work well on the Internet for the same reasons it doesn't work well in person when conversing with a stranger ... but the stranger has the advantage of reading your body language. I was brought up in a family that used sarcasm heavily and "turning it down" on the Internet has been a real challenge. Sometimes I still fail (something I believe is obviously sarcastic is misread).
  - Swizec 12y ago
    Being European is the worst when it comes to sarcasm. I can't even express my excitement/happiness over something without it coming off as sarcastic.
- bodski 12y ago
  I think what will happen is for any given target a weighting will be inferred, giving the likelihood of that person being sarcastic or joking. As more and more data about us is collected, about what we say/type vs. what we actually do, this has the potential to become more accurate over time. Facial and voice-tone analysis, analysis of our peers' responses to what we say/write, analysis of our actual movements/purchases compared to what we say/write ("I'm so going along to that Nickelback concert at the weekend...!") etc., combined with leading brains in stats and ML lead me to think this problem could be tractable to some extent in the future. Scary :-/ If something like this comes to be, and is deployed 'wholesale', it has the potential to influence our behaviour, and over generations, alter our definition of 'sarcasm' by what we are able to get away with saying. Insidious! That said I hope Charlie is right and that humans can game the system significantly that it is a non-starter.
  - 12y ago
danohuiginn 12y ago
This relevant Monty Python skit is worth hoisting out of Stross' comments and posting here: http://www.youtube.com/watch?v=-fNvi6xG-5Y http://www.youtube.com/watch?v=-fNvi6xG-5Y from about 1:00
sirsar 12y ago
This comment thread seems like a good place to test detectors on.
skizm 12y ago
Poe's Law says this is pretty much impossible. It is tough enough for humans to tell sarcasm. How are we going to get machines to do it well. http://en.wikipedia.org/wiki/Poe's_law http://en.wikipedia.org/wiki/Poe's_law
- chillingeffect 12y ago
  > well. Well enough to protect the president, or well enough to perpetuate the taxpayer->government->university financial cycle by making something that kinda sorta works but could get better with uint8_t more years and uint32_t more dollars‽
- jdbernard 12y ago
  I would just like to point out that there are many things that are hard for humans to do but easy for machines (cars drive much faster then humans can run, for example). So that argument alone is not persuasive. Better would be to argue that it is not only hard for humans to determine between sarcasm and genuine speech, we have no idea of how it would be done well. It is not a problem which is dominated by our limited abilities, it is dominated by imperfect knowledge (the inability to know the internal thoughts of another person). This same limitation seems to apply to any automated solution which only had access to the public speech (text, tweet, etc.). Therein lies the intractable difficulty. However, consider if we had a device that could read minds. Maybe some model is built that can accurately translate brain waves, or the input of subdermal probes into thoughts and intentions. Then it would be much simpler to write a program that could detect sarcasm. It might still need the physical presence of the author, but again there may be a creative solution to that. Just because it is hard for humans doesn't mean we can't make it easy using our tools. That is one of the prime characteristics of our species.
  - skizm 12y ago
    My point is, for any given computer program, a human, given enough time, can sit down with pencil+paper and figure out the output given an input. If humans can't yet detect sarcasm accurately, that is evidence that it will be near impossible to do it programmatically. Edit: I'm not saying to not try. Clark's first law says we should always try things, even the crazy stuff. (as you can tell I like my eponymous laws).
kndyry 12y ago
The original SOW, if anyone's interested: https://www.fbo.gov/utils/view?id=fb2ca11b7d9ca8c61e5ee6d8ae69ad4d https://www.fbo.gov/utils/view?id=fb2ca11b7d9ca8c61e5ee6d8ae...
- chillingeffect 12y ago
  Thanks for that. It's always great to see more of the original concept than any columnist's hyperfocused version of it. It seems "Ability to sarcasm and false positives" is just one of the large list of 22 reqs. The list also includes: • Compatibility with Internet Explorer 8;
  - ph0rque 12y ago
    > Compatibility with Internet Explorer 8 That was sarcasm.
    - smileysteve 12y ago
      Winner!
  - coldcode 12y ago
    Compatibility with Internet Explorer 8 is an anti-terror requirement. Clearly no terrorist is dumb enough to use it still so it must be safe.
Zarkonnen 12y ago
Reminds me of the Twitter Joke Trial incident: http://en.wikipedia.org/wiki/Twitter_Joke_Trial http://en.wikipedia.org/wiki/Twitter_Joke_Trial Also, self-linkingly, the problem is that people don't tend to broadcast their violent intentions very reliably: http://zarkonnen.com/terrorists_on_twitter/ http://zarkonnen.com/terrorists_on_twitter/
stcredzero 12y ago
We Homo sapiens were intelligently designed by an omnitrollistic god as the ultimate act of sarcasm. It purposely designed an entity guaranteed to destroy itself by soiling its own world and creating ravenous hegemonic super optimizing AIs. Then it planted the fossil record as an elaborate prank.
drzaiusapelord 12y ago
I think in the future people will be perplexed by how sci-fi during this period was absolutely obsessed by the so-called singularity. Its a current fad, accepted by many, but not backed up by anything substantial like a lot of fads from the past. Its a little embarrassing to read some sharp thinkers from the 1970s who fell into cheap counter-culture concepts like astrological signs or shammy Indian gurus or shoddy California-style Buddhism, that often has little or nothing to do with the Buddhavacana source materials. It really surprises me that we think there's a realistic path to the singularity and that we obsessed about it. Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. Then there's the false assumption that we can get to this level before doing things like, say, solving all mental health issues and that it can magically be scaled to super-human speeds. For all we know cognition happens at a certain speed and any attempts past that are troublesome. Look at how easily we suffer from mania and other issues when our natural limiters go haywire. Singularity prediction is almost a cargo cult. Its weird that so many take it seriously.
- jp555 12y ago
  We are currently in the midst of a singularity of tools. Stone > Bronze > Steel > Silicon > Software > It's the "geek rapture" version that's more fantastic and extremely unlikely.
  - _archon_ 12y ago
    This is a very interesting perspective. What do you think is the next step? I am asking for an educated guess, or even interested speculation; people in the middle of the bronze age likely didn't know that steel was the next step until after iron was extracted for the first time. If I had to guess, I would put the next phase at "integration" or "control." In that once we have functional, reliable libraries and virtual instruments, the logical next step is to integrate the high-level control of such into our physiologies. Alternatively, I could see "space" being an age we expand into, drawing our resources increasingly from space and broadening the scope of our potential that way. I'd like to see these happen in parallel if possible.
VLM 12y ago
Never overlook the power of organizational CYA. Likely the next announcement will be they've discovered automated sarcasm analysis of twitter feeds is, in fact, as suspected, completely impossible. Then announcement after that one will be a policy change to cease to analyze twitter feeds because its impossible to triage fast enough to keep up and its all noise anyway. Then the announcement after something tragic happens will be a steaming pile of CYA about how the worlds best minds couldn't find a way to automate the detection of XYZ but if our budget were only a little larger ... (never let a crisis go to waste...) This is beginning to read like a Stross book plot and maybe he's pissed because yet again he's scooped where the world is getting weirder faster than his plots are getting weird, like happened with the halting state series (as I recall). The laundry series is still safe from this danger, at least as far as we know...
3am 12y ago
This article rubbed me the wrong way. The Secret Service is trying to improve their methods in an open, competitive way. Maybe they don't have to perfectly solve sarcasm recognition to reduce their false positive rates. And let me add that anyone that's stupid enough to try to troll/game this system deserves all the consequences of their actions. edit: Remembering Bayes: Given a threat on someone the SS is charged w/ protecting, what is the probability it is not actionable?
mmmooo 12y ago
Anyone care to guesstimate how many assassination plots could have been prevented if just read the assailants public twitter feed in which he or she outlined, in plain english, his/her intentions to perform said act? Somehow I feel like the people that are serious about this level of criminal activity, and capable of actually accomplishing it, aren't first announcing their intentions on twitter.
- deleted 12y ago
  [deleted]
- VLM 12y ago
  Thought experiment: As a culture we've been pretty effective lately at electing miserable disappointing failures to public office. This seems a relatively uncontroversial observation, and note I'm careful not to name any names so as to not appear biased toward one side or the other of the one coin we've been dealt. Precondition 2 is everyone has social media accounts where they say stuff no one cares about, constantly. The CB radio of this decade. Precondition 3 is anyone planning an assassination is smart enough to self censor and say nothing about the entire topic. Given the above preconditions, the only rational conclusion is 99.9% of the population will continually be whining and complaining, and you can focus your attentions on the 0.1% who are at least considering doing something bad enough to self censor. So the thought experiment is Obama comes to visit a city and the guy who gets a SS visit is the only guy in the whole city who's not rambling on in social media about "Kenya" or "Can't believe I wasted my vote on him" or writing racial slurs or "he broke every campaign promise" or whatever.
  - cstross 12y ago
    Disagree. As a culture we have developed public offices that are structured to prevent anyone who might be elected with a mandate to effect change from doing so. The iron law of bureaucracy applies: after the first generation most of the staff of any organization see their job not as pursuing the mission statement of the org but as preserving their own jobs. Attempts to change things generate resistance from within because, hey, jobs might be threatened. Hell, political candidates who might challenge their party's ability to win future elections by accomplishing change (which might be unwelcome to some elements of the voting -- or election-buying -- public or oligaarchs) are weeded out before they get a chance to run for office. The "disappointing" incumbent is merely a printed-paper face on the front of a machine. Which some loons choose to use for target practice. Resulting in the existence of a vast reactionary bureaucracy dedicated to extirpating threats to printed-paper faces.
kator 12y ago
My favorite NLP/Computer Language problem: Moe: Hey is Curly back from Vacation yet? Larry: I saw a Red Lamborghini in the parking lot. Moe: Cool We all assume Curly now drives a Red Lamborghini meanwhile most computer language systems would be lost at the inference here. Children learn this trick early computers are very challenged to understand these sorts of language challenges.
- nkuttler 12y ago
  I don't think the main challenge is to teach this "trick" to computers. The problem is probably to give them the proper context. If the system knows Curly drives a red lambo, and that it is a rare car in this area, and that Curly usually only parks in the parking lot when he's around, and etc. etc. etc.
  - mcguire 12y ago
    Are you by any chance channelling Doug Lennat?
  - nmc 12y ago
    > "If the system knows [...]" He was outlining that the difference is: humans do not need the context. I do not know who Curly is, and I still got what car he drives just by reading the three-line dialog.
    - click170 12y ago
      Except that you already know what a red lambo is, and that its safe to assume its a rare car almost anywhere, et al. You know this from experience, as do I and most other people. I think that was the parents point.
    - sosborn 12y ago
      >I still got what car he drives just by reading the three-line dialog. I inferred that too, but it doesn't mean that it is correct.
  - throwaway283719 12y ago
quantail 12y ago
We'll bring the pony and the stick.
- eli_gottlieb 12y ago
  I was going to say, if you buy the right figurine, you can give the client a moon-pony.
mcv 12y ago
Nice of him to provide some test data.
snowwrestler 12y ago
HN (and similar tech-oriented forums) often host outraged discussions of government over-reactions. They also host excited discussions of linguistic technologies like Siri or Watson. Here we have a government agency attempting to push the envelope of technology in hopes of reducing their over-reactions. Seems like the sort of thing that would be greeted warmly but, nope, apparently not.
- deleted 12y ago
  [deleted]
- javajosh 12y ago
  Ah, but therein lies the (too often hidden) assumption behind the angst: we don't want law-enforcement to scale. It should take a human to destroy a human life. It should take a human to violate another human's privacy. Government, with it's monopoly on force, has the capability of doing these things at it's whim, but we are fundamentally opposed to treating these actions as "productivity" and the use of computers as "productivity enhancements" in those contexts. So, while the minor impulse (reduce overreach) is right, the major impulse (automated justice) is still very wrong.
pjc50 12y ago
For use of language to circumvent government filtering, see http://en.wikipedia.org/wiki/Baidu_10_Mythical_Creatures_%28Internet_meme%29 http://en.wikipedia.org/wiki/Baidu_10_Mythical_Creatures_%28... (I suppose that's timely with the Tianamen Square anniversary)
madaxe_again 12y ago
What I want to know is how Charlie knows the subject line of all my client emails ever.
milliams 12y ago
This wonderful comic comes to mind every time the discussion of sarcasm comes up http://www.qwantz.com/index.php?comic=168 http://www.qwantz.com/index.php?comic=168
flatline 12y ago
I think the most salient point was further down in a comment: > ...the real target is the autonomous exercise of professional judgement.
js2 12y ago
I come from a family with a dry sort of sarcasm that we deploy often. My sister and I were having a conversation the other day when it slipped into sarcasm and we proceeded to go back and forth till one of us said something serious. Or so the other thought. Eventually neither of us could tell. My daughter (13) observing this suddenly chimed in: "I think you're caught in a sarcasm trap." She claims to have invented the term on the spot. My 13 year old knows it's sometimes not possible for humans to detect sarcasm, much less a computer program. It's a shame that the secret service doesn't too. (I would venture to say that more people struggle with sarcasm than not. It often requires shared culture/experience to detect if it's not delivered in an obvious tone.)
- mcguire 12y ago
  The ability to reliably identify sarcasm, with a low false-positive rate but even with a very high false-negative rate, would likely save the Secret Service a significant amount of money and a very significant amount of bad press. Also, your daughter is much more perceptive than the vast majority of the people I know.
louthy 12y ago
Origin of the phrase: Moon on a stick "phrase used widely by teenagers and young adults in the mid 90s as it was made popular by the comedy geniuses Stew Lee & Richard Herring. they used it mostly in the second series of their hit tv & radio show 'Fist of Fun'.[1] It means basically to want everything, if you want the moon on a stick then you want everything, including things you can't have."[2] [1] https://www.youtube.com/watch?v=ERDUbAv8Qz0 https://www.youtube.com/watch?v=ERDUbAv8Qz0 [2] http://www.urbandictionary.com/define.php?term=moon%20on%20a%20stick http://www.urbandictionary.com/define.php?term=moon%20on%20a...
- Brakenshire 12y ago
  If you like this sort of thing, you might want to try out Stewart Lee's Comedy Vehicle: http://www.youtube.com/watch?v=-yUDh_IErT4 http://www.youtube.com/watch?v=-yUDh_IErT4
restless 12y ago
In case nobody mentioned The Simpsons yet "Oh a sarcasm detector that's a really useful invention"Comic Book Guy http://www.youtube.com/watch?v=mSy5mEcmgwU http://www.youtube.com/watch?v=mSy5mEcmgwU
dcalacci 12y ago
shamelessly promoting work by a colleague of mine that detects sarcasm in online product reviews: http://www.cs.huji.ac.il/~oren/papers/sarcasmAmazonICWSM10.pdf http://www.cs.huji.ac.il/~oren/papers/sarcasmAmazonICWSM10.p...
e12e 12y ago
> Or they could just ban sarcasm on the internet. I think what we're all half-expecting, is that they'll define the output of the program to be true; thereby a) ensuring the program is 100% correct, and b) In the event of real threats (as identified by the program) have actionable intelligence. Almost, but not quite, the same as banning sarcasm on the Internet.
kingtoe 12y ago
Summary: "No system to monitor or control the population will work because the entire population would obsess over gaming the system." Except no one cares enough to game it, or ever would. Sigh Another programmer who thinks that his buddies who obsess over gaming systems represent more than .001% of the population in question. Next.
davidgerard 12y ago
Favourite comment: "I went to a presentation from a sentiment analysis business last year. They were discussing how they were using it on the firehose feed of everything from twitter. Rather than assassinations, they were more interested in marketing, branding, and the impact on stock prices (there are apparently some iiiiiiinteresting correlations). The thing that entertained me was they said they had to special-case content coming from the UK — because our levels of sarcasm/irony were skewing their results. I think it was the most patriotic moment of my life <sniff> ;-)"

208 points by cstross 12y ago

On the other hand, handing out huge wads of cash in the hope that the recipient will do the impossible may not be likely to be directly successful, but it often has interesting results. I wonder what they will discover. I just hope that it won't be classified.

danielweber 12y ago
Think of the spinoff technologies! (That was, btw, sarcasm. Normally I don't like it, but it seemed appropriate here.)
arethuza 12y ago
I can certainly imagine automated disinformation techniques - or as Neal Stephenson described it in Anathem: "Artificial Inanity".
- logfromblammo 12y ago
  What book was it that posited an automated system for the retroactive logical justification of predetermined conclusions back to reasonable-sounding premises? That was Adams's _Dirk Gently's Holistic Detective Agency_, right? A program called Anthem? That would also be a plausible spinoff.

phkahler 12y ago

My daughter has been identifying sarcasm since she was 5 or 6 years old. At 8 she's really good at it. While non-trivial for machines, this problem is not really as advanced for people as it may seem.

Torn 12y ago
Non-trivial for machines? That's a massive understatement. (Or were you being sarcastic?)
- tomswartz07 12y ago
  Suure he was...
- j03w 12y ago
  I think he's Aussie so that's normal.
sanderjd 12y ago
Not sure whether or not this is sarcasm... Too bad we don't have that detector yet!
- scj 12y ago
  Ambiguity implies guilt! I'm pretty sure the secret service could be convinced to proceed on that maxim.
deleted 12y ago
[deleted]
Houshalter 12y ago
We also have clues about the tone it was said in, which is more difficult in text.
VLM 12y ago
Her world is probably extremely small. So sarcastic commentary about adult political matters is highly likely to fly over her head. This might be an interesting way to detect / work around / operationally neutralize the theorized sarcasm detector, as it should be easy to make a google-verifiable sarcastic comment thats ridiculously out of line for most mouth breathing humans WRT CS or quantum mechanics or math in general. What does this mean to the hypothetical algo: "If P=NP then I'm killing Heisenbergs cat". (edited to add, I hope the algo tries to prove P=NP or crashes when it thinks about it) Even more fun way to hack with the hypothetical AI, tag something obscure in wikipedia with your site as an external source, then use that obscure reference in a theoretical threat against the prez and log the accesses to your site and research your site accesses accordingly. Perhaps by careful, fast, automated wikipedia edits if you can make changes faster than the generic spider can figure out what to look at, you could theoretically SWAT people using the sarcasm detector. So if you posted in 2011, "Obama's going to get Wisconsin in 2012" as in electoral college votes, by proper automated wikipedia editing you could turn that into a SWATting incident by editing "Wisconsin" to be a military grade nerve gas before the sarcasm detector accesses it. Who cares if your automated edit gets reverted in 5 minutes if the men in black are already carrying M16s in the van on the way to the SWATting victim's house by the time its reverted...
SoftwareMaven 12y ago
Kids are amazingly bad at detecting sarcasm, due to the lack of shared context. Instead, they rely on vocal cues to identify it[1]. My sarcasm is very dry, and my 15 year old daughter still has trouble recognizing it. But, with the NSA databases, any government solution will have that shared context, right? (I can't tell if I'm being sarcastic...) 1. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8624.1990.tb03568.x/abstract http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8624.1990....
perlgeek 12y ago
A 5 or 6 year old can also speak one or several natural languages very fluently, drive a bicycle, and do lots of other stuff that computers (and robots) are very bad at. So that's actually a pretty high bar in those areas that humans have traditionally much better at than computers.

cafard 12y ago

Almost 40 years ago, some kid jumped up in the front row of an audience and squirted Ronald Reagan with a squirt gun--water only, no injury inflicted. After the Secret Service tackled him, cuffed him, and took him off for psychiatric evaluation, they started to inquire among persons who had previously known him. One was an acquaintance of mine, who had lived down the hall from the kid in a college dorm. As he recounted it, the conversation went in part like this: Secret Service guys: Did he seem like the sort of person who would do something like this? Acquaintance: No, really pretty serious. Now, I might think it was funny to do something like that. [Smiles, chuckles.] SSg: [Silence, no smiles, no chuckles] Let's face it, they aren't paid to have a sense of humor.

keithpeter 12y ago
Ford survived two assassination attempts. In both cases, the assailants were able to get close, but no bullets fired. Regan survived an actual attack - shots fired and one bullet stopped. The assailant was seen in the waiting guests visibly nervous, abrupt and aggressive. The 'special agent' (the ones with guns who travel with the president) got in the way to block further shots. I suspect it is a bit edgy being the one just behind the president...

JacobAldridge 12y ago

Funnily enough, I once sponsored a Booth at a convention for presidential assassins.

andyjohnson0 12y ago

According to this [1] BBC News article from last year, a French company called Spotter [2] offers an analytics tool that they claim can identify sarcasm in web and social media comments. Supports multiple languages to 80% accuracy. I've no idea if it works and their site is rather light on detail. [1] http://www.bbc.co.uk/news/technology-23160583 http://www.bbc.co.uk/news/technology-23160583 [2] http://spotter.com/ http://spotter.com/

vog 12y ago
It would be interesting to know how this algorithm behaved on fundamentalist texts. (see Poe's law, https://en.wikipedia.org/wiki/Poe%27s_law https://en.wikipedia.org/wiki/Poe%27s_law)
opium_tea 12y ago
This video suggests their early prototypes required some further calibration: https://www.youtube.com/watch?v=mSy5mEcmgwU https://www.youtube.com/watch?v=mSy5mEcmgwU
Create 12y ago
DARPA / IBM http://www.research.ibm.com/cognitive-computing/index.shtml http://www.research.ibm.com/cognitive-computing/index.shtml
rainforest 12y ago
This paper appears to claim up to 91% accuracy (precision) [1] by looking at hashtags, with a description of the approach they use (and a link to the software). The approaches probably aren't directly comparable since one depends on hashtags while the other doesn't though. [1] : https://gate.ac.uk/sale/lrec2014/arcomem/sarcasm.pdf https://gate.ac.uk/sale/lrec2014/arcomem/sarcasm.pdf
eli_gottlieb 12y ago
Yeah, really. It's creating a whole human mind that's hard. Particular, isolated tasks or features about particular things in particular cultures are not actually that hard to learn/classify in themselves, as your data corpus will be more uniform than if you were trying to perform some more general cognition across a wider variety of contexts. More uniformity makes it easier to learn.

digitalengineer 12y ago

Or.. force a public ID (or FaceBook) on every Internet-user and arrest/scare the hell out of a few teens being "sarcastic". Send the bill for the SWAT team or something like that. That will stop most people from behaving "sarcastic" on the social media. Did you ever try sarcasm during a conversation with the police/TSA? Doesn't end well.

sbarre 12y ago
Are you being sarcastic?
- digitalengineer 12y ago
  Ha ha! No, I was just trying to see the world through their eyes... In their view, wanting these functions isn't the problem. We are the problem.

rquantz 12y ago

And lest we forget, reliably identifying sarcasm on the internet is pretty hard even for humans who have all the requisite cultural knowledge. Unless the secret service can turn on the twitterer's webcam and observe their facial expression while they type, this really is a no go. And we all know that could never happen.

smoyer 12y ago
Sarcasm doesn't work well on the Internet for the same reasons it doesn't work well in person when conversing with a stranger ... but the stranger has the advantage of reading your body language. I was brought up in a family that used sarcasm heavily and "turning it down" on the Internet has been a real challenge. Sometimes I still fail (something I believe is obviously sarcastic is misread).
- Swizec 12y ago
  Being European is the worst when it comes to sarcasm. I can't even express my excitement/happiness over something without it coming off as sarcastic.
bodski 12y ago
I think what will happen is for any given target a weighting will be inferred, giving the likelihood of that person being sarcastic or joking. As more and more data about us is collected, about what we say/type vs. what we actually do, this has the potential to become more accurate over time. Facial and voice-tone analysis, analysis of our peers' responses to what we say/write, analysis of our actual movements/purchases compared to what we say/write ("I'm so going along to that Nickelback concert at the weekend...!") etc., combined with leading brains in stats and ML lead me to think this problem could be tractable to some extent in the future. Scary :-/ If something like this comes to be, and is deployed 'wholesale', it has the potential to influence our behaviour, and over generations, alter our definition of 'sarcasm' by what we are able to get away with saying. Insidious! That said I hope Charlie is right and that humans can game the system significantly that it is a non-starter.
- 12y ago

danohuiginn 12y ago

This relevant Monty Python skit is worth hoisting out of Stross' comments and posting here: http://www.youtube.com/watch?v=-fNvi6xG-5Y http://www.youtube.com/watch?v=-fNvi6xG-5Y from about 1:00

sirsar 12y ago

This comment thread seems like a good place to test detectors on.

skizm 12y ago

Poe's Law says this is pretty much impossible. It is tough enough for humans to tell sarcasm. How are we going to get machines to do it well. http://en.wikipedia.org/wiki/Poe's_law http://en.wikipedia.org/wiki/Poe's_law

chillingeffect 12y ago
> well. Well enough to protect the president, or well enough to perpetuate the taxpayer->government->university financial cycle by making something that kinda sorta works but could get better with uint8_t more years and uint32_t more dollars‽
jdbernard 12y ago
I would just like to point out that there are many things that are hard for humans to do but easy for machines (cars drive much faster then humans can run, for example). So that argument alone is not persuasive. Better would be to argue that it is not only hard for humans to determine between sarcasm and genuine speech, we have no idea of how it would be done well. It is not a problem which is dominated by our limited abilities, it is dominated by imperfect knowledge (the inability to know the internal thoughts of another person). This same limitation seems to apply to any automated solution which only had access to the public speech (text, tweet, etc.). Therein lies the intractable difficulty. However, consider if we had a device that could read minds. Maybe some model is built that can accurately translate brain waves, or the input of subdermal probes into thoughts and intentions. Then it would be much simpler to write a program that could detect sarcasm. It might still need the physical presence of the author, but again there may be a creative solution to that. Just because it is hard for humans doesn't mean we can't make it easy using our tools. That is one of the prime characteristics of our species.
- skizm 12y ago
  My point is, for any given computer program, a human, given enough time, can sit down with pencil+paper and figure out the output given an input. If humans can't yet detect sarcasm accurately, that is evidence that it will be near impossible to do it programmatically. Edit: I'm not saying to not try. Clark's first law says we should always try things, even the crazy stuff. (as you can tell I like my eponymous laws).

kndyry 12y ago

The original SOW, if anyone's interested: https://www.fbo.gov/utils/view?id=fb2ca11b7d9ca8c61e5ee6d8ae69ad4d https://www.fbo.gov/utils/view?id=fb2ca11b7d9ca8c61e5ee6d8ae...

chillingeffect 12y ago
Thanks for that. It's always great to see more of the original concept than any columnist's hyperfocused version of it. It seems "Ability to sarcasm and false positives" is just one of the large list of 22 reqs. The list also includes: • Compatibility with Internet Explorer 8;
- ph0rque 12y ago
  > Compatibility with Internet Explorer 8 That was sarcasm.
  - smileysteve 12y ago
    Winner!
- coldcode 12y ago
  Compatibility with Internet Explorer 8 is an anti-terror requirement. Clearly no terrorist is dumb enough to use it still so it must be safe.

Zarkonnen 12y ago

Reminds me of the Twitter Joke Trial incident: http://en.wikipedia.org/wiki/Twitter_Joke_Trial http://en.wikipedia.org/wiki/Twitter_Joke_Trial Also, self-linkingly, the problem is that people don't tend to broadcast their violent intentions very reliably: http://zarkonnen.com/terrorists_on_twitter/ http://zarkonnen.com/terrorists_on_twitter/

stcredzero 12y ago

We Homo sapiens were intelligently designed by an omnitrollistic god as the ultimate act of sarcasm. It purposely designed an entity guaranteed to destroy itself by soiling its own world and creating ravenous hegemonic super optimizing AIs. Then it planted the fossil record as an elaborate prank.

drzaiusapelord 12y ago

I think in the future people will be perplexed by how sci-fi during this period was absolutely obsessed by the so-called singularity. Its a current fad, accepted by many, but not backed up by anything substantial like a lot of fads from the past. Its a little embarrassing to read some sharp thinkers from the 1970s who fell into cheap counter-culture concepts like astrological signs or shammy Indian gurus or shoddy California-style Buddhism, that often has little or nothing to do with the Buddhavacana source materials. It really surprises me that we think there's a realistic path to the singularity and that we obsessed about it. Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. Then there's the false assumption that we can get to this level before doing things like, say, solving all mental health issues and that it can magically be scaled to super-human speeds. For all we know cognition happens at a certain speed and any attempts past that are troublesome. Look at how easily we suffer from mania and other issues when our natural limiters go haywire. Singularity prediction is almost a cargo cult. Its weird that so many take it seriously.

jp555 12y ago
We are currently in the midst of a singularity of tools. Stone > Bronze > Steel > Silicon > Software > It's the "geek rapture" version that's more fantastic and extremely unlikely.
- _archon_ 12y ago
  This is a very interesting perspective. What do you think is the next step? I am asking for an educated guess, or even interested speculation; people in the middle of the bronze age likely didn't know that steel was the next step until after iron was extracted for the first time. If I had to guess, I would put the next phase at "integration" or "control." In that once we have functional, reliable libraries and virtual instruments, the logical next step is to integrate the high-level control of such into our physiologies. Alternatively, I could see "space" being an age we expand into, drawing our resources increasingly from space and broadening the scope of our potential that way. I'd like to see these happen in parallel if possible.

VLM 12y ago

Never overlook the power of organizational CYA. Likely the next announcement will be they've discovered automated sarcasm analysis of twitter feeds is, in fact, as suspected, completely impossible. Then announcement after that one will be a policy change to cease to analyze twitter feeds because its impossible to triage fast enough to keep up and its all noise anyway. Then the announcement after something tragic happens will be a steaming pile of CYA about how the worlds best minds couldn't find a way to automate the detection of XYZ but if our budget were only a little larger ... (never let a crisis go to waste...) This is beginning to read like a Stross book plot and maybe he's pissed because yet again he's scooped where the world is getting weirder faster than his plots are getting weird, like happened with the halting state series (as I recall). The laundry series is still safe from this danger, at least as far as we know...

3am 12y ago

This article rubbed me the wrong way. The Secret Service is trying to improve their methods in an open, competitive way. Maybe they don't have to perfectly solve sarcasm recognition to reduce their false positive rates. And let me add that anyone that's stupid enough to try to troll/game this system deserves all the consequences of their actions. edit: Remembering Bayes: Given a threat on someone the SS is charged w/ protecting, what is the probability it is not actionable?

mmmooo 12y ago

Anyone care to guesstimate how many assassination plots could have been prevented if just read the assailants public twitter feed in which he or she outlined, in plain english, his/her intentions to perform said act? Somehow I feel like the people that are serious about this level of criminal activity, and capable of actually accomplishing it, aren't first announcing their intentions on twitter.

deleted 12y ago
[deleted]
VLM 12y ago
Thought experiment: As a culture we've been pretty effective lately at electing miserable disappointing failures to public office. This seems a relatively uncontroversial observation, and note I'm careful not to name any names so as to not appear biased toward one side or the other of the one coin we've been dealt. Precondition 2 is everyone has social media accounts where they say stuff no one cares about, constantly. The CB radio of this decade. Precondition 3 is anyone planning an assassination is smart enough to self censor and say nothing about the entire topic. Given the above preconditions, the only rational conclusion is 99.9% of the population will continually be whining and complaining, and you can focus your attentions on the 0.1% who are at least considering doing something bad enough to self censor. So the thought experiment is Obama comes to visit a city and the guy who gets a SS visit is the only guy in the whole city who's not rambling on in social media about "Kenya" or "Can't believe I wasted my vote on him" or writing racial slurs or "he broke every campaign promise" or whatever.
- cstross 12y ago
  Disagree. As a culture we have developed public offices that are structured to prevent anyone who might be elected with a mandate to effect change from doing so. The iron law of bureaucracy applies: after the first generation most of the staff of any organization see their job not as pursuing the mission statement of the org but as preserving their own jobs. Attempts to change things generate resistance from within because, hey, jobs might be threatened. Hell, political candidates who might challenge their party's ability to win future elections by accomplishing change (which might be unwelcome to some elements of the voting -- or election-buying -- public or oligaarchs) are weeded out before they get a chance to run for office. The "disappointing" incumbent is merely a printed-paper face on the front of a machine. Which some loons choose to use for target practice. Resulting in the existence of a vast reactionary bureaucracy dedicated to extirpating threats to printed-paper faces.

kator 12y ago

My favorite NLP/Computer Language problem: Moe: Hey is Curly back from Vacation yet? Larry: I saw a Red Lamborghini in the parking lot. Moe: Cool We all assume Curly now drives a Red Lamborghini meanwhile most computer language systems would be lost at the inference here. Children learn this trick early computers are very challenged to understand these sorts of language challenges.

nkuttler 12y ago
I don't think the main challenge is to teach this "trick" to computers. The problem is probably to give them the proper context. If the system knows Curly drives a red lambo, and that it is a rare car in this area, and that Curly usually only parks in the parking lot when he's around, and etc. etc. etc.
- mcguire 12y ago
  Are you by any chance channelling Doug Lennat?
- nmc 12y ago
  > "If the system knows [...]" He was outlining that the difference is: humans do not need the context. I do not know who Curly is, and I still got what car he drives just by reading the three-line dialog.
  - click170 12y ago
    Except that you already know what a red lambo is, and that its safe to assume its a rare car almost anywhere, et al. You know this from experience, as do I and most other people. I think that was the parents point.
  - sosborn 12y ago
    >I still got what car he drives just by reading the three-line dialog. I inferred that too, but it doesn't mean that it is correct.
- throwaway283719 12y ago

quantail 12y ago

We'll bring the pony and the stick.

eli_gottlieb 12y ago
I was going to say, if you buy the right figurine, you can give the client a moon-pony.

mcv 12y ago

Nice of him to provide some test data.

snowwrestler 12y ago

HN (and similar tech-oriented forums) often host outraged discussions of government over-reactions. They also host excited discussions of linguistic technologies like Siri or Watson. Here we have a government agency attempting to push the envelope of technology in hopes of reducing their over-reactions. Seems like the sort of thing that would be greeted warmly but, nope, apparently not.

deleted 12y ago
[deleted]
javajosh 12y ago
Ah, but therein lies the (too often hidden) assumption behind the angst: we don't want law-enforcement to scale. It should take a human to destroy a human life. It should take a human to violate another human's privacy. Government, with it's monopoly on force, has the capability of doing these things at it's whim, but we are fundamentally opposed to treating these actions as "productivity" and the use of computers as "productivity enhancements" in those contexts. So, while the minor impulse (reduce overreach) is right, the major impulse (automated justice) is still very wrong.

pjc50 12y ago

For use of language to circumvent government filtering, see http://en.wikipedia.org/wiki/Baidu_10_Mythical_Creatures_%28Internet_meme%29 http://en.wikipedia.org/wiki/Baidu_10_Mythical_Creatures_%28... (I suppose that's timely with the Tianamen Square anniversary)

madaxe_again 12y ago

What I want to know is how Charlie knows the subject line of all my client emails ever.

milliams 12y ago

This wonderful comic comes to mind every time the discussion of sarcasm comes up http://www.qwantz.com/index.php?comic=168 http://www.qwantz.com/index.php?comic=168

flatline 12y ago

I think the most salient point was further down in a comment: > ...the real target is the autonomous exercise of professional judgement.

js2 12y ago

I come from a family with a dry sort of sarcasm that we deploy often. My sister and I were having a conversation the other day when it slipped into sarcasm and we proceeded to go back and forth till one of us said something serious. Or so the other thought. Eventually neither of us could tell. My daughter (13) observing this suddenly chimed in: "I think you're caught in a sarcasm trap." She claims to have invented the term on the spot. My 13 year old knows it's sometimes not possible for humans to detect sarcasm, much less a computer program. It's a shame that the secret service doesn't too. (I would venture to say that more people struggle with sarcasm than not. It often requires shared culture/experience to detect if it's not delivered in an obvious tone.)

mcguire 12y ago
The ability to reliably identify sarcasm, with a low false-positive rate but even with a very high false-negative rate, would likely save the Secret Service a significant amount of money and a very significant amount of bad press. Also, your daughter is much more perceptive than the vast majority of the people I know.

louthy 12y ago

Origin of the phrase: Moon on a stick "phrase used widely by teenagers and young adults in the mid 90s as it was made popular by the comedy geniuses Stew Lee & Richard Herring. they used it mostly in the second series of their hit tv & radio show 'Fist of Fun'.[1] It means basically to want everything, if you want the moon on a stick then you want everything, including things you can't have."[2] [1] https://www.youtube.com/watch?v=ERDUbAv8Qz0 https://www.youtube.com/watch?v=ERDUbAv8Qz0 [2] http://www.urbandictionary.com/define.php?term=moon%20on%20a%20stick http://www.urbandictionary.com/define.php?term=moon%20on%20a...

Brakenshire 12y ago
If you like this sort of thing, you might want to try out Stewart Lee's Comedy Vehicle: http://www.youtube.com/watch?v=-yUDh_IErT4 http://www.youtube.com/watch?v=-yUDh_IErT4

restless 12y ago

In case nobody mentioned The Simpsons yet "Oh a sarcasm detector that's a really useful invention"Comic Book Guy http://www.youtube.com/watch?v=mSy5mEcmgwU http://www.youtube.com/watch?v=mSy5mEcmgwU

dcalacci 12y ago

shamelessly promoting work by a colleague of mine that detects sarcasm in online product reviews: http://www.cs.huji.ac.il/~oren/papers/sarcasmAmazonICWSM10.pdf http://www.cs.huji.ac.il/~oren/papers/sarcasmAmazonICWSM10.p...

e12e 12y ago

> Or they could just ban sarcasm on the internet. I think what we're all half-expecting, is that they'll define the output of the program to be true; thereby a) ensuring the program is 100% correct, and b) In the event of real threats (as identified by the program) have actionable intelligence. Almost, but not quite, the same as banning sarcasm on the Internet.

kingtoe 12y ago

Summary: "No system to monitor or control the population will work because the entire population would obsess over gaming the system." Except no one cares enough to game it, or ever would. Sigh Another programmer who thinks that his buddies who obsess over gaming systems represent more than .001% of the population in question. Next.

davidgerard 12y ago

Favourite comment: "I went to a presentation from a sentiment analysis business last year. They were discussing how they were using it on the firehose feed of everything from twitter. Rather than assassinations, they were more interested in marketing, branding, and the impact on stock prices (there are apparently some iiiiiiinteresting correlations). The thing that entertained me was they said they had to special-case content coming from the UK — because our levels of sarcasm/irony were skewing their results. I think it was the most patriotic moment of my life <sniff> ;-)"