5 ms·

"The goal of the project is to make text hard to read for computers yet fairly easy to read for humans" At first look it doesn't pass the search engine test. h

"The goal of the project is to make text hard to read for computers yet fairly easy to read for humans" At first look it doesn't pass the search engine test. https://duckduckgo.com/?q=I%27ve+bien+aloune+whyth+jou+eenside+mi+mynd&t=ffab&ia=web https://duckduckgo.com/?q=I%27ve+bien+aloune+whyth+jou+eensi...

alex_duf 9y ago
That's actually very impressive!
cies 9y ago
Good test! No this is one way: you know the obfuscated. What if you want to use it in the opposite direction? Let's call it the CIA/NSA direction, where you know what unobfuscated phrase you want to find, but not how someone may have obfuscated it. This is arguably much harder, especially if you do not store some sort of representation of the content "as sounds/phonemes".
- saurik 9y ago
  The point is that if you are the CIA, searching through a bunch of obfuscated text, just like you are already (hopefully) already compensating for basic spelling errors, it apparently would not be very difficult to expand the scope of that same mechanism (in the same way that Duck Duck Go has) to support searching through and understanding this obfuscated text.
  - richmarr 9y ago
    Absolutely. Something I've had in the back of my mind for a while is the idea of swapping out a regular Lucene tokenisation & analysis for one that treats phonemes as tokens instead of (stemmed/analysed) words being the tokens... with a similar arrangement on the query side. I think it'd have interesting capabilities... this being one.
  - deleted 9y ago
    [deleted]
- deleted 9y ago
  [deleted]
deanclatworthy 9y ago
I suspect there is some part of their algorithm using levenshtein distance behind the scenes (or something similar). Spellfucker would not help with this.
- jheriko 9y ago
  actually it does deal with this particular metric somewhat in a bunch of cases. e.g. replacing one letter with three that sound similar but are different adds 4 to the distance - which is quite a significant distance already.
  - cknoxrun 9y ago
    So perhaps an algorithm that combines soundex with Levenshtein distance would handle this?
microcolonel 9y ago
Actually, it fails on Google. DuckDuckGo just apparently has an excellent spelling correction system.
- ryfm 9y ago
  i think it uses bing's spell correction.
plcancel 9y ago
Obscuring "I've" sneaks Lionel past the Duck. https://duckduckgo.com/?q=eyeve+bien+aloune+whyth+jou+eenside+mi+mynd&t=ffab&ia=lyrics https://duckduckgo.com/?q=eyeve+bien+aloune+whyth+jou+eensid...
- cryptarch 9y ago
  Check the 3rd result, it's the lyrics.
  - odabaxok 9y ago
    Sorry this might be OT, but I found it interesting that the 3rd result for you is the lyrics and for me it is something completely different (Lionel is just the tenth result). I thought Duckduckgo returns the same results for everyone and that the results are not personalized. Does someone know what kind of personalization does Duckduckgo use? In settings, region and language can be selected, but I have it set to 'All Region' and 'Browser preferred' which is en-US. Does Duckduckgo after all track me and this way customizes the results? Or am I missing something here?
    - jakobegger 9y ago
      It's probably just the region/language settings. Set region to Austria, and Ritchie is result nr 1. set Region to "All Regions" and he's result nr 10.
      - odabaxok 9y ago
        That seems reassuring. I think I have panicked too soon.
    - plcancel 9y ago
      Interesting. Yes, number 10 for me (All regions; Browser preferred lang.) The Instant Answer (IA) is lyrics to "Inside Out" by Eve 6.
  - plcancel 9y ago
bussie 9y ago
"The goal of the project is to make text hard to read for computers yet fairly easy to read for humans" What's the advantage to this?
- majewsky 9y ago
  Maybe provide a challenge for NLP researchers?
- keithly 9y ago
  Spam comes to mind.
  - emodendroket 9y ago
    I don't think anybody would have the patience for a spam message that had gone through this.
- emodendroket 9y ago
  Defeating censorship, frustrating accounts to analyze or index communications, etc.
  - tarboreus 9y ago
    Spamming people and making it impossible for blind people to use the internet?

alex_duf 9y ago

That's actually very impressive!

cies 9y ago

Good test! No this is one way: you know the obfuscated. What if you want to use it in the opposite direction? Let's call it the CIA/NSA direction, where you know what unobfuscated phrase you want to find, but not how someone may have obfuscated it. This is arguably much harder, especially if you do not store some sort of representation of the content "as sounds/phonemes".

saurik 9y ago
The point is that if you are the CIA, searching through a bunch of obfuscated text, just like you are already (hopefully) already compensating for basic spelling errors, it apparently would not be very difficult to expand the scope of that same mechanism (in the same way that Duck Duck Go has) to support searching through and understanding this obfuscated text.
- richmarr 9y ago
  Absolutely. Something I've had in the back of my mind for a while is the idea of swapping out a regular Lucene tokenisation & analysis for one that treats phonemes as tokens instead of (stemmed/analysed) words being the tokens... with a similar arrangement on the query side. I think it'd have interesting capabilities... this being one.
- deleted 9y ago
  [deleted]
deleted 9y ago
[deleted]

deanclatworthy 9y ago

I suspect there is some part of their algorithm using levenshtein distance behind the scenes (or something similar). Spellfucker would not help with this.

jheriko 9y ago
actually it does deal with this particular metric somewhat in a bunch of cases. e.g. replacing one letter with three that sound similar but are different adds 4 to the distance - which is quite a significant distance already.
- cknoxrun 9y ago
  So perhaps an algorithm that combines soundex with Levenshtein distance would handle this?

microcolonel 9y ago

Actually, it fails on Google. DuckDuckGo just apparently has an excellent spelling correction system.

ryfm 9y ago
i think it uses bing's spell correction.

plcancel 9y ago

Obscuring "I've" sneaks Lionel past the Duck. https://duckduckgo.com/?q=eyeve+bien+aloune+whyth+jou+eenside+mi+mynd&t=ffab&ia=lyrics https://duckduckgo.com/?q=eyeve+bien+aloune+whyth+jou+eensid...

cryptarch 9y ago
Check the 3rd result, it's the lyrics.
- odabaxok 9y ago
  Sorry this might be OT, but I found it interesting that the 3rd result for you is the lyrics and for me it is something completely different (Lionel is just the tenth result). I thought Duckduckgo returns the same results for everyone and that the results are not personalized. Does someone know what kind of personalization does Duckduckgo use? In settings, region and language can be selected, but I have it set to 'All Region' and 'Browser preferred' which is en-US. Does Duckduckgo after all track me and this way customizes the results? Or am I missing something here?
  - jakobegger 9y ago
    It's probably just the region/language settings. Set region to Austria, and Ritchie is result nr 1. set Region to "All Regions" and he's result nr 10.
    - odabaxok 9y ago
      That seems reassuring. I think I have panicked too soon.
  - plcancel 9y ago
    Interesting. Yes, number 10 for me (All regions; Browser preferred lang.) The Instant Answer (IA) is lyrics to "Inside Out" by Eve 6.
- plcancel 9y ago

bussie 9y ago

"The goal of the project is to make text hard to read for computers yet fairly easy to read for humans" What's the advantage to this?

majewsky 9y ago
Maybe provide a challenge for NLP researchers?
keithly 9y ago
Spam comes to mind.
- emodendroket 9y ago
  I don't think anybody would have the patience for a spam message that had gone through this.
emodendroket 9y ago
Defeating censorship, frustrating accounts to analyze or index communications, etc.
- tarboreus 9y ago
  Spamming people and making it impossible for blind people to use the internet?