15 ms·

DNS over Wikipedia

423 points by pyinstallwoes 4y ago

galaxyLogic 4y ago
Bad information is like viruses. Right?. So what we need is an immune-system. Bad information will still be there but it is prevented (to an extent) from multiplying. What would an immune-system for online information look like?
- dontbenebby 4y ago
  How do you define “bad” information? I found a list of CSAM sites via a link in a Wikipedia book about onion services- having a known evil target to practice getting a dot onion to reveal its ip was both good and bad data, depending on who you ask. (The CFAA has since expired, and those rude little pedophiles DEFINITELY noticed me.)
  - galaxyLogic 4y ago
    Bad information is like viruses. They look like legitimate parts of a biological system but in the end are dangerous to its health. Note that "wrong" information is not bad as such, it is just wrong. It can be corrected. It becomes bad when its purpose is to mislead people, in which case its originators will never want to correct it.
- meindnoch 4y ago
  https://en.m.wikipedia.org/wiki/Web_of_trust https://en.m.wikipedia.org/wiki/Web_of_trust
smsm42 4y ago
Using Wikidata [1] (or Wikidata SPARQL Query engine [2]) may be easier for automated tools. [1] https://www.wikidata.org/wiki/Q21980377#P856 https://www.wikidata.org/wiki/Q21980377#P856 [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial
billpg 4y ago
Is it actually DNS, or is there a custom name-to-IP lookup process going on? Also, am I just needlessly splitting hairs?
- TOGoS 4y ago
  > Also, am I just needlessly splitting hairs? Since the linked README doesn't even mention "IP", I suspect the author misunderstands what "DNS" is about, and this is actually a search tool, and the headline is bad.
  - Joker_vD 4y ago
    DNS is for associating various information with domain names, and looking that information up. Yes, the most popular information to associate with a domain name is IPv4- and IPv6-addresses via A and AAAA fields; but CNAME records exist as well, you know.
    - notpushkin 4y ago
      This is not even CNAME, more like a Redirect header in DNS (if this was actually a thing, that is). Still, this is arguably a domain name system, just one that isn't compatible with the DNS as we know it.
  - pushedx 4y ago
    Many record types contain hostnames as values. So many that it isn’t even worth listing all of them here.
- Group_B 4y ago
  It's not actually DNS
- andyp-kw 4y ago
  Wikipedia is acting as a system for domain name retrieval.
  - 4y ago
throwoutway 4y ago
> Instead of googling for the site, I google for the site's Wikipedia article ("schihub wiki") which usually has an up-to-date link to the site in the sidebar, whereas Google is forced to censor their results. In the video, it doesn’t show this. It shows going to the scihub.idk domain. And then a redirect happens. So does this tool just host a local a domain resolver (and HTTP redirect server) for all .idk domains that does a wiki search and then responds with a HTTP redirect?
- trynewideas 4y ago
  The comment is context for what inspired the tool. The functionality is about 40 lines of pretty readable Rust: https://github.com/aaronjanse/dns-over-wikipedia/blob/master/hosts-file/src/wikipedia.rs https://github.com/aaronjanse/dns-over-wikipedia/blob/master... 1. Make a Wikipedia search API request for the .idk domain, using the name as the article name. 2. Retrieve the rendered page contents if found. 3. Find the first Wikipedia infobox table on the page. 4. Extract the first "URL" or "Website" entry in that infobox. 5. Return the entry's value, if it's a link. All this runs in a nickel.rs server on 127.0.0.1:80, which routes the requests as permanent redirects to the destination. Using dnsmasq,[1] if it's an .idk domain, it routes the request through the above Wikipedia resolver. 1: https://github.com/aaronjanse/dns-over-wikipedia/tree/master/hosts-file https://github.com/aaronjanse/dns-over-wikipedia/tree/master...
  - derhuerst 4y ago
    The extension could also use Wikidata [1] entries – which (AFAIK almost always) hold the data that is displayed in Wikipedia article's infobox – because then it wouldn't have to resort to parsing HTML. Specifically, Wikidata has a "official website" property [2] that seems to be used. If there are multiple extensions, like in Sci-Hub's case [3], it could pick one based on user preferences. [1] https://www.wikidata.org/ https://www.wikidata.org/ [2] https://www.wikidata.org/wiki/Property:P856 https://www.wikidata.org/wiki/Property:P856 [3] https://www.wikidata.org/wiki/Q21980377 https://www.wikidata.org/wiki/Q21980377
gumby 4y ago
This is actually better than the title suggests. I thought it might be something like automated edits to Wikipedia talk pages as a database or cache, which would be abusive. But actually it's using Wikipedia entries as a smart name search. This is read-only and a clever and legit use of the WP. If the browsers allowed the use of spaces in the domain name portion URLs it would be even more useful, but it's probably better that they don't (out of spec for TLDs)
- pyinstallwoes 4y ago
  Exactly. Almost gives credence to the idea of a well-behaved decentralized peer-to-peer network.
- cuttysnark 4y ago
  On the other hand, Wikipedia pages seemingly get "vandalized" all the time and sometimes aren't corrected immediately. Doesn't this create a situation where a bad actor could change the Wikipedia page for a `semi-popular-brand.com` url listing to something bad? Anyone who used `semi-popular-brand.idk` in that timeframe would land in the bad page. Perhaps I'm misunderstanding.
  - gumby 4y ago
    This is possible with the DNS already which is why there have been various efforts to try to harden the protocol, to varying effect so far.
    - MivLives 4y ago
      Sure but that requires knowledge of DNS, and a bad edit on wikipedia is much more accessible.
  - imiric 4y ago
    Are you saying it's not a good idea to rely on domain lookup from a public wiki? I think you might be onto something...
  - 4y ago
camhart 4y ago
What's the use case? Or just a fun programming task? Wouldn't this hammer wikipedia unnecessarily? DNS are very quick, lightweight requests.
- davrosthedalek 4y ago
  This is not doing DNS. This is "domain name autocomplete".
  - Joker_vD 4y ago
    It's using Wikipedia as a provider for CNAME records. Does it count as DNS?
    - fundad 4y ago
      No because using this also requires DNS
- michaelt 4y ago
  If you want to visit a site like 'The Pirate Bay' or 'Sci-Hub' by name, Wikipedia provides the URL, whereas google doesn't.
- deelowe 4y ago
  Use cases are given on the github page. It's useful for sites that aren't indexed by Google and for avoiding spam.
amadeuspagel 4y ago
I thought the chrome extension didn't work at first, but you have to type "https://scihub.idk https://scihub.idk" rather then just "scihub.idk", which leads to a google search.
- Joker_vD 4y ago
  I think you can also do "scihub.idk/", the trailing slash forces Chrome to treat it as an URL.
- deleted 4y ago
  [deleted]
Kiboneu 4y ago
It redirects to URL entries in the infobox (present in many wiki pages) associated with the body of your entered URL (ending with ‘.idk’). What it does not do is prevent anyone from temporarily editing the associated wikipedia page and phishing you into an attacker-controlled website.
- cxr 4y ago
  Not sure when this project first began exactly, but the Show HN for it is from 2020 <https://news.ycombinator.com/item?id=22790425 https://news.ycombinator.com/item?id=22790425>, and various things suggest that timeframe for its genesis. The same concern was raised by the user "segfaultbuserr" in that Show HN thread[1]. In 2019 I wrote up this very idea as part of a brain dump of long-stalled thoughts that I had been kicking around but not published anywhere. There are mitigations to the attack you describe, which I included in my original terse writeup about such a "Wikipedia Name System"[2]. I pasted the explanation in a comment in the original Show HN thread. The idea lies in the fact that although wikis can be edited to point to your own fake honeytrap, there are things that can't be faked; it's a public ledger sort of thought exercise in the vein of Bitcoin—but no need for proof of work or what is currently associated with cryptocurrency, etc. As I (re-)explained three years ago: > Not as trivially compromised as it sounds like it would be; could be faked with (inevitably short-lived) edits, but temporality can't be faked. If a system were rolled out tomorrow, nothing that happens after rollout [...] would alter the fact that for the last N years, Wikipedia has understood that the website for Facebook is facebook.com. Newly created, low-traffic articles and short-lived edits would fail the trust threshold. After rollout, there would be increased attention to make sure that longstanding edits getting in that misrepresent the link between domain and identity [can never reach maturity]. Would-be attackers would be discouraged to the point of not even trying. (This also provides mitigation for what is (currently) the top comment in this thread—the issue of Wikipedians censoring certain domains, provided that they have a long enough history to meet the trust threshold before they were decided to be too contentious to share information about them.) 1. See <https://news.ycombinator.com/item?id=22791534 https://news.ycombinator.com/item?id=22791534> 2. Originally published, along with other thoughts from that month, at <https://www.colbyrussell.com/2019/05/15/may-integration.html#wikipedia-name-system https://www.colbyrussell.com/2019/05/15/may-integration.html...>
account-5 4y ago
I always do this, just manually.
ilyt 4y ago
Please don't abuse public good infrastructure
- thepangolino 4y ago
  How's that abusing public infrastructure? 1. Wikipedia is a private entity. 2. It is very well funded. 3. That extension probably uses less resources than someone loading an entire webpage with all associated media just to look up a url.
- Somatochlora 4y ago
  In what way is this abuse?
  - GNOMES 4y ago
    Depending if this became "popular", could overload them with traffic. I presume this is a fun test project though, so I doubt it would cause much harm, but it could be there.
    - rippercushions 4y ago
      It's hitting the Wikipedia API, which I can assure you is used to seeing a lot worse.
    - Joker_vD 4y ago
      Overload wikipedia with traffic? As opposed to manually opening a Wikipedia page and copypasting the URL from it?
deleted 4y ago
[deleted]
some_random 4y ago
I love this so much, although it sucks that this is actually useful. ITT: people who didn't actually read what the project does
mmahemoff 4y ago
If the point is to get to the site faster, why is the TLD "idk"? Couldn't it be 1 letter or at least the same three letters? On a separate note, someone could actually make a website do this without any extension. It would need to use wikipedia dumps and has the advantage it can't be suddenly edited by a malicious actor.
- priteau 4y ago
  I assume idk means I don’t know. As in, get me to this website for which I don’t know the domain name.
- aew4ytasghe5 4y ago
  > On a separate note, someone could actually make a website do this without any extension. It would need to use wikipedia dumps and has the advantage it can't be suddenly edited by a malicious actor. Why would we trust "a website" more than random wikipedia edits to provide correct or up-to-date data? The purpose of this app is to get the current url of those sites, which keeps changing url semi-frequently, so a offline copy of wikipedia would not work.
jmbwell 4y ago
This reminds me of the old "AOL Keywords" system in its later stages, when it was just a shortcode for a web site.
voxic11 4y ago
Wikipedia editors have noticed that people use wikipedia for this purpose (finding the current location of sites) and so they have started to censor links for sites they don't want to encourage people to access. Right now this mostly impacts sites few would condone, like 8chan or kiwifarms. But in theory the policy applies to any site with illegal material so places like the pirate bay or scihub could have their links removed at any time. https://en.wikipedia.org/wiki/Talk:Kiwi_Farms#URL https://en.wikipedia.org/wiki/Talk:Kiwi_Farms#URL
- kevmo314 4y ago
  Coming soon: DNS over ChatGPT
- _def 4y ago
  I had never heard of kiwi farms before. Disgusting...
- charlieyu1 4y ago
  Well I can see links about Hong Kong being censored then. I mean anything about Hong Kong is censored already.
- computerfriend 4y ago
  Here's a well-reasoned argument linked from that thread for keeping extremist URL in articles about extremist websites. Strange what can change in just a couple of years. https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(policy)/Archive_164#Extremist_groups_and_their_URL%27s https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(poli...
  - ranger_danger 4y ago
    Got a TL;DR ?
  - matheusmoreira 4y ago
    Ah, the ever expanding definition of extremism. By now it already means anything that isn't politically correct. Organize terrorist attacks? I bet it's just protests they don't like. Lots of protests going on in my country right now and it's really tiresome to watch the media call them terrorists. It's like they all use the same arguments to silence and censor people.
tjpnz 4y ago
Keep in mind that Wikipedia has a habit of censoring the infobox URL for sites their editors don't like.
- psychphysic 4y ago
  You say that like the content and topic choice isn't usually heavily biased!
kikokikokiko 4y ago
We live in a distopia where one of the last search engines that returns real results related to anything the western mainstream "hidden forces" do not want you to be able to see is Yandex, a company located under a dictatorship. In 2022, the only way for a regular citizen to approximate what is reality, what is the "truth", is to look for information provided from the opposite side of any argument you're trying to understand more of. Everything is fake or hidden from you. It's the stable diffusion distopia.
- ajross 4y ago
  - deleted 4y ago
    [deleted]
  - Cyberdog 4y ago
    > How about this: try posting an URL for kiwifarms or stormfront or whatever right here, in this thread. See if it stays up. See who's willing to tolerate it. Man, I almost did. But people just get so irrationally angry about words on the internet nowadays that I decided against tempting fate. I don't need people trying to destroy my career because I don't irrationally hate the right people right now. Yes, Kiwi Farms exists and has information and opinions you might not like. It certainly has ones that I don't like. But at the end of the day, it's just words on the internet; the stuff about "organized harassment campaigns" and all that is a media lie. I really wish people would just visit the place and realize that it's just a bunch of shitposters with a clear "look, laugh, but don't touch" mentality and not the demonic bogeyman the press has made it out be. But I realize that irrational hatred is faster and easier, so…
    - dale_glass 4y ago
      > I don't need people trying to destroy my career because I don't irrationally hate the right people right now. You don't "irrationally" hate them probably because you happen not to fit the target profile. EDIT: And are you serious? You truly think that KiwiFarms is a quirky, harmless place that just collects info to do nothing with it, and at the same time believe that on HN you'd be targeted by people who'd try to ruin your life? What force keeps such nasty people off kiwifarms but allows their presence here, I wonder? > Yes, Kiwi Farms exists and has information and opinions you might not like. It certainly has ones that I don't like. But at the end of the day, it's just words on the internet; There's no such thing as "just words on the internet". Words express beliefs and intent. I take them seriously. > the stuff about "organized harassment campaigns" and all that is a media lie. Of course. The sort of people that compile elaborate profiles on people including personal information are some sort of enlightened beings that despite obsessively tracking down and compiling every embarrassing or weird thing about another person would never actually use that information for anything at all.
nashashmi 4y ago
DNS is a good name to help understand what it does but I am perturbed at the misappropriation. If it ramifies to calling everything DNS, search for real dns services will be cluttered with irrelevant results. This a redirect to the right domain. It is not an IP address lookup. This could have just as well been an “I feel lucky” search box using Wikipedia as the source. Or a Duck !bang.
super256 4y ago
> If you Google "Piratebay", the first search result is a fake "thepirate-bay.org" (with a dash) but the Wikipedia article lists the right one. — shpx I'm laughing in pain. I detest google so much for being so bad at its only job - finding stuff - merely to maximize $$$. Google was such a great product and genuinely serving humanity. Now it's only a shell of its former self.
- Spivak 4y ago
  Google’s approach is not, in general, capable of handling adversarial inputs. This isn’t a Google problem as much as Google is currently losing the SEO war.
- 6chars 4y ago
  Google is the worst search engine besides every other one
  - 6chars 4y ago
    (In my experience at least. I'm open to recommendations!)
    - super256 4y ago
      Amazon product search is worse ;)
      - aryamaan 4y ago
        that is what OP meant that Google is the worst and every thing else is worse.
Cyberdog 4y ago
Once upon a time I built an iOS language translation/dictionary app which used Wikipedia as a back end. Basically, if you entered a word in language X and the Wikipedia version for language X had an entry, it could find the translation for that word in language Y by looking at the "this article in other languages" information in the source for that page. It was useful for finding translations or transliterations of terms that wouldn't commonly appear in standard dictionaries, like neologisms, brand names, or even just really obscure topics. Unfortunately it eventually got taken down from the App Store because Wikipedia's asshole lawyers didn't like that I used a serifed W in the logo even though I made it clear it wasn't an official Wikipedia product everywhere and I didn't have the motivation to "correct" my audacious use of an upper-case W which Wikipedia now apparently holds the copyrights to. I still use this technique "manually" for finding translations when dictionaries, though. Wikipedia has a lot of information which isn't in a typical API-friendly format, but with a bit of regex there's some interesting possibilities. If this sounds interesting, the Objective-C source is still available here: https://github.com/garrettalbright/wptrans https://github.com/garrettalbright/wptrans You might also be able to find clones of it in the App Store since I know people were taking my source and republishing it back in the day, some even for a profit (mine was always free).
- teddyh 4y ago
  Your app was probably taken down because you apparently don’t understand what a trademark is, or what it is for. > Wikipedia's asshole lawyers didn't like that I used a serifed W in the logo It’s not the lawyers which don’t like it. Trademark exists to protect consumer confusion. If a reasonable person could reasonably (but incorrectly) believe that your app was officially associated with Wikipedia based on only looking at the logo, then you have created consumer confusion. Trademark law disallows you from (using trademarked branding) letting people believe that your app is official. > I made it clear it wasn't an official Wikipedia product everywhere Posting “No copyright infringement intended” on a YouTube video does not, contrary to masses of people, turn a copyright infringement into something else. Likewise, just writing that your app is not official will not make a trademark infringement vanish. (Of course, if Wikipedia does not have a valid trademark which you infringed, that’s another story. But it’s not what I would guess has happened here.)
kfiven 4y ago
Why Wikipedia though? Wikidata should be one to get this kind of data and all Wikipedia articles has Wikidata items. On top of that Wikidata even keep records of site name, like when someone change their site name.
- ranger_danger 4y ago
  Literally never heard of wikidata, and I doubt most other people have either.
- dontbenebby 4y ago
  I think a lot of people conflate Wikipedia with the larger Wikimedia organization. https://www.wikimedia.org/ https://www.wikimedia.org/
- xd1936 4y ago
  I suppose the thought process is, more eyeballs are on corresponding Wikipedia pages for entities than are on Wikidata entries, so it's more likely to be corrected/up-to-date.
  - kfiven 4y ago
    In many type of Infobox on Wikipedia, it's Wikidata which is working from the behind and editing the data in such articles is basically being done at Wikidata.
alpos 4y ago
What are you going to do if people start to trust this and then someone edits the wiki sources to redirect people to phishing sites mocked up to look real?
- anenefan 4y ago
  This is a good point, this it something that has happened (I've personally witnessed) and probably still happens from time to time unless wikipedia has worked to stop that sort of abuse. Long story short, about ten years ago, my recommendation of a specific search tool (shareaza) to a friend went south, since it was over the phone. I'd found telling people to use web searches didn't work out well, wikipedia on the other hand, usually had an up to date address -- I no longer recall what trouble it caused, but I spent some time wondering what malware was redirecting to the bogus software site -- there was none, it was the edited entry at wikipedia. (When I tried the bogus software I recall it looked like it was doing something, but in fact it was doing nothing as far as searching p2p networks. Being busy I wasn't interested to see what its objectives were before deleting.)
lxgr 4y ago
This is neat, but I expected something completely different from the name. I already had my hopes up for doing VPN/IP over DNS over Wikipedia. Frustratingly, beyond not actually offering that (which is entirely reasonable), it does not even seem to be using DNS for the implementation of what it does.
9edda054-232f 4y ago
Why don't just buy a domain like DNSOver.Wiki so people can just scihub.dnsover.wiki without installing the browser extension..
- aryamaan 4y ago
  it's actually a clever idea
752963e64 4y ago
shp0ngle 4y ago
8kun and kiwifarms are both censored on wikipedia. Maybe that’s a good thing. But let’s not pretend it’s less censored than google!
Thorentis 4y ago
This isn't DNS (resolving IPs to FQDNs) as much as resolving DNS prefixes to their current suffixes. As others have pointed out, Wikipedia is starting to censor results just like Google. Maybe it would be better for this extension to pivot into providing their own service that performs this task.
pessimizer 4y ago
If this takes off, it's just going to result in a court order against Wikipedia. Z-lib getting popular on Tiktok = no z-lib anymore. Getting Wikipedia censored will ultimately make it harder for people to find sites like scihub.
- pyinstallwoes 4y ago
  Perhaps we should do the same idea on top of a DHT
SpaghettiX 4y ago
Is this HTTP redirect using Wikipedia, rather than DNS over Wikipedia?

423 points by pyinstallwoes 4y ago

Bad information is like viruses. Right?. So what we need is an immune-system. Bad information will still be there but it is prevented (to an extent) from multiplying. What would an immune-system for online information look like?

dontbenebby 4y ago
How do you define “bad” information? I found a list of CSAM sites via a link in a Wikipedia book about onion services- having a known evil target to practice getting a dot onion to reveal its ip was both good and bad data, depending on who you ask. (The CFAA has since expired, and those rude little pedophiles DEFINITELY noticed me.)
- galaxyLogic 4y ago
  Bad information is like viruses. They look like legitimate parts of a biological system but in the end are dangerous to its health. Note that "wrong" information is not bad as such, it is just wrong. It can be corrected. It becomes bad when its purpose is to mislead people, in which case its originators will never want to correct it.
meindnoch 4y ago
https://en.m.wikipedia.org/wiki/Web_of_trust https://en.m.wikipedia.org/wiki/Web_of_trust

smsm42 4y ago

Using Wikidata [1] (or Wikidata SPARQL Query engine [2]) may be easier for automated tools. [1] https://www.wikidata.org/wiki/Q21980377#P856 https://www.wikidata.org/wiki/Q21980377#P856 [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial

billpg 4y ago

Is it actually DNS, or is there a custom name-to-IP lookup process going on? Also, am I just needlessly splitting hairs?

TOGoS 4y ago
> Also, am I just needlessly splitting hairs? Since the linked README doesn't even mention "IP", I suspect the author misunderstands what "DNS" is about, and this is actually a search tool, and the headline is bad.
- Joker_vD 4y ago
  DNS is for associating various information with domain names, and looking that information up. Yes, the most popular information to associate with a domain name is IPv4- and IPv6-addresses via A and AAAA fields; but CNAME records exist as well, you know.
  - notpushkin 4y ago
    This is not even CNAME, more like a Redirect header in DNS (if this was actually a thing, that is). Still, this is arguably a domain name system, just one that isn't compatible with the DNS as we know it.
- pushedx 4y ago
  Many record types contain hostnames as values. So many that it isn’t even worth listing all of them here.
Group_B 4y ago
It's not actually DNS
andyp-kw 4y ago
Wikipedia is acting as a system for domain name retrieval.
- 4y ago

throwoutway 4y ago

> Instead of googling for the site, I google for the site's Wikipedia article ("schihub wiki") which usually has an up-to-date link to the site in the sidebar, whereas Google is forced to censor their results. In the video, it doesn’t show this. It shows going to the scihub.idk domain. And then a redirect happens. So does this tool just host a local a domain resolver (and HTTP redirect server) for all .idk domains that does a wiki search and then responds with a HTTP redirect?

trynewideas 4y ago
The comment is context for what inspired the tool. The functionality is about 40 lines of pretty readable Rust: https://github.com/aaronjanse/dns-over-wikipedia/blob/master/hosts-file/src/wikipedia.rs https://github.com/aaronjanse/dns-over-wikipedia/blob/master... 1. Make a Wikipedia search API request for the .idk domain, using the name as the article name. 2. Retrieve the rendered page contents if found. 3. Find the first Wikipedia infobox table on the page. 4. Extract the first "URL" or "Website" entry in that infobox. 5. Return the entry's value, if it's a link. All this runs in a nickel.rs server on 127.0.0.1:80, which routes the requests as permanent redirects to the destination. Using dnsmasq,[1] if it's an .idk domain, it routes the request through the above Wikipedia resolver. 1: https://github.com/aaronjanse/dns-over-wikipedia/tree/master/hosts-file https://github.com/aaronjanse/dns-over-wikipedia/tree/master...
- derhuerst 4y ago
  The extension could also use Wikidata [1] entries – which (AFAIK almost always) hold the data that is displayed in Wikipedia article's infobox – because then it wouldn't have to resort to parsing HTML. Specifically, Wikidata has a "official website" property [2] that seems to be used. If there are multiple extensions, like in Sci-Hub's case [3], it could pick one based on user preferences. [1] https://www.wikidata.org/ https://www.wikidata.org/ [2] https://www.wikidata.org/wiki/Property:P856 https://www.wikidata.org/wiki/Property:P856 [3] https://www.wikidata.org/wiki/Q21980377 https://www.wikidata.org/wiki/Q21980377

gumby 4y ago

This is actually better than the title suggests. I thought it might be something like automated edits to Wikipedia talk pages as a database or cache, which would be abusive. But actually it's using Wikipedia entries as a smart name search. This is read-only and a clever and legit use of the WP. If the browsers allowed the use of spaces in the domain name portion URLs it would be even more useful, but it's probably better that they don't (out of spec for TLDs)

pyinstallwoes 4y ago
Exactly. Almost gives credence to the idea of a well-behaved decentralized peer-to-peer network.
cuttysnark 4y ago
On the other hand, Wikipedia pages seemingly get "vandalized" all the time and sometimes aren't corrected immediately. Doesn't this create a situation where a bad actor could change the Wikipedia page for a `semi-popular-brand.com` url listing to something bad? Anyone who used `semi-popular-brand.idk` in that timeframe would land in the bad page. Perhaps I'm misunderstanding.
- gumby 4y ago
  This is possible with the DNS already which is why there have been various efforts to try to harden the protocol, to varying effect so far.
  - MivLives 4y ago
    Sure but that requires knowledge of DNS, and a bad edit on wikipedia is much more accessible.
- imiric 4y ago
  Are you saying it's not a good idea to rely on domain lookup from a public wiki? I think you might be onto something...
- 4y ago

camhart 4y ago

What's the use case? Or just a fun programming task? Wouldn't this hammer wikipedia unnecessarily? DNS are very quick, lightweight requests.

davrosthedalek 4y ago
This is not doing DNS. This is "domain name autocomplete".
- Joker_vD 4y ago
  It's using Wikipedia as a provider for CNAME records. Does it count as DNS?
  - fundad 4y ago
    No because using this also requires DNS
michaelt 4y ago
If you want to visit a site like 'The Pirate Bay' or 'Sci-Hub' by name, Wikipedia provides the URL, whereas google doesn't.
deelowe 4y ago
Use cases are given on the github page. It's useful for sites that aren't indexed by Google and for avoiding spam.

amadeuspagel 4y ago

I thought the chrome extension didn't work at first, but you have to type "https://scihub.idk https://scihub.idk" rather then just "scihub.idk", which leads to a google search.

Joker_vD 4y ago
I think you can also do "scihub.idk/", the trailing slash forces Chrome to treat it as an URL.
deleted 4y ago
[deleted]

Kiboneu 4y ago

It redirects to URL entries in the infobox (present in many wiki pages) associated with the body of your entered URL (ending with ‘.idk’). What it does not do is prevent anyone from temporarily editing the associated wikipedia page and phishing you into an attacker-controlled website.

cxr 4y ago
Not sure when this project first began exactly, but the Show HN for it is from 2020 <https://news.ycombinator.com/item?id=22790425 https://news.ycombinator.com/item?id=22790425>, and various things suggest that timeframe for its genesis. The same concern was raised by the user "segfaultbuserr" in that Show HN thread[1]. In 2019 I wrote up this very idea as part of a brain dump of long-stalled thoughts that I had been kicking around but not published anywhere. There are mitigations to the attack you describe, which I included in my original terse writeup about such a "Wikipedia Name System"[2]. I pasted the explanation in a comment in the original Show HN thread. The idea lies in the fact that although wikis can be edited to point to your own fake honeytrap, there are things that can't be faked; it's a public ledger sort of thought exercise in the vein of Bitcoin—but no need for proof of work or what is currently associated with cryptocurrency, etc. As I (re-)explained three years ago: > Not as trivially compromised as it sounds like it would be; could be faked with (inevitably short-lived) edits, but temporality can't be faked. If a system were rolled out tomorrow, nothing that happens after rollout [...] would alter the fact that for the last N years, Wikipedia has understood that the website for Facebook is facebook.com. Newly created, low-traffic articles and short-lived edits would fail the trust threshold. After rollout, there would be increased attention to make sure that longstanding edits getting in that misrepresent the link between domain and identity [can never reach maturity]. Would-be attackers would be discouraged to the point of not even trying. (This also provides mitigation for what is (currently) the top comment in this thread—the issue of Wikipedians censoring certain domains, provided that they have a long enough history to meet the trust threshold before they were decided to be too contentious to share information about them.) 1. See <https://news.ycombinator.com/item?id=22791534 https://news.ycombinator.com/item?id=22791534> 2. Originally published, along with other thoughts from that month, at <https://www.colbyrussell.com/2019/05/15/may-integration.html#wikipedia-name-system https://www.colbyrussell.com/2019/05/15/may-integration.html...>

account-5 4y ago

I always do this, just manually.

ilyt 4y ago

Please don't abuse public good infrastructure

thepangolino 4y ago
How's that abusing public infrastructure? 1. Wikipedia is a private entity. 2. It is very well funded. 3. That extension probably uses less resources than someone loading an entire webpage with all associated media just to look up a url.
Somatochlora 4y ago
In what way is this abuse?
- GNOMES 4y ago
  Depending if this became "popular", could overload them with traffic. I presume this is a fun test project though, so I doubt it would cause much harm, but it could be there.
  - rippercushions 4y ago
    It's hitting the Wikipedia API, which I can assure you is used to seeing a lot worse.
  - Joker_vD 4y ago
    Overload wikipedia with traffic? As opposed to manually opening a Wikipedia page and copypasting the URL from it?

deleted 4y ago

[deleted]

some_random 4y ago

I love this so much, although it sucks that this is actually useful. ITT: people who didn't actually read what the project does

mmahemoff 4y ago

If the point is to get to the site faster, why is the TLD "idk"? Couldn't it be 1 letter or at least the same three letters? On a separate note, someone could actually make a website do this without any extension. It would need to use wikipedia dumps and has the advantage it can't be suddenly edited by a malicious actor.

priteau 4y ago
I assume idk means I don’t know. As in, get me to this website for which I don’t know the domain name.
aew4ytasghe5 4y ago
> On a separate note, someone could actually make a website do this without any extension. It would need to use wikipedia dumps and has the advantage it can't be suddenly edited by a malicious actor. Why would we trust "a website" more than random wikipedia edits to provide correct or up-to-date data? The purpose of this app is to get the current url of those sites, which keeps changing url semi-frequently, so a offline copy of wikipedia would not work.

jmbwell 4y ago

This reminds me of the old "AOL Keywords" system in its later stages, when it was just a shortcode for a web site.

voxic11 4y ago

Wikipedia editors have noticed that people use wikipedia for this purpose (finding the current location of sites) and so they have started to censor links for sites they don't want to encourage people to access. Right now this mostly impacts sites few would condone, like 8chan or kiwifarms. But in theory the policy applies to any site with illegal material so places like the pirate bay or scihub could have their links removed at any time. https://en.wikipedia.org/wiki/Talk:Kiwi_Farms#URL https://en.wikipedia.org/wiki/Talk:Kiwi_Farms#URL

kevmo314 4y ago
Coming soon: DNS over ChatGPT
_def 4y ago
I had never heard of kiwi farms before. Disgusting...
charlieyu1 4y ago
Well I can see links about Hong Kong being censored then. I mean anything about Hong Kong is censored already.
computerfriend 4y ago
Here's a well-reasoned argument linked from that thread for keeping extremist URL in articles about extremist websites. Strange what can change in just a couple of years. https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(policy)/Archive_164#Extremist_groups_and_their_URL%27s https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(poli...
- ranger_danger 4y ago
  Got a TL;DR ?
- matheusmoreira 4y ago
  Ah, the ever expanding definition of extremism. By now it already means anything that isn't politically correct. Organize terrorist attacks? I bet it's just protests they don't like. Lots of protests going on in my country right now and it's really tiresome to watch the media call them terrorists. It's like they all use the same arguments to silence and censor people.

tjpnz 4y ago

Keep in mind that Wikipedia has a habit of censoring the infobox URL for sites their editors don't like.

psychphysic 4y ago
You say that like the content and topic choice isn't usually heavily biased!

kikokikokiko 4y ago

We live in a distopia where one of the last search engines that returns real results related to anything the western mainstream "hidden forces" do not want you to be able to see is Yandex, a company located under a dictatorship. In 2022, the only way for a regular citizen to approximate what is reality, what is the "truth", is to look for information provided from the opposite side of any argument you're trying to understand more of. Everything is fake or hidden from you. It's the stable diffusion distopia.

ajross 4y ago
- deleted 4y ago
  [deleted]
- Cyberdog 4y ago
  > How about this: try posting an URL for kiwifarms or stormfront or whatever right here, in this thread. See if it stays up. See who's willing to tolerate it. Man, I almost did. But people just get so irrationally angry about words on the internet nowadays that I decided against tempting fate. I don't need people trying to destroy my career because I don't irrationally hate the right people right now. Yes, Kiwi Farms exists and has information and opinions you might not like. It certainly has ones that I don't like. But at the end of the day, it's just words on the internet; the stuff about "organized harassment campaigns" and all that is a media lie. I really wish people would just visit the place and realize that it's just a bunch of shitposters with a clear "look, laugh, but don't touch" mentality and not the demonic bogeyman the press has made it out be. But I realize that irrational hatred is faster and easier, so…
  - dale_glass 4y ago
    > I don't need people trying to destroy my career because I don't irrationally hate the right people right now. You don't "irrationally" hate them probably because you happen not to fit the target profile. EDIT: And are you serious? You truly think that KiwiFarms is a quirky, harmless place that just collects info to do nothing with it, and at the same time believe that on HN you'd be targeted by people who'd try to ruin your life? What force keeps such nasty people off kiwifarms but allows their presence here, I wonder? > Yes, Kiwi Farms exists and has information and opinions you might not like. It certainly has ones that I don't like. But at the end of the day, it's just words on the internet; There's no such thing as "just words on the internet". Words express beliefs and intent. I take them seriously. > the stuff about "organized harassment campaigns" and all that is a media lie. Of course. The sort of people that compile elaborate profiles on people including personal information are some sort of enlightened beings that despite obsessively tracking down and compiling every embarrassing or weird thing about another person would never actually use that information for anything at all.

nashashmi 4y ago

DNS is a good name to help understand what it does but I am perturbed at the misappropriation. If it ramifies to calling everything DNS, search for real dns services will be cluttered with irrelevant results. This a redirect to the right domain. It is not an IP address lookup. This could have just as well been an “I feel lucky” search box using Wikipedia as the source. Or a Duck !bang.

super256 4y ago

> If you Google "Piratebay", the first search result is a fake "thepirate-bay.org" (with a dash) but the Wikipedia article lists the right one. — shpx I'm laughing in pain. I detest google so much for being so bad at its only job - finding stuff - merely to maximize $$$. Google was such a great product and genuinely serving humanity. Now it's only a shell of its former self.

Spivak 4y ago
Google’s approach is not, in general, capable of handling adversarial inputs. This isn’t a Google problem as much as Google is currently losing the SEO war.
6chars 4y ago
Google is the worst search engine besides every other one
- 6chars 4y ago
  (In my experience at least. I'm open to recommendations!)
  - super256 4y ago
    Amazon product search is worse ;)
    - aryamaan 4y ago
      that is what OP meant that Google is the worst and every thing else is worse.

Cyberdog 4y ago

Once upon a time I built an iOS language translation/dictionary app which used Wikipedia as a back end. Basically, if you entered a word in language X and the Wikipedia version for language X had an entry, it could find the translation for that word in language Y by looking at the "this article in other languages" information in the source for that page. It was useful for finding translations or transliterations of terms that wouldn't commonly appear in standard dictionaries, like neologisms, brand names, or even just really obscure topics. Unfortunately it eventually got taken down from the App Store because Wikipedia's asshole lawyers didn't like that I used a serifed W in the logo even though I made it clear it wasn't an official Wikipedia product everywhere and I didn't have the motivation to "correct" my audacious use of an upper-case W which Wikipedia now apparently holds the copyrights to. I still use this technique "manually" for finding translations when dictionaries, though. Wikipedia has a lot of information which isn't in a typical API-friendly format, but with a bit of regex there's some interesting possibilities. If this sounds interesting, the Objective-C source is still available here: https://github.com/garrettalbright/wptrans https://github.com/garrettalbright/wptrans You might also be able to find clones of it in the App Store since I know people were taking my source and republishing it back in the day, some even for a profit (mine was always free).

teddyh 4y ago
Your app was probably taken down because you apparently don’t understand what a trademark is, or what it is for. > Wikipedia's asshole lawyers didn't like that I used a serifed W in the logo It’s not the lawyers which don’t like it. Trademark exists to protect consumer confusion. If a reasonable person could reasonably (but incorrectly) believe that your app was officially associated with Wikipedia based on only looking at the logo, then you have created consumer confusion. Trademark law disallows you from (using trademarked branding) letting people believe that your app is official. > I made it clear it wasn't an official Wikipedia product everywhere Posting “No copyright infringement intended” on a YouTube video does not, contrary to masses of people, turn a copyright infringement into something else. Likewise, just writing that your app is not official will not make a trademark infringement vanish. (Of course, if Wikipedia does not have a valid trademark which you infringed, that’s another story. But it’s not what I would guess has happened here.)

kfiven 4y ago

Why Wikipedia though? Wikidata should be one to get this kind of data and all Wikipedia articles has Wikidata items. On top of that Wikidata even keep records of site name, like when someone change their site name.

ranger_danger 4y ago
Literally never heard of wikidata, and I doubt most other people have either.
dontbenebby 4y ago
I think a lot of people conflate Wikipedia with the larger Wikimedia organization. https://www.wikimedia.org/ https://www.wikimedia.org/
xd1936 4y ago
I suppose the thought process is, more eyeballs are on corresponding Wikipedia pages for entities than are on Wikidata entries, so it's more likely to be corrected/up-to-date.
- kfiven 4y ago
  In many type of Infobox on Wikipedia, it's Wikidata which is working from the behind and editing the data in such articles is basically being done at Wikidata.

alpos 4y ago

What are you going to do if people start to trust this and then someone edits the wiki sources to redirect people to phishing sites mocked up to look real?

anenefan 4y ago
This is a good point, this it something that has happened (I've personally witnessed) and probably still happens from time to time unless wikipedia has worked to stop that sort of abuse. Long story short, about ten years ago, my recommendation of a specific search tool (shareaza) to a friend went south, since it was over the phone. I'd found telling people to use web searches didn't work out well, wikipedia on the other hand, usually had an up to date address -- I no longer recall what trouble it caused, but I spent some time wondering what malware was redirecting to the bogus software site -- there was none, it was the edited entry at wikipedia. (When I tried the bogus software I recall it looked like it was doing something, but in fact it was doing nothing as far as searching p2p networks. Being busy I wasn't interested to see what its objectives were before deleting.)

lxgr 4y ago

This is neat, but I expected something completely different from the name. I already had my hopes up for doing VPN/IP over DNS over Wikipedia. Frustratingly, beyond not actually offering that (which is entirely reasonable), it does not even seem to be using DNS for the implementation of what it does.

9edda054-232f 4y ago

Why don't just buy a domain like DNSOver.Wiki so people can just scihub.dnsover.wiki without installing the browser extension..

aryamaan 4y ago
it's actually a clever idea

752963e64 4y ago

shp0ngle 4y ago

8kun and kiwifarms are both censored on wikipedia. Maybe that’s a good thing. But let’s not pretend it’s less censored than google!

Thorentis 4y ago

This isn't DNS (resolving IPs to FQDNs) as much as resolving DNS prefixes to their current suffixes. As others have pointed out, Wikipedia is starting to censor results just like Google. Maybe it would be better for this extension to pivot into providing their own service that performs this task.

pessimizer 4y ago

If this takes off, it's just going to result in a court order against Wikipedia. Z-lib getting popular on Tiktok = no z-lib anymore. Getting Wikipedia censored will ultimately make it harder for people to find sites like scihub.

pyinstallwoes 4y ago
Perhaps we should do the same idea on top of a DHT

SpaghettiX 4y ago

Is this HTTP redirect using Wikipedia, rather than DNS over Wikipedia?