6 ms·

Tell HN: All .fj domains have gone offline

93 points by markdown 5y ago

domains.fj fiji.gov.fj usp.ac.fj etc. http://www.iana.org/domains/root/db/fj.html

kube-system 5y ago
This is why I always say you should think twice when choosing a ccTLD if you don’t have a presence there. Not every registry has the same uptime as Verisign.
- markdown 5y ago
  And yet to my knowledge this has never happened before.
  - sofixa 5y ago
    Didn't .io ( which is a crime against humanity to boot) go down for a few hours a few years ago?
    - markdown 5y ago
      No idea. I was referring to .fj
    - poopsmithe 5y ago
      Why do you say .io is a crime against humanity?
      - kergonath 5y ago
        .io is the TLD for the British Indian Ocean Territory, which is associated with the forced displacement of the Chagos islander people to make room for the Diego Garcia military base, amongst others. The British unilaterally decided to detach the Chagos archipelago from Mauritius before it got independent, which the deported people say (not without reason) is a crime against their human rights.
        andylynch 5y ago
        To back that up - the Chagosian claim is backed by the UK and rulings at the International Court of Justice. The UN supports Mauritius' claim to the territory.
      - pxeger1 5y ago
        Even besides the literal crimes against humanity, the .io domain really doesn't need to exist - what with the British Indian Ocean Territory having no permanent population and really not being a country at all.
        Symbiote 5y ago
        "Being a country" isn't the criteria for the ISO code or domain. Consider AQ, TW, GG, SJ, EH, HK and many others.
        blowski 5y ago
        TW is a controversial inclusion in your list.
        Symbiote 5y ago
        Several of them are controversial to someone. AQ is controversial to countries who want to exploit Antarctica's natural resources. TW is controversial to China. GG is controversial where it's used for tax evasion etc. EH to Morocco, HK to the UK/China.
        deleted 5y ago
        [deleted]
    - trashcan 5y ago
      Yes. https://hackernoon.com/stop-using-io-domain-names-for-production-traffic-b6aa17eeac20 https://hackernoon.com/stop-using-io-domain-names-for-produc... Also there were some comments from the owner at the time that I thought were pretty unprofessional about it, too.
  - kube-system 5y ago
    Maybe not, but it is predicable that the quality and quantity of resources and governance available to a registry has an impact on their operations.
  - yosamino 5y ago
    I vividly remember .de being offline causing trouble for a couple of hours in 2010. https://www.theregister.com/2010/05/12/germany_top_level_domain_glitch/ https://www.theregister.com/2010/05/12/germany_top_level_dom...
    - markdown 5y ago
      It's too late to edit my comment, but I was referring to .fj, not ccTLD's in general.
- Tepix 5y ago
  Given the amount of past outages of ccTLDs we've seen (almost none), i'd say this problem is negligible and nothing to worry about. There is one very recent exception however, the russian plans to partly separate from the internet shortly (especially regarding DNS) could well be an issue.
  - kube-system 5y ago
    Technical and legal issues that cause issues for customers at small ccTLDs are not uncommon. It all depends on who is running the registry. Try "cctld hijacked" and "cctld downtime" in your favorite search engine for a dozen or so examples. Registries range in resources from "highly experienced billion-dollar organizations", to literally "we have a guy who updates the zone file in notepad", and they range in legal environment from, "functional democratic state with well-established judiciary" to "our government was violently overthrown this week". Whether or not the problem is negligible depends on your particular use-case. For some, it might be a good choice, for others, it might not be.
- teddyh 5y ago
  Like I have said before¹²: Neither your registrar nor your chosen TLD registry should be in the habit of suspending domains at the drop of a hat, or be at risk of going out of business suddenly. If you mostly trust your local government, your national ccTLD should suffice. In fact, it should be your default choice unless you have strong indications it does not fulfill the above criteria. 1. https://news.ycombinator.com/item?id=29108933 https://news.ycombinator.com/item?id=29108933 2. https://news.ycombinator.com/item?id=21614298 https://news.ycombinator.com/item?id=21614298
herpderperator 5y ago
It's true... couldn't get address for 'ns4.fj': not found couldn't get address for 'ns5.fj': not found couldn't get address for 'ns2.fj': not found couldn't get address for 'ns1.fj': not found couldn't get address for 'ns3.fj': not found dig: couldn't get address for 'ns4.fj': no more
niftich 5y ago
Fijivillage.com reports: "All websites and apps hosted in Fiji with the dotcom.fj suffix are currently down and this has also affected Vodafone’s M-PAiSA services. This is due to an outage in the University of the South Pacific hosted dotcom.fj domain." [1][2] [1] https://www.fijivillage.com/news/All-websites--apps-in-Fiji-with-dotcomfj-suffix-are-down-and-this-has-also-affected-M-PAiSA-services-x854fr/ https://www.fijivillage.com/news/All-websites--apps-in-Fiji-... [2] https://twitter.com/fijivillage/status/1501070675691278339 https://twitter.com/fijivillage/status/1501070675691278339
- londons_explore 5y ago
  And this folks is one of the reasons you don't use a 1 minute TTL on your DNS responses...
  - Mo3 5y ago
    shamefully opens domain registrar and changes TTL
    - Maxburn 5y ago
      15 min?
      - belorn 5y ago
        I would recommend an hour for almost everything except where very fast updates is expected, in which case 5m is my lowest number (I work at a registrar).
        slenk 5y ago
        Obviously registrars allow 1m - is there a reason not too besides extra load on DNS servers?
        remram 5y ago
mike_d 5y ago
3 of the 5 nameservers for fj are online and responding (they are all anycasted around the world). It looks like connectivity to Fiji itself is what is disrupted. Two are operated by gransy.com and one by PCH.net (which you should consider donating to and helping random country codes stay online: https://www.pch.net/about/donate https://www.pch.net/about/donate).
- markdown 5y ago
  It's not connectivity to Fiji. The internet works just fine across the country. Only .fj domains (and email accounts) are affected.
Animats 5y ago
University of the South Pacific, which runs the DNS servers, [1] is up. Google's DNS server at 8.8.8.8 is having problems. Lookup using DNS server of sonic.net, which is a normal DNS server. nslookup usp.ac.fj Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: Name: usp.ac.fj Address: 144.120.141.80 Lookup with Google DNS: nslookup > server 8.8.8.8 Default server: 8.8.8.8 Address: 8.8.8.8#53 > usp.ac.fj Server: 8.8.8.8 Address: 8.8.8.8#53 ** server can't find usp.ac.fj: SERVFAIL Checking example.com to make sure Google DNS server is live. > example.com Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: Name: example.com Address: 93.184.216.34 Name: example.com Address: 2606:2800:220:1:248:1893:25c8:1946 Also fails with 1.1.1.1 DNS server. The Fiji DNS servers are unreachable, but you can get as far as a router at the University of the South Pacific. So the problem is local to there. traceroute 144.120.146.1 traceroute to 144.120.146.1 (144.120.146.1), 30 hops max, 60 byte packets ... 8 AARNET-PTY.edge5.London1.Level3.net (217.163.113.74) 274.795 ms 274.476 ms 273.781 ms 9 xe-0-0-0.pe1.a.suv.aarnet.net.au (113.197.15.213) 312.534 ms 311.764 ms 311.766 ms 10 fastethernet0-0.aarnie.usp.ac.fj (202.158.204.194) 313.356 ms 312.176 ms 312.363 ms 11 * * * 12 * * * [1] https://www.usp.ac.fj/ https://www.usp.ac.fj/
- markdown 5y ago
  > University of the South Pacific, which runs the DNS servers, [1] is up. Not up from within Fiji. All government websites (and email) are down. Banks, Universities, mobile money wallets, etc.
  - Animats 5y ago
    DNS for Fiji seems to be back up.
    - markdown 5y ago
      Yeah, came back online this morning. Was down for around 18hrs I reckon.
- 5y ago
throwaway190712 5y ago
This might be a DNSSEC issue rather than a nameserver reachability issue: https://dnsviz.net/d/fj/dnssec/ https://dnsviz.net/d/fj/dnssec/
- benatkin 5y ago
  I guess Google's DNS rigidly adheres to DNSSEC.
  - arantius 5y ago
    https://developers.google.com/speed/public-dns/faq#dnssec https://developers.google.com/speed/public-dns/faq#dnssec "Google Public DNS is a validating, security-aware resolver. All responses from DNSSEC signed zones are validated unless clients explicitly set the CD flag in DNS requests to disable the validation."
- helb 5y ago
  Looks like you were right: https://lists.dns-oarc.net/pipermail/dns-operations/2022-March/021593.html https://lists.dns-oarc.net/pipermail/dns-operations/2022-Mar...
adamkochanowicz 5y ago
Reachable for me.
- markdown 5y ago
  Would you mind saying where in the world you are?
- MaxBarraclough 5y ago
  I'm late to the party, but https://fiji.gov.fj/ https://fiji.gov.fj/ worked for me, although it was rather slow to load. I'm in England, for what that's worth.
  - _1tan 5y ago
    Working slowly from Germany as well.
- kinow 5y ago
  Not working for me, in New Zealand.
deleted 5y ago
[deleted]
blikdak 5y ago
Is there anyone here in Fiji who can say they are OK?
- goldforever 5y ago
chimbosonic 5y ago
Using a Recursive DNS server its down for me. UK based. I also setup a RIPE measurement for those interested. It asks for the NS record of ns1.fj. https://atlas.ripe.net/measurements/39252885/#general https://atlas.ripe.net/measurements/39252885/#general

93 points by markdown 5y ago

domains.fj fiji.gov.fj usp.ac.fj etc. http://www.iana.org/domains/root/db/fj.html

This is why I always say you should think twice when choosing a ccTLD if you don’t have a presence there. Not every registry has the same uptime as Verisign.

markdown 5y ago
And yet to my knowledge this has never happened before.
- sofixa 5y ago
  Didn't .io ( which is a crime against humanity to boot) go down for a few hours a few years ago?
  - markdown 5y ago
    No idea. I was referring to .fj
  - poopsmithe 5y ago
    Why do you say .io is a crime against humanity?
    - kergonath 5y ago
      .io is the TLD for the British Indian Ocean Territory, which is associated with the forced displacement of the Chagos islander people to make room for the Diego Garcia military base, amongst others. The British unilaterally decided to detach the Chagos archipelago from Mauritius before it got independent, which the deported people say (not without reason) is a crime against their human rights.
      - andylynch 5y ago
        To back that up - the Chagosian claim is backed by the UK and rulings at the International Court of Justice. The UN supports Mauritius' claim to the territory.
    - pxeger1 5y ago
      Even besides the literal crimes against humanity, the .io domain really doesn't need to exist - what with the British Indian Ocean Territory having no permanent population and really not being a country at all.
      - Symbiote 5y ago
        "Being a country" isn't the criteria for the ISO code or domain. Consider AQ, TW, GG, SJ, EH, HK and many others.
        blowski 5y ago
        TW is a controversial inclusion in your list.
        Symbiote 5y ago
        Several of them are controversial to someone. AQ is controversial to countries who want to exploit Antarctica's natural resources. TW is controversial to China. GG is controversial where it's used for tax evasion etc. EH to Morocco, HK to the UK/China.
        deleted 5y ago
        [deleted]
  - trashcan 5y ago
    Yes. https://hackernoon.com/stop-using-io-domain-names-for-production-traffic-b6aa17eeac20 https://hackernoon.com/stop-using-io-domain-names-for-produc... Also there were some comments from the owner at the time that I thought were pretty unprofessional about it, too.
- kube-system 5y ago
  Maybe not, but it is predicable that the quality and quantity of resources and governance available to a registry has an impact on their operations.
- yosamino 5y ago
  I vividly remember .de being offline causing trouble for a couple of hours in 2010. https://www.theregister.com/2010/05/12/germany_top_level_domain_glitch/ https://www.theregister.com/2010/05/12/germany_top_level_dom...
  - markdown 5y ago
    It's too late to edit my comment, but I was referring to .fj, not ccTLD's in general.
Tepix 5y ago
Given the amount of past outages of ccTLDs we've seen (almost none), i'd say this problem is negligible and nothing to worry about. There is one very recent exception however, the russian plans to partly separate from the internet shortly (especially regarding DNS) could well be an issue.
- kube-system 5y ago
  Technical and legal issues that cause issues for customers at small ccTLDs are not uncommon. It all depends on who is running the registry. Try "cctld hijacked" and "cctld downtime" in your favorite search engine for a dozen or so examples. Registries range in resources from "highly experienced billion-dollar organizations", to literally "we have a guy who updates the zone file in notepad", and they range in legal environment from, "functional democratic state with well-established judiciary" to "our government was violently overthrown this week". Whether or not the problem is negligible depends on your particular use-case. For some, it might be a good choice, for others, it might not be.
teddyh 5y ago
Like I have said before¹²: Neither your registrar nor your chosen TLD registry should be in the habit of suspending domains at the drop of a hat, or be at risk of going out of business suddenly. If you mostly trust your local government, your national ccTLD should suffice. In fact, it should be your default choice unless you have strong indications it does not fulfill the above criteria. 1. https://news.ycombinator.com/item?id=29108933 https://news.ycombinator.com/item?id=29108933 2. https://news.ycombinator.com/item?id=21614298 https://news.ycombinator.com/item?id=21614298

herpderperator 5y ago

It's true... couldn't get address for 'ns4.fj': not found couldn't get address for 'ns5.fj': not found couldn't get address for 'ns2.fj': not found couldn't get address for 'ns1.fj': not found couldn't get address for 'ns3.fj': not found dig: couldn't get address for 'ns4.fj': no more

niftich 5y ago

Fijivillage.com reports: "All websites and apps hosted in Fiji with the dotcom.fj suffix are currently down and this has also affected Vodafone’s M-PAiSA services. This is due to an outage in the University of the South Pacific hosted dotcom.fj domain." [1][2] [1] https://www.fijivillage.com/news/All-websites--apps-in-Fiji-with-dotcomfj-suffix-are-down-and-this-has-also-affected-M-PAiSA-services-x854fr/ https://www.fijivillage.com/news/All-websites--apps-in-Fiji-... [2] https://twitter.com/fijivillage/status/1501070675691278339 https://twitter.com/fijivillage/status/1501070675691278339

londons_explore 5y ago
And this folks is one of the reasons you don't use a 1 minute TTL on your DNS responses...
- Mo3 5y ago
  shamefully opens domain registrar and changes TTL
  - Maxburn 5y ago
    15 min?
    - belorn 5y ago
      I would recommend an hour for almost everything except where very fast updates is expected, in which case 5m is my lowest number (I work at a registrar).
      - slenk 5y ago
        Obviously registrars allow 1m - is there a reason not too besides extra load on DNS servers?
        remram 5y ago

mike_d 5y ago

3 of the 5 nameservers for fj are online and responding (they are all anycasted around the world). It looks like connectivity to Fiji itself is what is disrupted. Two are operated by gransy.com and one by PCH.net (which you should consider donating to and helping random country codes stay online: https://www.pch.net/about/donate https://www.pch.net/about/donate).

markdown 5y ago
It's not connectivity to Fiji. The internet works just fine across the country. Only .fj domains (and email accounts) are affected.

Animats 5y ago

University of the South Pacific, which runs the DNS servers, [1] is up. Google's DNS server at 8.8.8.8 is having problems. Lookup using DNS server of sonic.net, which is a normal DNS server. nslookup usp.ac.fj Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: Name: usp.ac.fj Address: 144.120.141.80 Lookup with Google DNS: nslookup > server 8.8.8.8 Default server: 8.8.8.8 Address: 8.8.8.8#53 > usp.ac.fj Server: 8.8.8.8 Address: 8.8.8.8#53 ** server can't find usp.ac.fj: SERVFAIL Checking example.com to make sure Google DNS server is live. > example.com Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: Name: example.com Address: 93.184.216.34 Name: example.com Address: 2606:2800:220:1:248:1893:25c8:1946 Also fails with 1.1.1.1 DNS server. The Fiji DNS servers are unreachable, but you can get as far as a router at the University of the South Pacific. So the problem is local to there. traceroute 144.120.146.1 traceroute to 144.120.146.1 (144.120.146.1), 30 hops max, 60 byte packets ... 8 AARNET-PTY.edge5.London1.Level3.net (217.163.113.74) 274.795 ms 274.476 ms 273.781 ms 9 xe-0-0-0.pe1.a.suv.aarnet.net.au (113.197.15.213) 312.534 ms 311.764 ms 311.766 ms 10 fastethernet0-0.aarnie.usp.ac.fj (202.158.204.194) 313.356 ms 312.176 ms 312.363 ms 11 * * * 12 * * * [1] https://www.usp.ac.fj/ https://www.usp.ac.fj/

markdown 5y ago
> University of the South Pacific, which runs the DNS servers, [1] is up. Not up from within Fiji. All government websites (and email) are down. Banks, Universities, mobile money wallets, etc.
- Animats 5y ago
  DNS for Fiji seems to be back up.
  - markdown 5y ago
    Yeah, came back online this morning. Was down for around 18hrs I reckon.
5y ago

throwaway190712 5y ago

This might be a DNSSEC issue rather than a nameserver reachability issue: https://dnsviz.net/d/fj/dnssec/ https://dnsviz.net/d/fj/dnssec/

benatkin 5y ago
I guess Google's DNS rigidly adheres to DNSSEC.
- arantius 5y ago
  https://developers.google.com/speed/public-dns/faq#dnssec https://developers.google.com/speed/public-dns/faq#dnssec "Google Public DNS is a validating, security-aware resolver. All responses from DNSSEC signed zones are validated unless clients explicitly set the CD flag in DNS requests to disable the validation."
helb 5y ago
Looks like you were right: https://lists.dns-oarc.net/pipermail/dns-operations/2022-March/021593.html https://lists.dns-oarc.net/pipermail/dns-operations/2022-Mar...

adamkochanowicz 5y ago

Reachable for me.

markdown 5y ago
Would you mind saying where in the world you are?
MaxBarraclough 5y ago
I'm late to the party, but https://fiji.gov.fj/ https://fiji.gov.fj/ worked for me, although it was rather slow to load. I'm in England, for what that's worth.
- _1tan 5y ago
  Working slowly from Germany as well.
kinow 5y ago
Not working for me, in New Zealand.

deleted 5y ago

[deleted]

blikdak 5y ago

Is there anyone here in Fiji who can say they are OK?

goldforever 5y ago

chimbosonic 5y ago

Using a Recursive DNS server its down for me. UK based. I also setup a RIPE measurement for those interested. It asks for the NS record of ns1.fj. https://atlas.ripe.net/measurements/39252885/#general https://atlas.ripe.net/measurements/39252885/#general