9 ms·

Free full BGP feed. IPv4 and IPv6 (2020)

48 points by pm2222 4mo ago

tiernano 4mo ago
Seems to be from 6 years ago.
kirkifier 4mo ago
[dead]
Bender 4mo ago
Just to have a compressed static file that contains all the routes one could download from time to time I think would be interesting to analyze. Same goes for the anycast root DNS servers. To have a full dump from them could be interesting. Not to be confused with the root.zone [1] I mean the whole kit and caboodle. [1] - https://www.internic.net/domain/root.zone https://www.internic.net/domain/root.zone
- icehawk 4mo ago
  RIPE keeps this https://ris.ripe.net/docs/mrt/#name-and-location https://ris.ripe.net/docs/mrt/#name-and-location
  - Bender 4mo ago
    The collectors are browsable nice, Thankyou! The collectors seem to be overloaded so made a temporary mirror of the one for AMS In AMS. [1] I only mirrored the latest files. Feel free to beat it up. The .gz files decompress to just under 4GB [1] - https://ams.nochan.net/data.ris.ripe.net/rrc00/ https://ams.nochan.net/data.ris.ripe.net/rrc00/
    - Bender 4mo ago
      No interest in this mirror, gonna nuke it.
- toast0 4mo ago
  > Just to have a compressed static file that contains all the routes one could download from time to time I think would be interesting to analyze. When I needed this for some work stuff, it was pretty easy to find table dumps and work with them? I don't remember where it was, but I'm sure you can find some. After an acquisition, we had our own ASN and I was able to get table dumps from our own infrastructure. > Same goes for the anycast root DNS servers. To have a full dump from them could be interesting. Not to be confused with the root.zone [1] I mean the whole kit and caboodle. What do you think the root servers have that's not in the root.zone? I think you can AXFR from the root servers, too, but it should be the same thing as from the https site.
  - Bender 4mo ago
    What do you think the root servers have that's not in the root.zone? Every glue record for every domain name. The glue is the valuable goo. Without the goo everyone goes back to /etc/hosts which I am not opposed to.
    - toast0 4mo ago
      The root servers don't have the glue record for every domain name though. All they have is the glue records for the TLDs. (which is in the zone file you linked)
      - Bender 4mo ago
        Every time I stand up new name servers I have to add the glue records into the root servers or the name servers do not exist. In fairness to me I keep forgetting the trend is to shove everything into big centralized DNS servers as it is something I would never do at least not as a primary.
        toast0 4mo ago
        Afaik, those glue records are held at your TLD's registry and served by the tld nameservers, not the root servers. It might be nice to get a zone transfer for every tld, but that's not possible for the public. (I understand there's some way to get many of them, but $$$$)
- deleted 4mo ago
  [deleted]
- johnhtodd 4mo ago
  This is conflating BGP routes and DNS. DNS data: Root server data is available via AXFR ("dig . AXFR @f.root-servers.net") but this isn't what you're referencing. Second level TLD server data is available is available at CZDS. (https://czds.icann.org/home https://czds.icann.org/home) but some TLDs don't participate, but this also isn't what you're referencing. What I think you want: There is no canonical list of all zones that exist - there is no "central repository" once you pass the root zone downwards - that's a feature, not a bug. Some organizations have partial views based on large recursive resolver data (DomainTools, Google, Cisco, Cloudflare, Quad9) but access to that data is limited to vetted researchers or more typically only available at a cost (disclaimer: I work for Quad9.) Smaller versions of recursive data sets exist, but are usually significantly limited by geography and demographics of the user community that generates the data set. BGP route data: This exists in many forms in realtime like the site referenced above, though historic data is difficult to track. No matter what the source or latency, there is bias in the data set because BGP pathing is unique to each ASN that collects it - no two views of the table are identical, and any data set is as "best guess" at state conditions at that time. Here are some possible data sets for BGP: Packet Clearing House (PCH) provides a set of snapshots going back 20+ years (though it seems to be offline at the moment): https://www.pch.net/resources/Routing_Data/ https://www.pch.net/resources/Routing_Data/ Cymru has a live version you can query via various APIs (including ironically via DNS): https://www.team-cymru.com/ip-asn-mapping https://www.team-cymru.com/ip-asn-mapping Routeviews from University of Oregon also has a data set that is widely used by researchers: https://www.routeviews.org/routeviews/ https://www.routeviews.org/routeviews/
  - Bender 4mo ago
    This is conflating BGP routes and DNS. No I was fantasizing that I could get these two things. They are two separate things. If I had a nickle for every time I said something off topic I could by everyone a cup of coffee.
johnea 4mo ago
Ignorant question here: Isn't this an open port for BGP poisoning?
- ronsor 4mo ago
  Only if they haven't enabled filtering, which would be ludicrous.
afshari 4mo ago
[shameless plug] I made a tool to look into BGP data as well as other networky things at https://god.ad https://god.ad. Can be added to homescreen as a PWA app. https://god.ad/view/AS15169 https://god.ad/view/AS15169
- satyamkapoor 4mo ago
  Thanks, looks very llmish
  - satyamkapoor 4mo ago
    Doesn’t mean it’s bad. Thanks for the work
  - afshari 4mo ago
    Front-end was indeed done with Claude, with many iterations and tweaks though
- motohagiography 4mo ago
  this is great, thank you.
faangguyindia 4mo ago
does anyone know where can i buy rDNS records? purpose is find all domains hosted on specific ip for a monthly price? I think crawling all my own is big environment waste of resources, i'd be happy to pay for a subscription or somethign
- yjftsjthsd-h 4mo ago
  > does anyone know where can i buy rDNS records? purpose is find all domains hosted on specific ip for a monthly price Are rdns records always populated? I thought they were optional and many sites didn't bother
  - faangguyindia 4mo ago
    i am talking about this data: https://host.io/lgyy.cc https://host.io/lgyy.cc
    - zamadatix 4mo ago
      They mean anyone will only ever be able to get some rather than all because of how DNS works. Even that site is unable to find all such mappings for every IP. I.e. the first option to build such a dataset is to use rDNS lookups, but it's optional to configure those and many don't bother (opting to just have the forward lookup). The other way about it is collecting domains and seeing which IPs they resolve to, but this is only ever partial (there is no such thig as a conplete lists of domains in the world) and has additional problems with CDNs or things hosted in more than one place at once (you need to do a lookup from every possible location each domain could conceivably be hosted to be sure you get all of the reverse mappings for that domain). As an example: Unless you know my domains ahead of time you'll never be able to come up with what domains are hosted on my IPs because I don't bother to configure rDNS. So those IPs will look like they host no servers (or only some if you only had a partial list of my domains) rather than all of them on those IPs. Anyways, for free data sources trying to get a partial view of this you can check out Rapid7's Sonar or Common Crawl. Each should have the pieces needed to construct this kind of view from the data.
deleted 4mo ago
[deleted]

48 points by pm2222 4mo ago

tiernano 4mo ago

Seems to be from 6 years ago.

kirkifier 4mo ago

[dead]

Bender 4mo ago

Just to have a compressed static file that contains all the routes one could download from time to time I think would be interesting to analyze. Same goes for the anycast root DNS servers. To have a full dump from them could be interesting. Not to be confused with the root.zone [1] I mean the whole kit and caboodle. [1] - https://www.internic.net/domain/root.zone https://www.internic.net/domain/root.zone

icehawk 4mo ago
RIPE keeps this https://ris.ripe.net/docs/mrt/#name-and-location https://ris.ripe.net/docs/mrt/#name-and-location
- Bender 4mo ago
  The collectors are browsable nice, Thankyou! The collectors seem to be overloaded so made a temporary mirror of the one for AMS In AMS. [1] I only mirrored the latest files. Feel free to beat it up. The .gz files decompress to just under 4GB [1] - https://ams.nochan.net/data.ris.ripe.net/rrc00/ https://ams.nochan.net/data.ris.ripe.net/rrc00/
  - Bender 4mo ago
    No interest in this mirror, gonna nuke it.
toast0 4mo ago
> Just to have a compressed static file that contains all the routes one could download from time to time I think would be interesting to analyze. When I needed this for some work stuff, it was pretty easy to find table dumps and work with them? I don't remember where it was, but I'm sure you can find some. After an acquisition, we had our own ASN and I was able to get table dumps from our own infrastructure. > Same goes for the anycast root DNS servers. To have a full dump from them could be interesting. Not to be confused with the root.zone [1] I mean the whole kit and caboodle. What do you think the root servers have that's not in the root.zone? I think you can AXFR from the root servers, too, but it should be the same thing as from the https site.
- Bender 4mo ago
  What do you think the root servers have that's not in the root.zone? Every glue record for every domain name. The glue is the valuable goo. Without the goo everyone goes back to /etc/hosts which I am not opposed to.
  - toast0 4mo ago
    The root servers don't have the glue record for every domain name though. All they have is the glue records for the TLDs. (which is in the zone file you linked)
    - Bender 4mo ago
      Every time I stand up new name servers I have to add the glue records into the root servers or the name servers do not exist. In fairness to me I keep forgetting the trend is to shove everything into big centralized DNS servers as it is something I would never do at least not as a primary.
      - toast0 4mo ago
        Afaik, those glue records are held at your TLD's registry and served by the tld nameservers, not the root servers. It might be nice to get a zone transfer for every tld, but that's not possible for the public. (I understand there's some way to get many of them, but $$$$)
deleted 4mo ago
[deleted]
johnhtodd 4mo ago
This is conflating BGP routes and DNS. DNS data: Root server data is available via AXFR ("dig . AXFR @f.root-servers.net") but this isn't what you're referencing. Second level TLD server data is available is available at CZDS. (https://czds.icann.org/home https://czds.icann.org/home) but some TLDs don't participate, but this also isn't what you're referencing. What I think you want: There is no canonical list of all zones that exist - there is no "central repository" once you pass the root zone downwards - that's a feature, not a bug. Some organizations have partial views based on large recursive resolver data (DomainTools, Google, Cisco, Cloudflare, Quad9) but access to that data is limited to vetted researchers or more typically only available at a cost (disclaimer: I work for Quad9.) Smaller versions of recursive data sets exist, but are usually significantly limited by geography and demographics of the user community that generates the data set. BGP route data: This exists in many forms in realtime like the site referenced above, though historic data is difficult to track. No matter what the source or latency, there is bias in the data set because BGP pathing is unique to each ASN that collects it - no two views of the table are identical, and any data set is as "best guess" at state conditions at that time. Here are some possible data sets for BGP: Packet Clearing House (PCH) provides a set of snapshots going back 20+ years (though it seems to be offline at the moment): https://www.pch.net/resources/Routing_Data/ https://www.pch.net/resources/Routing_Data/ Cymru has a live version you can query via various APIs (including ironically via DNS): https://www.team-cymru.com/ip-asn-mapping https://www.team-cymru.com/ip-asn-mapping Routeviews from University of Oregon also has a data set that is widely used by researchers: https://www.routeviews.org/routeviews/ https://www.routeviews.org/routeviews/
- Bender 4mo ago
  This is conflating BGP routes and DNS. No I was fantasizing that I could get these two things. They are two separate things. If I had a nickle for every time I said something off topic I could by everyone a cup of coffee.

johnea 4mo ago

Ignorant question here: Isn't this an open port for BGP poisoning?

ronsor 4mo ago
Only if they haven't enabled filtering, which would be ludicrous.

afshari 4mo ago

[shameless plug] I made a tool to look into BGP data as well as other networky things at https://god.ad https://god.ad. Can be added to homescreen as a PWA app. https://god.ad/view/AS15169 https://god.ad/view/AS15169

satyamkapoor 4mo ago
Thanks, looks very llmish
- satyamkapoor 4mo ago
  Doesn’t mean it’s bad. Thanks for the work
- afshari 4mo ago
  Front-end was indeed done with Claude, with many iterations and tweaks though
motohagiography 4mo ago
this is great, thank you.

faangguyindia 4mo ago

does anyone know where can i buy rDNS records? purpose is find all domains hosted on specific ip for a monthly price? I think crawling all my own is big environment waste of resources, i'd be happy to pay for a subscription or somethign

yjftsjthsd-h 4mo ago
> does anyone know where can i buy rDNS records? purpose is find all domains hosted on specific ip for a monthly price Are rdns records always populated? I thought they were optional and many sites didn't bother
- faangguyindia 4mo ago
  i am talking about this data: https://host.io/lgyy.cc https://host.io/lgyy.cc
  - zamadatix 4mo ago
    They mean anyone will only ever be able to get some rather than all because of how DNS works. Even that site is unable to find all such mappings for every IP. I.e. the first option to build such a dataset is to use rDNS lookups, but it's optional to configure those and many don't bother (opting to just have the forward lookup). The other way about it is collecting domains and seeing which IPs they resolve to, but this is only ever partial (there is no such thig as a conplete lists of domains in the world) and has additional problems with CDNs or things hosted in more than one place at once (you need to do a lookup from every possible location each domain could conceivably be hosted to be sure you get all of the reverse mappings for that domain). As an example: Unless you know my domains ahead of time you'll never be able to come up with what domains are hosted on my IPs because I don't bother to configure rDNS. So those IPs will look like they host no servers (or only some if you only had a partial list of my domains) rather than all of them on those IPs. Anyways, for free data sources trying to get a partial view of this you can check out Rapid7's Sonar or Common Crawl. Each should have the pieces needed to construct this kind of view from the data.

deleted 4mo ago

[deleted]