20 ms·

Why and How I Got My Own ASN

311 points by zdw 5y ago

glecedric 5y ago
The Ipv6 pool is quite vast. Yet I am a little surprised that an individual can receive a /48 without much trouble: that a lot of IPs.
- cbg0 5y ago
  This is based on the recommendations in https://datatracker.ietf.org/doc/html/rfc3177 https://datatracker.ietf.org/doc/html/rfc3177 A relevant snippet: > This means that we feel comfortable about the prospect of allocating 178 billions /48 prefixes under that scheme before problems start to appear. To understand how big that number is, one has to compare 178 billion to 10 billion, which is the projected population on earth in year 2050
  - cassianoleal 5y ago
    ~So that's a maximum average 17.8 IPs per person? Seems awfully low, considering people these days have multiple devices connected to the Internet at any given time: phone, watch, tablet, laptop(s), perhaps desktop(s), security cameras, IP phones, refrigerators, washing machines, TVs, video game consoles, I could go on...~ Edit: oh, wow I misread that completely. It's 17.8 /48 ranges per person on average. Yep, that should be enough for quite a while.
    - klysm 5y ago
      Every phone gets a /48!
- chippiewill 5y ago
  IIRC /64s are ideally meant to address to individual subnets (the bottom 64 bits aren't intended for routing) so it's only really 64k addresses in the conventional sense. /48 is actually a relatively standard allocation even for home connections, although /56 is more common.
  - jagger27 5y ago
    I think what Rogers in Canada does is reasonable. They issue a /64 by default and a /56 if you set an assignment hint.
  - hotpotamus 5y ago
    This was key to me understanding IPv6 - you don't really do variable length subnets. You just get a /48 and break that up into /64's. Because the space is so vast, you can do inefficient things with no chance of running into allocation issues like v4 and you don't have to mentally try and subnet a 128 bit address which maybe some can handle but hurts my head.
  - kloch 5y ago
    The intent of the standard /48 end user size was to prevent residential ISP's from being stingy and only assigning one IP. They wanted homes to have as many publicly routable subnets as the would ever need or want. This was later revised to "/56 by default, /48 if you ask with no justification needed"
- normaler 5y ago
  I was surprised about that too. I got 1 static ipv4 Form my ISP and a /48 for IPv6. Not sure what to do with that, but still cool.
- throw0101a 5y ago
  > The Ipv6 pool is quite vast. Yet I am a little surprised that an individual can receive a /48 without much trouble: that a lot of IPs. A /48 is considered one "site" in current thinking. Since IPv6 subnets are /64, you have 16 bits between the /48 and the /64. This is the equivalent of using 10/8 for your network and using /24 IPv4 subnets: in both cases you can have upto 2^16 subnets. The main difference being that a /24 can have ~250 hosts, but a /64 IPv6 subnet can hold the equivalent of four billion Internets (2^32 * 2^32). But one of the selling points of IPv6 is reducing/eliminating the mental math about worrying about if you have "enough" addresses (and then carving things into /26, /30, etc).
  - dsl 5y ago
    > A /48 is considered one "site" in current thinking. This is the really important part. As they continue to hand out IPv6 like candy, the minimum prefix length will get shorter. No ISP wants a hundred million+ routes in their routing table, so people will start to drop anything shorter than a /42, /40, /38, etc. until the table gets small enough and shunt everyone elses traffic off to Hurricane Electric or the like as a default route. People have this mindset of "we added a bunch of zeros, its an infinite resource now!" which is how we ended up in this mess to start with.
    - throw0101a 5y ago
      > This is the really important part. As they continue to hand out IPv6 like candy, the minimum prefix length will get shorter. Unlikely. > No ISP wants a hundred million+ routes in their routing table […] Too late. IPv4 are set to hit 1024K (2^20) in January 2024 at current trends: * https://blog.apnic.net/2021/03/03/what-will-happen-when-the-routing-table-hits-1024k/ https://blog.apnic.net/2021/03/03/what-will-happen-when-the-... Already close to 10^6: > I see 904560 IPv4 prefixes. This is 30 fewer prefixes than 6 hours ago and 416 more than a week ago. 59.10% of prefixes are /24. * https://twitter.com/bgp4_table https://twitter.com/bgp4_table * https://bgp.potaroo.net https://bgp.potaroo.net * https://www.cidr-report.org/as2.0/ https://www.cidr-report.org/as2.0/ Also: https://twitter.com/bgp6_table https://twitter.com/bgp6_table
- jleahy 5y ago
  I have a /29, there's a lot of IPv6.
- zahllos 5y ago
  Another point to consider is the IPv6 global routing table for internet routers. Suppose you decided to give end users /120s instead and those /120s were all routable on the internet. That means there are 120 bits of addressing just to find the network. There are 2^120 such networks in theory. If you could actually enumerate this, you'd be well on your way to bruteforcing AES128. In other words, this is just infeasible. By handing out /48s the routing table stays manageable. This is the smallest address block you can announce via BGP for this reason. Given the utter vastness of IPv6 we are also able to do things like carve out an entire /7, fc00::/7, for unique local addresses, and still tell people they shouldn't actually need these addresses at all. As to the actual process of getting a PI block, I think it is likely to involve some questions. A similar objection exists to handing out smaller than /48s: more people having their own block implies more routing entries. Much better if an existing provider carves you a /48 out of their allocation and routes you traffic. This is probably balanced against the fact that by requiring you to deal with an existing LIR and set up BGP (and your LIR won't want you to muck that up) the number of people who will actually do this just for fun is limited.
- kloch 5y ago
  The IETF IPv6 allocation scheme was crazy from the start. The standard subnet size for autoconfiguration purposes is /64 = 18 Billion Trillion addresses. Of course reality is that autoconfiguration is only marginally useful and lots of things like servers and infrastructure links are statically assigned or assigned through dhcpv6. Most of the time I assign /112's to server subnets to limit the risk of IPv6 neighbor discovery attacks. The standard /48 assignment size from ISP's to end-users was targeted mainly at stingy residential ISP's that were only assigning one IP per customer. They wanted any customer to have as many publicly routable subnets as they would ever need or want, but 65k is a lot of subnets. This was later updated to a default of /56 (256 subnets) but you could still get at /48 just for asking. It got even weirder on RIR to ISP portable allocations. One of the original schemes would have RIR's allocating a "TLA" /16 (!) to only a few mega ISP's. Most ISP's/hosts would have to get their "NLA" address blocks from and be subservient to the 800 pound gorillas in the industry. https://www.rfc-editor.org/rfc/rfc2450.txt https://www.rfc-editor.org/rfc/rfc2450.txt, https://www.rfc-editor.org/rfc/rfc2374 https://www.rfc-editor.org/rfc/rfc2374 This was loosened somewhat in 2000 with RFC2928 which designated a "Sub-TLA" /29 as the initial ISP size. This would open it up to many more potential registrants, at least in theory. https://www.rfc-editor.org/rfc/rfc2928.html https://www.rfc-editor.org/rfc/rfc2928.html The TLA/Sub-TLA/NLA system was abandoned in 2003 with RFC3587 after panic set in that nobody was deploying IPv6. https://www.rfc-editor.org/rfc/rfc3587.txt https://www.rfc-editor.org/rfc/rfc3587.txt. ARIN had only made 30 Sub-TLA assignments by the end of 2002. While RFC3587 wisely punted allocation policy to the RIR communities, the RIR's still had very restrictive policies inherited from RFC2450. Only around 2004 did they loosen to the point that ordinary networks could start requesting addresses. Google for example got their first IPv6 allocation in 2005. The problem is by then few networks were interested or just assumed they wouldn't qualify. When I got my first /32 in late 2004 there were less than 1000 routes in the global IPv6 table! Today there are ~135,000. These more permissive rules only applied to ISP's/Hosts. End user orgs of any size were not allowed to start requesting portable /48's directly from RIR's until 2006 after much debate in RIR communities and vocal objections from IETF members and certain large ISP's. Today router silicon and RIR policies have converged to a reasonably functional state. Too bad it took 20 years to get here or IPv6 might actually be on it's way to replacing IPV4. Instead that is still about 20 years away.
teddyh 5y ago
> I had to escape all the : in my shell Why? The colon character is not a special character in shell commands.
- craftkiller 5y ago
  The author uses OpenBSD which uses ksh as their default shell. I just ran ksh on Arch Linux and even though `touch foo:bar` worked, when I did tab-completion the shell escaped the colon anyway, making my command `touch foo\:bar`. I assume its related to this, but I've never used ksh before so I can't explain the difference in the tab-completion behavior.
  - Arnavion 5y ago
    bash is the same - `:` isn't special but tab-completion escapes it. Tab-completion is probably being conservative about escaping it in case you're using it as a path separator, like when setting a `$PATH`-like var.
    - dsl 5y ago
      The shell doesn't know what the underlying filesystem is. : is special on NTFS for alternative data streams, and I think HFS+ named forks? (The mac syntax is foo/..namedfork/rsrc, I don't know how that translates to linux) "foo\:bar" is a file name with a colon in it. "foo:bar" is an ADS named bar on the file foo.
      - Arnavion 5y ago
        Maybe. The motivation may have been lost to history. bash added `:` to the list of completion word break characters with bash 2.0 [1][2]. Bash 2.0 was released in 1996. NTFS-3G started in 2006 but it was "forked from the Linux-NTFS", and I can't find any history for this project. NTFS itself was released in 1993. HFS+ seems to have been released in 1998, so it can't be the reason. But even regular HFS had resource forks and it was released in 1985. No idea where to look in ksh's source. [1]: https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee65b8f368e9653e1380b#diff-b8fb328cf74196c51b622571b2c765c857d4c5e766b0ce4b550f77280892e64eL239 https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee6... [2]: https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee65b8f368e9653e1380b#diff-b8fb328cf74196c51b622571b2c765c857d4c5e766b0ce4b550f77280892e64eR129 https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee6...
johnklos 5y ago
It's lovely to read about people doing things that're out of the ordinary to learn & explore because it's what they want to do. We have too many people who don't like stuff like this, who say it's "too hard", and that people shouldn't do hard things (just see any thread on the Internet about people running their own email servers). It's wonderful to see someone sharing something non-trivial in a way that makes it ever so slightly more attainable to the rest of us.
- eatonphil 5y ago
  Setting up an email server once sounds awesome. Coding an email server system from scratch also sounds awesome. But running an email server indefinitely doesn't sound super fun.
  - johnklos 5y ago
    I've been running email servers since the '90s, and there's surprisingly little work that needs to be done once it's up and running.
    - eatonphil 5y ago
      Sure, you and many people. I'm saying there's an in-between level of enthusiasm where you do want to learn new things and experiment but without getting yourself into any long-term commitments. I'm sure you'd agree that it's also easier to scale your experiments/interest if you only do small experiments rather than go all the way in every dimension.
    - excitom 5y ago
      I ran email servers for my personal domains in the 90s and early 2000s. What eventually caused me to move everything to gmail was the massive pain of dealing with spam and trying to keep up with the latest spam filtering, which is essential.
      - georgyo 5y ago
        Rspamd has made that trivial for a number of years. It's an amazing tool.
holdenk 5y ago
If anyone is in the bay area and wants to run physical hardware and peer with folks, FCIX down in the HE facility in Fremont is very welcoming to small networks and doesn't have port fees (although you do need to be in the HE facility).
- Karrot_Kream 5y ago
  Oh wow that's fantastic! Are you referring to https://fcix.net/ https://fcix.net/ ? Thanks so much for the tip. Do you know how much it costs to grab a dedicated box in the HE Fremont facility?
  - holdenk 5y ago
    So they don't do dedicated boxes (sadly), they rent by the rack (including 1gb of transit + 15A power). https://he.net/colocation.html https://he.net/colocation.html has the info. If you just want a dedicated box that isn't drawing a lot of power I've probably got some room in my rack and we can chat (and I imagine some of the other BGP for fun group folks are in similar positions).
    - Karrot_Kream 5y ago
      Sure I'd be interested! Let me know if there's contact info on your site or some other method of contact. I'm fine with Email or Matrix.
    - errcorrectcode 5y ago
      Ah, blast from the past. HE brought us T-1's, T-3's, and IPv6 transit way back. Equinix owns PAIX on 529 Bryant St. as SV8. It's interesting how racks still aren't so much volume-, mass-, or network bandwidth-/transit-constrained, but power-constrained.
      - walrus01 5y ago
        It's very easy to run out of cumulative watts of grid power feed, cooling capacity and generator/UPS/power backup capacity in a facility before you run out of floor space in a datacenter. Even with what is now a "small" 5kW thermal power budget per 44U cabinet, measuring 24 inches wide x 48 inches deep, you can easily exceed that 5kW long before you physically fill the cabinet with servers.
mlyle 5y ago
The thing that really surprises me here is that he found cheap hosts willing to run a BGP session with him. Who offers this in North America? He mentions one of his VMs is in Toronto, but not with whom... edit: Looks like https://xenyth.net/products/vps-hosting/ https://xenyth.net/products/vps-hosting/ Xenyth Cloud. Curious if there's any VM hosts in e.g. California that would do this.
- rmkrmk 5y ago
  I've done this as well recently, and the list at bgp.services helped a lot. Vultr is a good example, they have locations across the globe.
  - ssabetan 5y ago
    +1 for Vultr. I currently use them to announce a /24.
    - therein 5y ago
      I wanted to like Vultr. I gave them a fair chance but then they changed the IP on my VPS without telling me. It changed from a Swedish IP to an American IP overnight and left me confused in every way. Everything broke and support wasn't willing to explain what happened. It felt like they decided my account was suspicious and they routed all my traffic through some firewalled/monitored egress to inspect my traffic. I couldn't even SSH in, I had to go the the dashboard and grab the IP they had reassigned to my host.
      - capableweb 5y ago
        Really? That sounds extremely shitty. As a customer of Vultr, it makes me a bit scared, but during the 5~ or something years I've used them, that never happened to me. You'd be doing everyone a service by continuing to dig out a reason for why that happened, and if they don't provide it, lambast them over social media or something so they do provide some sort of justification.
DanAtC 5y ago
They hit on a point I rarely see mentioned: ARIN makes acquiring assets as an individual prohibitively difficult and expensive. Especially for IPv6 where European LIRs hand them out like candy e.g. https://clients.stratagem.host/cart.php?gid=14 https://clients.stratagem.host/cart.php?gid=14
- techsupporter 5y ago
  Having dealt with ARIN, RIPE, and APNIC quite a lot over the years, this is bang on. APNIC is fairly ambivalent. ARIN expects everyone to be a formally organized entity. Only RIPE seems to truly get that an individual person could be doing something on their own behalf, probably because forming a proper limited company was such a huge paperwork endeavor in most of RIPE's service area. ARIN really needs the concept of sponsoring LIRs, because right now everyone has to deal directly with ARIN and that means ARIN has to fit their policies to try to fit everyone. While it's technically possible to get IPv6 PI space from ARIN as an individual person end user, and it's gotten easier than in the past, it still looks really daunting to the hobbyist or small network operator. As for LIRs handing them out like candy, I joined RIPE as an LIR and RIPE will allocate an IPv6 /29 to an LIR for the low cost of zero Euro (well, except for the membership, which is a bit spendy) and that's 524,288 /48s so no reason for me to not slap up a web page and see if I can earn back a few of the EUR I will be paying to RIPE. I'm glad people do it for the pittance we see here.
  - traceroute66 5y ago
    > RIPE will allocate an IPv6 /29 to an LIR Last time I checked it was a /32. More than that requires justification (and potential Euro payments depending on context/cirumstances). Edit to add: Grrr... I see there was a policy change after $myco received its IPv6 allocation [1]. Typical. ;-( [1] https://www.ripe.net/participate/policies/proposals/2012-10 https://www.ripe.net/participate/policies/proposals/2012-10
  - boris 5y ago
    I am curious, what happens if I get a /48 from you and some time later you decide you don't wish to be a LIR anymore. Do I get to keep my /48 somehow (i.e., does RIPE guarantee this)?
yuuta 5y ago
I got my first ASN in April, 2021 from a LIR. It was an amazing experience. If you live in the west part of North America like me, you can get a IX VPS at FreeRangeCloud (disclaimer: I'm not affiliated with them). You can connect to Unmetered Exchange, EVIX and OPTiX just using one machine. You can also contact HE to get a free transit at Unmetered Exchange. For software, I'm using Bird, because I couldn't get OpenBGPd insert routes into my FIB. (I'm using OpenBSD, not the portable one)
- Vigdis 5y ago
  I was renting a VM from FRC but I had to let it go because I had loss and nothing was done to fix it :( Regarding the routes in the FIB, I had the problem as well. Hopefully claudio@'s answer may help you (provided you want to test again) https://marc.info/?l=openbgpd-users&m=161510661928340&w=2 https://marc.info/?l=openbgpd-users&m=161510661928340&w=2 (ctrl-f "1. Why the FIB isn't filled?")
alberth 5y ago
Does anyone have a simple to read & concise resource to educate myself on this topic. I understand what an ASN is at a super high level, but have to admit that the vast majority of the acromyns in this article were new to me. Any good resources to read on this subject?
- stingraycharles 5y ago
  I don’t have any resources, but as a hobby homelab / networking guy with too much Mikrotik hardware at home, this is what I learned: * There are a bunch of organizations responsible for distributing all IP ranges (i.e. ARIN and RIPE); * Each of these ranges they assign, get a unique ASN; * Once I have such an ASN, I can go to an ISP and say, “please broadcast this for me and route it to my network” * ISP will verify whether you speak the truth (hopefully); * They will add an entry in their route tables for your range, so that any traffic they receive is correctly routed to your port: * They will start broadcasting your routes, using protocols such as RIP; * They will then start broadcasting your IP ranges with their uplinks they peer with using BGP; * The internet has now learned how to reach your servers behind your IP ranges. This is a bit simplified, but in general is how I understand things work. Would love it if someone with more knowledge on the matter can elaborate a bit on the BGP broadcasting part, and how ISPs protect against bad actors / scammers / whatever. I know there are recent improvements in BGP security, but it all still seems very insecure to me, and humans are still a critical factor here? Additionally, I understand that BGP is necessary because typical route table entries being broadcasted using RIP etc are too narrow, too smal, and you want to be able to aggregate a whole bunch of routes into large blocks, which is what BGP is for? Did I get that right?
  - alberth 5y ago
    This is a huge help as a quick start on this subject. Thanks a lot. Question: so if I want to host a web application at hosting provider X, and I have my own ASN and want to use my own IP addresses - I assume then that the hosting provider needs to "support BGP"? Is that accurate?
    - 5y ago
Wheaties466 5y ago
The amount of overhead it takes to encrypt a full tunnel on an endpoint device is absolutely not worth it compared to GRE.
- stingraycharles 5y ago
  Yeah, network hardware is already a bottleneck most of the time, it’s much more effective to keep encryption out of the critical path there. Besides, I wonder what the threat model is; you will still want end-to-end encryption on top of it, so why bother with encrypting the traffic below as well?
deleted 5y ago
[deleted]
hungryforcodes 5y ago
What?
rootsudo 5y ago
I like how he mentioned the fixes he did to BSD and even though they were cosmetic and man pages fix. I always spot those small things, but I always feel silly submitting them on github since sometimes they feel so minor.
- throwawayboise 5y ago
  I think all of the BSDs appreciate thoughtful patches to improve the documentation. Most of the BSD devs are so deeply familiar with the OS that they do not have the perspective of an inexperienced user trying to read the docs and figure things out.
- kelp 5y ago
  OpenBSD tends to really value high quality man pages. However to submit a patch you have to email a diff to their mailing lists. They even prefer it inline vs as an attachment. The GitHub repo linked in the post is just a read only mirror, not used by the OpenBSD project for development.
therein 5y ago
Aside from a nice article, what a great domain.
yuuta 5y ago
I also use WireGuard to connect my home router to a VPS to get my home connected to my network. However, VPNs highly depend on the availability of your residential ISP, the speed and stability are not desirable, especially when I run some servers at home. Having a dedicated line with BGP would be good, but there's no such a thing I guess for residential users.
- butlerm 5y ago
  BGP is only necessary to advertise large blocks of provider independent address space. It would still go through your provider's routers however, with appropriate BGP filtering, unless you managed to use some kind of tunneling system. Unless you had a large block of provider independent space and were physically multi-homed to multiple providers, using BGP is more likely to cause problems than help. If you do not connect to multiple ISPs there is not really any reason to use BGP at all.
danrl 5y ago
I went through the same journey some years ago. Now sharing my ASN 57821 with friends. We use OpenBSD for edge routing validate RPKI as well. My motivation was that I couldn’t get IPv6 otherwise. So I ended up with an ASN, v4 and v6 address space from RIPE. Not the worst deal of my life and the source of tons of practical learning regarding networking.
southerntofu 5y ago
Glad to see mention of self-organized non-profit ISPs in France on HN homepage! I'm sure more people around here would be interested to learn how it works and maybe start their own, if you have english-speaking resources at hand :) Congratulations on your network setup! It's impressive for such a low budget
megous 5y ago
The only unholy thing I have to do by splitting /64 from my VPS into multiple subnets at home is to use DHCPv6 in my home net instead of SLAAC and use NDP proxy on the VPS. Probably cheaper and less bothersome than acquiring an ASN and setting up BGP. :)
errcorrectcode 5y ago
Be careful who you peer with. You might catch some pathogenic routes. Always use protection.
errcorrectcode 5y ago
FYI: https://ipv6.he.net https://ipv6.he.net Try that before delving into (E)BGP and peering.
walrus01 5y ago
I can understand the people here complaining about how it's "difficult" to get an ARIN ASN. But you don't really need an ASN unless you intend to be an ISP. If you want to be serious about having and using an ASN for its intended purpose, you're looking at a minimum of a couple hundred dollars a month anyways for a 1U system in colocation with a hosting company that will set up a router-to-router BGP relationship with you. Even if to start, just singlehomed to one IP transit provider sending you full tables. And ideally also a connection to an IX that exists at the same location. In US dollars, budget something like $2400-3000 a year to be serious about it what I described above... The ARIN minimum bar to meet is actually not that high - you need to register an LLC or the local equivalent in your state or province, though of course something like a C-corp is perfectly acceptable as well. Other corporate legal entities like a US state's 501c3 or a Canadian non-profit society or registered charity are also acceptable. They are not set up for individual persons to get an ASN. One of the main purposes of having an ASN is to be truly multihomed, to 2 or more transit providers sending you full BGP tables, and you announce your own IP space to your upstream(s), as small as at least one /24 of ipv4, and whatever size you have for ipv6 such as a /32, or /36 or /48, and to join peering exchanges. The number of individual persons who really need to have a BGP-to-BGP full tables relationship with more than 1 ISP are really small. If you need such a thing you are almost certainly some sort of corporate entity (see above) and doing the ARIN membership joining paperwork should be very straightforward.
- rahimnathwani 5y ago
  What if you have two separate ISPs at home (for redundancy) but want your home devices to have stable IP addresses? Couldn't you achieve this with a simpler/cheaper setup (a single Vultr server set up with their BGP service), and your home devices tunneling everything through it? And doesn't this use case warrant your own address space?
  - walrus01 5y ago
    Residential last mile ISPs are highly unlikely to offer bgp as a service. What you'll get is default route DIA. If you actually do find a cooperative fiber based last mile ISP that will set up bgp with you in your house, I'd be shocked if the service quotation for such a custom weird thing doesn't start from a floor price of $400 a month.
xnyanta 5y ago
>Well wg(4) doesn't support multicast, and ospf6d (and even eigrpd) needs it. You can do without buuuut... I tried and struggled with ospf6d, so sticking with bgpd was way easier. Not sure what magic I'm pulling off but I run OSPFv2 and OSPFv3 between 3 routers in a triangle using WireGuard and FRR on VyOS and it's working fine. I just set allowedIPs to everything on the interfaces to let the multicast address traffic go through.

311 points by zdw 5y ago

The Ipv6 pool is quite vast. Yet I am a little surprised that an individual can receive a /48 without much trouble: that a lot of IPs.

cbg0 5y ago
This is based on the recommendations in https://datatracker.ietf.org/doc/html/rfc3177 https://datatracker.ietf.org/doc/html/rfc3177 A relevant snippet: > This means that we feel comfortable about the prospect of allocating 178 billions /48 prefixes under that scheme before problems start to appear. To understand how big that number is, one has to compare 178 billion to 10 billion, which is the projected population on earth in year 2050
- cassianoleal 5y ago
  ~So that's a maximum average 17.8 IPs per person? Seems awfully low, considering people these days have multiple devices connected to the Internet at any given time: phone, watch, tablet, laptop(s), perhaps desktop(s), security cameras, IP phones, refrigerators, washing machines, TVs, video game consoles, I could go on...~ Edit: oh, wow I misread that completely. It's 17.8 /48 ranges per person on average. Yep, that should be enough for quite a while.
  - klysm 5y ago
    Every phone gets a /48!
chippiewill 5y ago
IIRC /64s are ideally meant to address to individual subnets (the bottom 64 bits aren't intended for routing) so it's only really 64k addresses in the conventional sense. /48 is actually a relatively standard allocation even for home connections, although /56 is more common.
- jagger27 5y ago
  I think what Rogers in Canada does is reasonable. They issue a /64 by default and a /56 if you set an assignment hint.
- hotpotamus 5y ago
  This was key to me understanding IPv6 - you don't really do variable length subnets. You just get a /48 and break that up into /64's. Because the space is so vast, you can do inefficient things with no chance of running into allocation issues like v4 and you don't have to mentally try and subnet a 128 bit address which maybe some can handle but hurts my head.
- kloch 5y ago
  The intent of the standard /48 end user size was to prevent residential ISP's from being stingy and only assigning one IP. They wanted homes to have as many publicly routable subnets as the would ever need or want. This was later revised to "/56 by default, /48 if you ask with no justification needed"
normaler 5y ago
I was surprised about that too. I got 1 static ipv4 Form my ISP and a /48 for IPv6. Not sure what to do with that, but still cool.
throw0101a 5y ago
> The Ipv6 pool is quite vast. Yet I am a little surprised that an individual can receive a /48 without much trouble: that a lot of IPs. A /48 is considered one "site" in current thinking. Since IPv6 subnets are /64, you have 16 bits between the /48 and the /64. This is the equivalent of using 10/8 for your network and using /24 IPv4 subnets: in both cases you can have upto 2^16 subnets. The main difference being that a /24 can have ~250 hosts, but a /64 IPv6 subnet can hold the equivalent of four billion Internets (2^32 * 2^32). But one of the selling points of IPv6 is reducing/eliminating the mental math about worrying about if you have "enough" addresses (and then carving things into /26, /30, etc).
- dsl 5y ago
  > A /48 is considered one "site" in current thinking. This is the really important part. As they continue to hand out IPv6 like candy, the minimum prefix length will get shorter. No ISP wants a hundred million+ routes in their routing table, so people will start to drop anything shorter than a /42, /40, /38, etc. until the table gets small enough and shunt everyone elses traffic off to Hurricane Electric or the like as a default route. People have this mindset of "we added a bunch of zeros, its an infinite resource now!" which is how we ended up in this mess to start with.
  - throw0101a 5y ago
    > This is the really important part. As they continue to hand out IPv6 like candy, the minimum prefix length will get shorter. Unlikely. > No ISP wants a hundred million+ routes in their routing table […] Too late. IPv4 are set to hit 1024K (2^20) in January 2024 at current trends: * https://blog.apnic.net/2021/03/03/what-will-happen-when-the-routing-table-hits-1024k/ https://blog.apnic.net/2021/03/03/what-will-happen-when-the-... Already close to 10^6: > I see 904560 IPv4 prefixes. This is 30 fewer prefixes than 6 hours ago and 416 more than a week ago. 59.10% of prefixes are /24. * https://twitter.com/bgp4_table https://twitter.com/bgp4_table * https://bgp.potaroo.net https://bgp.potaroo.net * https://www.cidr-report.org/as2.0/ https://www.cidr-report.org/as2.0/ Also: https://twitter.com/bgp6_table https://twitter.com/bgp6_table
jleahy 5y ago
I have a /29, there's a lot of IPv6.
zahllos 5y ago
Another point to consider is the IPv6 global routing table for internet routers. Suppose you decided to give end users /120s instead and those /120s were all routable on the internet. That means there are 120 bits of addressing just to find the network. There are 2^120 such networks in theory. If you could actually enumerate this, you'd be well on your way to bruteforcing AES128. In other words, this is just infeasible. By handing out /48s the routing table stays manageable. This is the smallest address block you can announce via BGP for this reason. Given the utter vastness of IPv6 we are also able to do things like carve out an entire /7, fc00::/7, for unique local addresses, and still tell people they shouldn't actually need these addresses at all. As to the actual process of getting a PI block, I think it is likely to involve some questions. A similar objection exists to handing out smaller than /48s: more people having their own block implies more routing entries. Much better if an existing provider carves you a /48 out of their allocation and routes you traffic. This is probably balanced against the fact that by requiring you to deal with an existing LIR and set up BGP (and your LIR won't want you to muck that up) the number of people who will actually do this just for fun is limited.
kloch 5y ago
The IETF IPv6 allocation scheme was crazy from the start. The standard subnet size for autoconfiguration purposes is /64 = 18 Billion Trillion addresses. Of course reality is that autoconfiguration is only marginally useful and lots of things like servers and infrastructure links are statically assigned or assigned through dhcpv6. Most of the time I assign /112's to server subnets to limit the risk of IPv6 neighbor discovery attacks. The standard /48 assignment size from ISP's to end-users was targeted mainly at stingy residential ISP's that were only assigning one IP per customer. They wanted any customer to have as many publicly routable subnets as they would ever need or want, but 65k is a lot of subnets. This was later updated to a default of /56 (256 subnets) but you could still get at /48 just for asking. It got even weirder on RIR to ISP portable allocations. One of the original schemes would have RIR's allocating a "TLA" /16 (!) to only a few mega ISP's. Most ISP's/hosts would have to get their "NLA" address blocks from and be subservient to the 800 pound gorillas in the industry. https://www.rfc-editor.org/rfc/rfc2450.txt https://www.rfc-editor.org/rfc/rfc2450.txt, https://www.rfc-editor.org/rfc/rfc2374 https://www.rfc-editor.org/rfc/rfc2374 This was loosened somewhat in 2000 with RFC2928 which designated a "Sub-TLA" /29 as the initial ISP size. This would open it up to many more potential registrants, at least in theory. https://www.rfc-editor.org/rfc/rfc2928.html https://www.rfc-editor.org/rfc/rfc2928.html The TLA/Sub-TLA/NLA system was abandoned in 2003 with RFC3587 after panic set in that nobody was deploying IPv6. https://www.rfc-editor.org/rfc/rfc3587.txt https://www.rfc-editor.org/rfc/rfc3587.txt. ARIN had only made 30 Sub-TLA assignments by the end of 2002. While RFC3587 wisely punted allocation policy to the RIR communities, the RIR's still had very restrictive policies inherited from RFC2450. Only around 2004 did they loosen to the point that ordinary networks could start requesting addresses. Google for example got their first IPv6 allocation in 2005. The problem is by then few networks were interested or just assumed they wouldn't qualify. When I got my first /32 in late 2004 there were less than 1000 routes in the global IPv6 table! Today there are ~135,000. These more permissive rules only applied to ISP's/Hosts. End user orgs of any size were not allowed to start requesting portable /48's directly from RIR's until 2006 after much debate in RIR communities and vocal objections from IETF members and certain large ISP's. Today router silicon and RIR policies have converged to a reasonably functional state. Too bad it took 20 years to get here or IPv6 might actually be on it's way to replacing IPV4. Instead that is still about 20 years away.

teddyh 5y ago

> I had to escape all the : in my shell Why? The colon character is not a special character in shell commands.

craftkiller 5y ago
The author uses OpenBSD which uses ksh as their default shell. I just ran ksh on Arch Linux and even though `touch foo:bar` worked, when I did tab-completion the shell escaped the colon anyway, making my command `touch foo\:bar`. I assume its related to this, but I've never used ksh before so I can't explain the difference in the tab-completion behavior.
- Arnavion 5y ago
  bash is the same - `:` isn't special but tab-completion escapes it. Tab-completion is probably being conservative about escaping it in case you're using it as a path separator, like when setting a `$PATH`-like var.
  - dsl 5y ago
    The shell doesn't know what the underlying filesystem is. : is special on NTFS for alternative data streams, and I think HFS+ named forks? (The mac syntax is foo/..namedfork/rsrc, I don't know how that translates to linux) "foo\:bar" is a file name with a colon in it. "foo:bar" is an ADS named bar on the file foo.
    - Arnavion 5y ago
      Maybe. The motivation may have been lost to history. bash added `:` to the list of completion word break characters with bash 2.0 [1][2]. Bash 2.0 was released in 1996. NTFS-3G started in 2006 but it was "forked from the Linux-NTFS", and I can't find any history for this project. NTFS itself was released in 1993. HFS+ seems to have been released in 1998, so it can't be the reason. But even regular HFS had resource forks and it was released in 1985. No idea where to look in ksh's source. [1]: https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee65b8f368e9653e1380b#diff-b8fb328cf74196c51b622571b2c765c857d4c5e766b0ce4b550f77280892e64eL239 https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee6... [2]: https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee65b8f368e9653e1380b#diff-b8fb328cf74196c51b622571b2c765c857d4c5e766b0ce4b550f77280892e64eR129 https://github.com/bminor/bash/commit/ccc6cda312fea9f0468ee6...

johnklos 5y ago

It's lovely to read about people doing things that're out of the ordinary to learn & explore because it's what they want to do. We have too many people who don't like stuff like this, who say it's "too hard", and that people shouldn't do hard things (just see any thread on the Internet about people running their own email servers). It's wonderful to see someone sharing something non-trivial in a way that makes it ever so slightly more attainable to the rest of us.

eatonphil 5y ago
Setting up an email server once sounds awesome. Coding an email server system from scratch also sounds awesome. But running an email server indefinitely doesn't sound super fun.
- johnklos 5y ago
  I've been running email servers since the '90s, and there's surprisingly little work that needs to be done once it's up and running.
  - eatonphil 5y ago
    Sure, you and many people. I'm saying there's an in-between level of enthusiasm where you do want to learn new things and experiment but without getting yourself into any long-term commitments. I'm sure you'd agree that it's also easier to scale your experiments/interest if you only do small experiments rather than go all the way in every dimension.
  - excitom 5y ago
    I ran email servers for my personal domains in the 90s and early 2000s. What eventually caused me to move everything to gmail was the massive pain of dealing with spam and trying to keep up with the latest spam filtering, which is essential.
    - georgyo 5y ago
      Rspamd has made that trivial for a number of years. It's an amazing tool.

holdenk 5y ago

If anyone is in the bay area and wants to run physical hardware and peer with folks, FCIX down in the HE facility in Fremont is very welcoming to small networks and doesn't have port fees (although you do need to be in the HE facility).

Karrot_Kream 5y ago
Oh wow that's fantastic! Are you referring to https://fcix.net/ https://fcix.net/ ? Thanks so much for the tip. Do you know how much it costs to grab a dedicated box in the HE Fremont facility?
- holdenk 5y ago
  So they don't do dedicated boxes (sadly), they rent by the rack (including 1gb of transit + 15A power). https://he.net/colocation.html https://he.net/colocation.html has the info. If you just want a dedicated box that isn't drawing a lot of power I've probably got some room in my rack and we can chat (and I imagine some of the other BGP for fun group folks are in similar positions).
  - Karrot_Kream 5y ago
    Sure I'd be interested! Let me know if there's contact info on your site or some other method of contact. I'm fine with Email or Matrix.
  - errcorrectcode 5y ago
    Ah, blast from the past. HE brought us T-1's, T-3's, and IPv6 transit way back. Equinix owns PAIX on 529 Bryant St. as SV8. It's interesting how racks still aren't so much volume-, mass-, or network bandwidth-/transit-constrained, but power-constrained.
    - walrus01 5y ago
      It's very easy to run out of cumulative watts of grid power feed, cooling capacity and generator/UPS/power backup capacity in a facility before you run out of floor space in a datacenter. Even with what is now a "small" 5kW thermal power budget per 44U cabinet, measuring 24 inches wide x 48 inches deep, you can easily exceed that 5kW long before you physically fill the cabinet with servers.

mlyle 5y ago

The thing that really surprises me here is that he found cheap hosts willing to run a BGP session with him. Who offers this in North America? He mentions one of his VMs is in Toronto, but not with whom... edit: Looks like https://xenyth.net/products/vps-hosting/ https://xenyth.net/products/vps-hosting/ Xenyth Cloud. Curious if there's any VM hosts in e.g. California that would do this.

rmkrmk 5y ago
I've done this as well recently, and the list at bgp.services helped a lot. Vultr is a good example, they have locations across the globe.
- ssabetan 5y ago
  +1 for Vultr. I currently use them to announce a /24.
  - therein 5y ago
    I wanted to like Vultr. I gave them a fair chance but then they changed the IP on my VPS without telling me. It changed from a Swedish IP to an American IP overnight and left me confused in every way. Everything broke and support wasn't willing to explain what happened. It felt like they decided my account was suspicious and they routed all my traffic through some firewalled/monitored egress to inspect my traffic. I couldn't even SSH in, I had to go the the dashboard and grab the IP they had reassigned to my host.
    - capableweb 5y ago
      Really? That sounds extremely shitty. As a customer of Vultr, it makes me a bit scared, but during the 5~ or something years I've used them, that never happened to me. You'd be doing everyone a service by continuing to dig out a reason for why that happened, and if they don't provide it, lambast them over social media or something so they do provide some sort of justification.

DanAtC 5y ago

They hit on a point I rarely see mentioned: ARIN makes acquiring assets as an individual prohibitively difficult and expensive. Especially for IPv6 where European LIRs hand them out like candy e.g. https://clients.stratagem.host/cart.php?gid=14 https://clients.stratagem.host/cart.php?gid=14

techsupporter 5y ago
Having dealt with ARIN, RIPE, and APNIC quite a lot over the years, this is bang on. APNIC is fairly ambivalent. ARIN expects everyone to be a formally organized entity. Only RIPE seems to truly get that an individual person could be doing something on their own behalf, probably because forming a proper limited company was such a huge paperwork endeavor in most of RIPE's service area. ARIN really needs the concept of sponsoring LIRs, because right now everyone has to deal directly with ARIN and that means ARIN has to fit their policies to try to fit everyone. While it's technically possible to get IPv6 PI space from ARIN as an individual person end user, and it's gotten easier than in the past, it still looks really daunting to the hobbyist or small network operator. As for LIRs handing them out like candy, I joined RIPE as an LIR and RIPE will allocate an IPv6 /29 to an LIR for the low cost of zero Euro (well, except for the membership, which is a bit spendy) and that's 524,288 /48s so no reason for me to not slap up a web page and see if I can earn back a few of the EUR I will be paying to RIPE. I'm glad people do it for the pittance we see here.
- traceroute66 5y ago
  > RIPE will allocate an IPv6 /29 to an LIR Last time I checked it was a /32. More than that requires justification (and potential Euro payments depending on context/cirumstances). Edit to add: Grrr... I see there was a policy change after $myco received its IPv6 allocation [1]. Typical. ;-( [1] https://www.ripe.net/participate/policies/proposals/2012-10 https://www.ripe.net/participate/policies/proposals/2012-10
- boris 5y ago
  I am curious, what happens if I get a /48 from you and some time later you decide you don't wish to be a LIR anymore. Do I get to keep my /48 somehow (i.e., does RIPE guarantee this)?

yuuta 5y ago

I got my first ASN in April, 2021 from a LIR. It was an amazing experience. If you live in the west part of North America like me, you can get a IX VPS at FreeRangeCloud (disclaimer: I'm not affiliated with them). You can connect to Unmetered Exchange, EVIX and OPTiX just using one machine. You can also contact HE to get a free transit at Unmetered Exchange. For software, I'm using Bird, because I couldn't get OpenBGPd insert routes into my FIB. (I'm using OpenBSD, not the portable one)

Vigdis 5y ago
I was renting a VM from FRC but I had to let it go because I had loss and nothing was done to fix it :( Regarding the routes in the FIB, I had the problem as well. Hopefully claudio@'s answer may help you (provided you want to test again) https://marc.info/?l=openbgpd-users&m=161510661928340&w=2 https://marc.info/?l=openbgpd-users&m=161510661928340&w=2 (ctrl-f "1. Why the FIB isn't filled?")

alberth 5y ago

Does anyone have a simple to read & concise resource to educate myself on this topic. I understand what an ASN is at a super high level, but have to admit that the vast majority of the acromyns in this article were new to me. Any good resources to read on this subject?

stingraycharles 5y ago
I don’t have any resources, but as a hobby homelab / networking guy with too much Mikrotik hardware at home, this is what I learned: * There are a bunch of organizations responsible for distributing all IP ranges (i.e. ARIN and RIPE); * Each of these ranges they assign, get a unique ASN; * Once I have such an ASN, I can go to an ISP and say, “please broadcast this for me and route it to my network” * ISP will verify whether you speak the truth (hopefully); * They will add an entry in their route tables for your range, so that any traffic they receive is correctly routed to your port: * They will start broadcasting your routes, using protocols such as RIP; * They will then start broadcasting your IP ranges with their uplinks they peer with using BGP; * The internet has now learned how to reach your servers behind your IP ranges. This is a bit simplified, but in general is how I understand things work. Would love it if someone with more knowledge on the matter can elaborate a bit on the BGP broadcasting part, and how ISPs protect against bad actors / scammers / whatever. I know there are recent improvements in BGP security, but it all still seems very insecure to me, and humans are still a critical factor here? Additionally, I understand that BGP is necessary because typical route table entries being broadcasted using RIP etc are too narrow, too smal, and you want to be able to aggregate a whole bunch of routes into large blocks, which is what BGP is for? Did I get that right?
- alberth 5y ago
  This is a huge help as a quick start on this subject. Thanks a lot. Question: so if I want to host a web application at hosting provider X, and I have my own ASN and want to use my own IP addresses - I assume then that the hosting provider needs to "support BGP"? Is that accurate?
  - 5y ago

Wheaties466 5y ago

The amount of overhead it takes to encrypt a full tunnel on an endpoint device is absolutely not worth it compared to GRE.

stingraycharles 5y ago
Yeah, network hardware is already a bottleneck most of the time, it’s much more effective to keep encryption out of the critical path there. Besides, I wonder what the threat model is; you will still want end-to-end encryption on top of it, so why bother with encrypting the traffic below as well?

deleted 5y ago

[deleted]

hungryforcodes 5y ago

What?

rootsudo 5y ago

I like how he mentioned the fixes he did to BSD and even though they were cosmetic and man pages fix. I always spot those small things, but I always feel silly submitting them on github since sometimes they feel so minor.

throwawayboise 5y ago
I think all of the BSDs appreciate thoughtful patches to improve the documentation. Most of the BSD devs are so deeply familiar with the OS that they do not have the perspective of an inexperienced user trying to read the docs and figure things out.
kelp 5y ago
OpenBSD tends to really value high quality man pages. However to submit a patch you have to email a diff to their mailing lists. They even prefer it inline vs as an attachment. The GitHub repo linked in the post is just a read only mirror, not used by the OpenBSD project for development.

therein 5y ago

Aside from a nice article, what a great domain.

yuuta 5y ago

I also use WireGuard to connect my home router to a VPS to get my home connected to my network. However, VPNs highly depend on the availability of your residential ISP, the speed and stability are not desirable, especially when I run some servers at home. Having a dedicated line with BGP would be good, but there's no such a thing I guess for residential users.

butlerm 5y ago
BGP is only necessary to advertise large blocks of provider independent address space. It would still go through your provider's routers however, with appropriate BGP filtering, unless you managed to use some kind of tunneling system. Unless you had a large block of provider independent space and were physically multi-homed to multiple providers, using BGP is more likely to cause problems than help. If you do not connect to multiple ISPs there is not really any reason to use BGP at all.

danrl 5y ago

I went through the same journey some years ago. Now sharing my ASN 57821 with friends. We use OpenBSD for edge routing validate RPKI as well. My motivation was that I couldn’t get IPv6 otherwise. So I ended up with an ASN, v4 and v6 address space from RIPE. Not the worst deal of my life and the source of tons of practical learning regarding networking.

southerntofu 5y ago

Glad to see mention of self-organized non-profit ISPs in France on HN homepage! I'm sure more people around here would be interested to learn how it works and maybe start their own, if you have english-speaking resources at hand :) Congratulations on your network setup! It's impressive for such a low budget

megous 5y ago

The only unholy thing I have to do by splitting /64 from my VPS into multiple subnets at home is to use DHCPv6 in my home net instead of SLAAC and use NDP proxy on the VPS. Probably cheaper and less bothersome than acquiring an ASN and setting up BGP. :)

errcorrectcode 5y ago

Be careful who you peer with. You might catch some pathogenic routes. Always use protection.

errcorrectcode 5y ago

FYI: https://ipv6.he.net https://ipv6.he.net Try that before delving into (E)BGP and peering.

walrus01 5y ago

I can understand the people here complaining about how it's "difficult" to get an ARIN ASN. But you don't really need an ASN unless you intend to be an ISP. If you want to be serious about having and using an ASN for its intended purpose, you're looking at a minimum of a couple hundred dollars a month anyways for a 1U system in colocation with a hosting company that will set up a router-to-router BGP relationship with you. Even if to start, just singlehomed to one IP transit provider sending you full tables. And ideally also a connection to an IX that exists at the same location. In US dollars, budget something like $2400-3000 a year to be serious about it what I described above... The ARIN minimum bar to meet is actually not that high - you need to register an LLC or the local equivalent in your state or province, though of course something like a C-corp is perfectly acceptable as well. Other corporate legal entities like a US state's 501c3 or a Canadian non-profit society or registered charity are also acceptable. They are not set up for individual persons to get an ASN. One of the main purposes of having an ASN is to be truly multihomed, to 2 or more transit providers sending you full BGP tables, and you announce your own IP space to your upstream(s), as small as at least one /24 of ipv4, and whatever size you have for ipv6 such as a /32, or /36 or /48, and to join peering exchanges. The number of individual persons who really need to have a BGP-to-BGP full tables relationship with more than 1 ISP are really small. If you need such a thing you are almost certainly some sort of corporate entity (see above) and doing the ARIN membership joining paperwork should be very straightforward.

rahimnathwani 5y ago
What if you have two separate ISPs at home (for redundancy) but want your home devices to have stable IP addresses? Couldn't you achieve this with a simpler/cheaper setup (a single Vultr server set up with their BGP service), and your home devices tunneling everything through it? And doesn't this use case warrant your own address space?
- walrus01 5y ago
  Residential last mile ISPs are highly unlikely to offer bgp as a service. What you'll get is default route DIA. If you actually do find a cooperative fiber based last mile ISP that will set up bgp with you in your house, I'd be shocked if the service quotation for such a custom weird thing doesn't start from a floor price of $400 a month.

xnyanta 5y ago

>Well wg(4) doesn't support multicast, and ospf6d (and even eigrpd) needs it. You can do without buuuut... I tried and struggled with ospf6d, so sticking with bgpd was way easier. Not sure what magic I'm pulling off but I run OSPFv2 and OSPFv3 between 3 routers in a triangle using WireGuard and FRR on VyOS and it's working fine. I just set allowedIPs to everything on the interfaces to let the multicast address traffic go through.