6 ms·
The lack of a good command line way to sort IPv6 addresses
- sargstuff 1y agoIPv6 addresses are typically written in canonical text representation recommended by RFC 5952[0]. 1) Regular expression / seamingly lost art of sprintf formatting are some methods to normalize 001:db8::2:1 to something usable for sorting aka 2001:db8:0000:0000:0000:0000:0002:0001. Perhaps restoring to rfc 5952 format when printing sorted results. 2) Modify hex to 'sortable utf-16 characters', modify back post sort[1] 3) avoid utf-8 / utf-16 issues, use relevant python libraries to handle ipv4/ipv6[2][3],[4]: ip2n < file | sort -n | n2ip ---------------------------- [0] : https://datatracker.ietf.org/doc/html/rfc5952 https://datatracker.ietf.org/doc/html/rfc5952 [1] : https://stackoverflow.com/questions/5797369/unix-sort-utility-use-hexadecimal-byte-value-as-delimiter https://stackoverflow.com/questions/5797369/unix-sort-utilit... [2] : https://stackoverflow.com/questions/75522231/how-to-sort-ipv4-and-ipv6-addresses-together-in-a-dataframe https://stackoverflow.com/questions/75522231/how-to-sort-ipv... [3] : https://www.geeksforgeeks.org/convert-ip-address-to-integer-and-vice-versa/ https://www.geeksforgeeks.org/convert-ip-address-to-integer-... [4] : https://ipfind.com/blog/how-to-convert-ip-address-to-integer-or-long/ https://ipfind.com/blog/how-to-convert-ip-address-to-integer...
- 0points 1y ago1. Convert to numeric representation 2. Sort 3. Convert back
- schoen 1y agoSuch a program in Python could be python3 -c 'import ipaddress, sys; print("\n".join(sorted(ipaddress.IPv6Address(x).exploded for x in sys.argv[1:])))' It takes the IP addresses to be sorted on the command line. Or, re-abbreviating them by removing zeroes and attempting to use :: where possible: python3 -c 'import ipaddress, sys; print("\n".join(str(ipaddress.IPv6Address(y)) for y in sorted(ipaddress.IPv6Address(x).exploded for x in sys.argv[1:])))' Both of these versions will crash if given input that isn't syntactically valid as an IPv6 address.
- fragmede 1y agoIt's ipv6. Seems like the "standard" thing to do would be to write and name a utility called sort6 which properly handles ipv6 addresses.
- Gabrys1 1y agoI imagine you wrote this sarcastically, but I do like this a lot!
- fragmede 1y agoThe original sort (and ping, and traceroute) could just have been updated to handle the newer format, rather than creating new utilities, was my non-sarcastic point. :)
- thesuitonym 1y agoTrue of ping and traceroute, but sort is not a network utility, so editing it to specifically handle IPv6 addresses is kind of a weird idea. Not necessarily a bad idea, but it makes sense that nobody has done it. Probably better in this case would be to have a small program that expands IPv6 addresses that you could then pipe into sort. That would be more in-keeping with the Unix Way™.
- pantalaimon 1y agoHow often do you need to sort IP addresses to begin with?
- j16sdiz 1y agoWhen you have lots of addresses and subnets, and want to see which addresses goes into what subnet
- fragmede 1y agoespecially since ipv6 can be multi-homed. on corporate networks you ended up with difficult configurations and need to figure out why the packets are going the wrong way frequently.
- ggm 1y agofor all addresses, v4 and v6 this is what I used to do 1) convert to a non-space-zeros-compressing hex string 2) sort on the hex string 3) convert back through inet_ntop() Only a minor variant needed to deal with prefix/len sort order.
- wpm 1y agoAlmost as if using hex values for IP addresses was a bad idea.
- oldnetguy 1y agoAgreed. By using decimal, you can explain to the average person what an address is and be able to work with them. IPv6 totally eliminates that. There is an argument that end users and consumers shouldn't need to concern themselves with this but in the field that this is not true.
- fragmede 1y agoHow often do consumers (ie not network professionals) need to do CIDR or anything more advanced than read off an address? Is reading 64512 colon 65152 colon colon 7237 that much easier over a bad phone connection than fc00:fe80::1c45? By the time you're that deep with someone non-technical having them read an address over the phone, you've already lost
- miyuru 1y agoI don't think any IP version was made in with sorting as a feature, its just a by product of the representation. This post also highlights a major thing I discovered when deploying and using IPv6, which is that you don't "Lift and shift" IPv4 to IPv6. This is one of the reason its hard to deploy, because people cannot use the same IPv4 concepts to IPv6. For unknown reason they do, they will find the same problem they had with v4.
- Ekaros 1y agoAlso IPv4 does have some allowed weird notations. Like 127.1 is equal to 127.0.0.1 or can represent it as decimal number. Or hex notation. Or some tools support octal... Which some also do not sort nicely. Specially when combined.
- monster_truck 1y agoIf you think these are weird I have bad news about ipv6
- mystified5016 1y agoAnecdata but I have never seen shortened IPv4 notation. I've only ever seen the full decimal notation.
- andix 1y agoThose representations for 127.0.0.1 work too, just try to ping them: 0177.1 (octal, 2 stripped 0s, decimal one) 017700000001 (octal) 2130706433 (decimal) 127.0.0x1 0x7f.0x0.0x0.0x1 127.000.000.001
- nimbius 1y agothe sheer size of ipv6 alone means a lot of the 'old ways' get left behind. "sorting by ip" for example is eschewed in favor of router advertisements where hosts pick their own subnet delegations. one of the ways you track all of this is using a dhcp6 server with dns updates turned on. the subnet delegation is picked up by the client, which in turn pings dhcp6 and reports its IP for update in the DNS server.
- deleted 1y ago[deleted]
- whalesalad 1y agoI have yet to use ipv6 and I will try to make it to my grave without adopting it.
- neilalexander 1y agoWeird take, but you do you.
- floating-io 1y agoNot really that odd. It's easy to live without IPv6 until someone creates the IPv6-only killer app, and that makes it... not worth the hassle for most folks. I'm in a moderate-to-high technology area in the grand scheme. They laid fiber here roughly two years ago, and lit it about a year ago, whereupon I immediately subscribed. They don't offer IPv6. At all. That should tell you just how unimportant IPv6 is perceived as outside of its core cadre of proponents. Note that this is not a commentary on how important it actually is; just on perception. In short, nobody cares.
- patchtopic 1y agoThe 85% of French and 75% of India adopting it don't care, they just use it :-) https://www.google.com/intl/en/ipv6/statistics.html#tab=per-country-ipv6-adoption https://www.google.com/intl/en/ipv6/statistics.html#tab=per-...
- dns_snek 1y agoMore interestingly, it looks like a year from now IPv6 is going to make up the majority of Google's global traffic. Cloudflare reports slightly lower 40% IPv6 adoption globally https://radar.cloudflare.com/adoption-and-usage https://radar.cloudflare.com/adoption-and-usage
- WorldMaker 1y agoMost consumer networks are IPv6 by default. Most cellular networks are IPv6-only/IPv6-native today with big DNS64/NAT64 gateways to the IPv4 web. Most consumers don't know or care, they just use IPv6, daily. A lot of the holdouts on IPv4-only traffic are corporate traffic. A lot of corporate traffic acts like it wants a separate internet anyway, and a lot of companies hoarded IPv4 spaces for long enough that they don't see the increasing costs of IPv4 addresses hit their bottom lines yet enough to care. It's a fascinating dynamic. Maybe corporations will just buy the remaining IPv4 internet entirely and keep it as their own separate but not equal network.
- xnorswap 1y agoAt least IPv6 addresses aren't "middle-endian" like UUIDs can be.
- WorldMaker 1y agoUUID sort orders get wild indeed. I have been down that rabbit hole.
- blueflow 1y agoThis is not a UUID problem, this is a Microsoft problem from the 90s. Just don't use Microsoft software (</s>) and use big endian as specified by the standard.
- WorldMaker 1y agoIt is a general UUID specification problem. The dashes represent a struct breakdown. That struct has internal endian issues. That struct is also weirdly laid out in a "made sense at the v1 time way" that doesn't make sense for versions after 1. Why is the version number in the middle? Why is the relatively static "Machine ID" at the end? If you were trying to cluster your sort by machine, you have to sort things "backwards". That's what SQL Server did, and why you might blame it on being a Microsoft problem, trying to avoid clustered index churn by assuming GUIDs were inserted by static Machine IDs. That assumption broke hard in later Versions of UUID when "Machine ID" just became "random entropy". But the idea to sort like that in the first place wasn't wrong for v1, it had a good sense to it. Just like it makes sense to sort v7 UUIDs by timestamp to get mostly "log ordered" clustered indexes. At least there the sort data is all up front, but it crosses "struct field" boundaries if you are still relying on the v1 chunking. (Ultimately UUID v1 was full of mistakes that we all will keep paying for.) For the record it is Java with the worst possible UUID sort algorithm, sorting parts of it as signed numbers: https://devblogs.microsoft.com/oldnewthing/20190913-00/?p=102859 https://devblogs.microsoft.com/oldnewthing/20190913-00/?p=10... (Friends don't let friends use Java. /s)
- zX41ZdbW 1y agoclickhouse-local is very handy for these tasks: $ echo "2a00:1450:400e:80c::200e 2606:4700::6810:85e5 2a03:2880:f145:82:face:b00c:0:25de 2603:1030:c02:8::14 2603:1030:20e:3::23c 2603:1030:b:3::152 2603:1020:201:10::10f 2603:1010:3:3::5b" | ch --structure 'ip IPv6' --query "SELECT ip FROM table ORDER BY ip" 2603:1010:3:3::5b 2603:1020:201:10::10f 2603:1030:b:3::152 2603:1030:20e:3::23c 2603:1030:c02:8::14 2606:4700::6810:85e5 2a00:1450:400e:80c::200e 2a03:2880:f145:82:face:b00c:0:25de
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- yuliyp 1y agoIf you're sorting data on a command line, you probably have some matter of control over how it's generated. So just fully-expand it and then text sorting just works. Pretty simple?
- neuroelectron 1y agoTrivial Task Has No Standard
- andix 1y agoI'm creating all those tiny shell helper scripts now with AI. One of the things AI is really good at. prompt: create a bash command to sort a text file of ipv6 adresses per line. sort numerically (128 bit representation) but keep the full line as originally formatted result: while read -r ip; do printf '%032x %s\n' \ "$(python3 -c 'import ipaddress,sys; print(int(ipaddress.IPv6Address(sys.argv[1])))' "$ip")" \ "$ip" done < ipv6.txt | sort | cut -d' ' -f2-
- kstrauser 1y agoThat’s launching Python once per address. Why not just pipe it into a short Python script at that point?