8 ms·
19 random digits is not enough to uniquely identify all human beings
- geoah 3y agoThe actual title is “19 random digits is not enough to uniquely identify all human beings”. The tl;dr is that you need at least 30 digits so it’s safe to assign a random number to a person with a close to zero probability of already being assigned. I’m not really sure why the author is talking about 19 digits. Must be a reference to something I guess but I really don’t know what. Also this doesn’t mention for how long this is valid for. New people keep getting born, so at some point 30 numbers won’t be enough unless we don’t care about reuse of the ones where people might be long gone.
- _0ffh 3y agoHmmm, 64 bits happens to work out as around 19.27 decimal digits. Maybe there's a connection here, although due to the nature of digits 19.27 effectively means up to 20 digits.
- version_five 3y agoThat's a good guess. He links to a related discussion about 64-bit numbers so presumably that's what he was thinking https://lemire.me/blog/2019/12/12/are-64-bit-random-identifiers-free-from-collision/ https://lemire.me/blog/2019/12/12/are-64-bit-random-identifi...
- aaron695 3y ago[dead]
- svachalek 3y agoAgree on the 64 bit guess. And note that digits are exponential so going to 38 digits (128 bits) can account for 10 million times as many people as 30.
- travisjungroth 3y agoI can’t say what inspired 19 in the mind of the author, but the thing on the page is that 19 is the tipping point. It goes from 96% collision at 19 to 27% at 20. It’s the most amount of digits where you’ll probably have a collision.
- andsoitis 3y agoUUID to the rescue. https://en.wikipedia.org/wiki/Universally_unique_identifier https://en.wikipedia.org/wiki/Universally_unique_identifier It: - is a standard - adoption is widespread in computing platforms - addresses the birthday problem (in v4, chances of a collision in a 103 trillion set is 1 in a billion)
- UncleEntity 3y ago> addresses the birthday problem (in v4, chances of a collision in a 103 trillion set is 1 in a billion) Does that mean if you randomly assign ID numbers to 8 billion people you’d get 8 collisions?
- themoonisachees 3y agoIt means that if you were to assign an ID to 103 trillion objects a billion times, the average amount of times you would get a collision at all is 1.
- hollasch 3y agoThe UUID scheme uses random digits. The article states that 19 random decimal digits (about 63 bits) yields a collision probability of 96%. More digits lowers the probability of collision to 0.0003% at 25 decimal digits (83 bits). So yeah, UUID is great, but only because it uses 128 random bits, which is pretty much exactly along the lines of what the article is asserting.
- nathanwh 3y agoPedantic, but UUID v4 (which is the random one) only has 122 bits of randomness, 6 bits are used to identify the version and variant. https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random) https://en.wikipedia.org/wiki/Universally_unique_identifier#...
- toast0 3y ago> Suppose that you assigned everyone an 19 digit number. What is the probability that two human beings would have the same number? If I assigner the numbers, I would carefully print 8 billion or so slips with different numbers, and place them in an urn. For each person, I'd draw from the urn, and then the probability of me assigning the same number to two humans is related to IT failures during generation. How many dollar bills share a serial number unintentionally?
- hollasch 3y agoA random-number scheme provides a way for any entity to generate an identifier without collaboration (this is why GUIDs are so popular). Your scheme, yes, provides a way to assign guaranteed unique identifiers, but at the cost of a centralized generator — both a performance and an authority bottleneck.
- Waterluvian 3y agoLet’s be pedantic: neither the quoted sentence nor the response said anything about random.
- IIAOPSW 3y agoHash the name and the time of birth. Done.
- SideQuark 3y agoHash collisions exist.
- throwawaytemp29 3y agoEleanor and Fake Elaneanor would disagree, although that was time of death.
- riffic 3y agonames are not that easy though. https://shinesolutions.com/2018/01/08/falsehoods-programmers-believe-about-names-with-examples/ https://shinesolutions.com/2018/01/08/falsehoods-programmers...
- armchairhacker 3y agoWhy 19 is important: log(2^63) ~= 19, which means a random 64-bit integer is not long enough to uniquely identify all human beings. A 128-bit integer or UUID is
- SideQuark 3y ago2^63 allows giving each of the 8 billion people on earth over a billion numbers each with no repeats. 19 digits is plenty, since it only takes 10 digits to count all the people.
- FabHK 3y agoTo elaborate: There are about 8 * 10^9 people, or nearly 10^10. So, if you allocate numbers sequentially with a central authority, 10 digits would be enough. The birthday paradox says that probability of collision with random identifiers becomes very big (a half or so) as soon as the number of instances approaches the square root of the number of available identifiers. In other words, if we used 20 digits, the birthday paradox rule of thumb would predict a probability of collision of around a half. (That is borne out by Lemire's table, which states 27%, which is close enough given the approximations here). The solution proposed here is to use at least 30 digits; I wonder whether that is a good rule of thumb in general: to avoid collisions with a N-bit hash function, do not hash more than 2^(N/3) elements. At any rate, hashing the 2^33 humans with 128 bits is good enough (probability of collision < 1e-18), and with 256 bits definitely.
- deleted 3y ago[deleted]
- qotgalaxy 3y ago[dead]
- photochemsyn 3y agoThis is the most dystopian thing I've read in a while. Prison Planet Earth, where everyone is uniquely identified by a number, perhaps surgically implanted on an RFID tag? How much compute resource would be needed to create an AI minder for each and every human on top of that, constantly updating their social credit score on a daily basis?
- version_five 3y agoIt's a short post by a well know computer scientist about the birthday paradox. It has nothing to do with that stuff.
- boringuser2 3y agoJust to note, an explicit social credit score to many would be more preferable and transparently accessible than the the data widely collected and used by companies to gauge your social credit in the west: bank records, employment history, credit, rental history, civil court records, etc, all freely available with common reporting software.
- OkayPhysicist 3y agoIf you don't live in the extreme 3rd world, you probably already have a unique identifier in the form of "Country or Territory+ID number". It turns out relying on names when billions of people are using the same list of a couple hundred names is troublesome for record keeping.
- acyou 3y agoWe are indeed happily discussing how to assign each human a unique serial number, without too much discussion of why or if it should be done.
- asdajksah2123 3y agoYour reasoning logic, applied consistently, would consider almost everything we have today dystopian. For example: "Using the Hindu-Arabic numeral system instead of the Roman numeral system is the most dystopian thing I've heard about in a while. Prison Planet Earth, where everyone is uniquely identified by a number, perhaps surgically implanted on an RFID tag? How much compute resource would be needed to create an AI minder for each and every human on top of that, constantly updating their social credit score on a daily basis?" It's the slippery slope argument taken to an extreme. The way to fight the slippery slope fallacy is to fight against the things that may be dystopian/bad. Not argue that anything that is good/neutral is bad because it may potentially make some minor dystopian/bad outcome slightly more likely. If you do want to make such an argument, then you have to weigh the pros vs the cons. And it's very hard to see how a word problem to illustrate the Birthday Paradox comes anywhere close to the bad outweighing any possible benefit from it.
- asow92 3y agoSpeaking of birthdays, what is the probability of two people with the same 19 digit number colliding with the same birthday as well?
- travisjungroth 3y agoGiven they’ve already collided on the 19 digit numbers, then it’s just the probability of having the same birthday: 1/365, smoothing over the rough edges of the real world. If we’re not taking them as already collided, you multiply the two probabilities since they’re independent. Edit: I realize you’re probably asking a more complex question I don’t yet know the answer to. Like a double birthday paradox.
- asow92 3y agoTo your edit, yes, and there would certainly still be a lot of collisions.
- travisjungroth 3y agoMy second answer is a lower bound on the correct answer and extremely close to correct. Think of finding the ID collision first. The probability of having at least one ID collision where they also share the same birthday is pr(2 matching ids) * pr(birthday match in 2 people) + pr(3 matching ids) * pr(birthday match in 3 people)... The probability of having 3 matching IDs is extremely low. Assume you already have a match of size 2. Now you're looking for 1/10^19 match with ~1/10^10 draws. Not good odds. So we can ignore everything after the ID match of size 2, and get very close with 96% * 1/365 or 0.26%.
- zeroonetwothree 3y agoIt just means that p = 1/(365 * 10^19). So comparable to a 21.5 digit number or so
- tanbog5 3y agoGive everyone an ip6 address?
- hgsgm 3y agoHN's digit removal algo is getting out of hand.
- DiggyJohnson 3y agoGreat point, I was internet frustrated for 5 seconds by the title when I came to this thread.
- bee_rider 3y agoIt is just getting philosophical. I’m not just a number, dude.
- tptacek 3y agoIt's doing the same thing it's always done. If you mail hn@ycombinator.com instead of complaining here, they'll change the title.
- corysama 3y ago> If you want the probability to be effectively zero, you should use 30 digits or so. You really don't need it to be effectively zero. Just significantly less than other sources of error. For 8 billion people, a 3% chance of there being single error somewhere on the whole planet is pretty good on the scale of issues that the QA team needs deal with. Especially when the fix is: Roll a second random number for that single unlucky person on Earth. That's pretty easily auto-detected and auto-fixed at generation time. I worked on a big game that used 32-bit hashes of asset names even though we expected to get on the order of 10 collisions across the dataset. The solution was to detect collisions and tell artists to tweak their file names. Happened about 10 times over the course of many years and hundreds of thousands of assets.
- dboreham 3y agoHuman-implement hash bucket overflow.
- thfuran 3y agoIf everyone who's generating the numbers can query a global centralized database, why would they be random in the first place? Isn't the whole point of a system like uuid (which this seems to be leaning in the general direction of) to avoid that sort of thing being necessary?
- m3kw9 3y agoI’d just assign an incremental number 9 digits
- bee_rider 3y agoThen you would have to coordinate between everybody issuing IDs. Of one option of course would be to provide different ID ranges to different issuing authorities (a la ip addresses).
- psaipetc 3y agoThe Birthday Paradox and Microsoft GUIDs or How I use Mathematica to Reassure Myself to Go Back to Sleep at Nights: https://www.atriumtech.com/pongskorn/birthdayparadox/birthdayparadox.htm https://www.atriumtech.com/pongskorn/birthdayparadox/birthda...
- macinjosh 3y agoWhy is HN removing numerals at the begin of a title? Second time I’ve seen it today.
- deleted 3y ago[deleted]
- teraflop 3y agoHN has worked that way for a long time. The submission guidelines say: > If the title contains a gratuitous number or number + adjective, we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to "Ys." Exception: when the number is meaningful, e.g. "The 5 Platonic Solids." ...but what they don't tell you is that at some point, somebody decided it would be a good idea for the site to automatically remove numbers from titles, with no regard for whether they're "meaningful".
- dredmorbius 3y agoIt's an anti-listicle thing, noted in guidelines and numerous moderator comments: E.g.: <https://news.ycombinator.com/item?id=26826830 https://news.ycombinator.com/item?id=26826830> <https://news.ycombinator.com/item?id=26826830 https://news.ycombinator.com/item?id=26826830> Search: <https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20magic%20numbers%20title&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...>
- tacocataco 3y agoI dont want to be a number. Why can't I be a word or some exclamation points?
- adolph 3y agoYou could do something similar to https://what3words.com/clip.apples.leap https://what3words.com/clip.apples.leap You can be whatever you want to even a bee.
- hedora 3y agoThe social security administration in the US solved this long ago. Reserve a few bits for “which source generated this integer”, then go sequential, or random w/o replacement for each shard. (Note that SSN’s were not meant to be unique when this scheme was invented. They were designed to be reused periodically. Name, DOB and SSN should be unique though.)
- bee_rider 3y agoIP addresses as well. The random case is nice because there’s no coordination needed, but “you get this block” is not so hard for coordination. Although I guess our ID issuing system will have to adjudicate issues of national sovereignty so maybe it is harder than it looks at first.
- stubish 3y agoGiven the requirement to detect collisions and a process to reissue an ID for various legal reasons, I don't see a problem. In fact, don't use too many digits. If you use too many digits, collisions will be so rare that when they occur nobody will know how to handle them. Better to have a process that gets exercised a few times a year at minimum.
- devcat 3y agothere are great solutions like https://en.m.wikipedia.org/wiki/Snowflake_ID https://en.m.wikipedia.org/wiki/Snowflake_ID if you give up randomness
- dredmorbius 3y agoAnother advantage to greatly-overeingineered random string lengths is that it makes brute-force namespace search technically infeasible. This is an increaasing problem with PSTN (public switched telephone networks) where there are simply too few digits in a phone number to prevent comprehensive dialing attacks. (Number reuse would be another principle problem.) Long ago I'd read a Douglas Hofstadter essay (probably from his Scientific American puzzles column and compiled in Metamagical Themas) where he'd commented on the apparent idiocy of having very long account numbers which were clearly far larger than the possible object space. That critique fails to consider any number of points, including the challenges of non-coordinated UUID assignments (as Lemire writes), the practice of coding other semantic information into parts of a larger string (e.g., branch or office identifiers, years, or other redundant information), systems which have grown out of mergers of multiple independent systems (where accounts from systems A and B might have collided, so A and B now require distinguishing, as well as C, D, E, ...), and as I've noted, the ease of searching the namespace for valid / assigned values.