6 ms·
User IDs probably shouldn't be passed around as ints (2018)
- rahimnathwani 4y agoThis seems to be the same point about type safety that's made in this recent front page post: https://news.ycombinator.com/item?id=33844117 https://news.ycombinator.com/item?id=33844117
- nivertech 4y ago1. use UUIDs for user IDs 2. use typed schemas for APIs, i.e. GraphQL with a custom scalar type for UserID. Other typed schemas for APIs: OpenAPI, AsyncAPI, protobufs/gRPC, etc. // add GraphQL parsing and serialization code scalar UserID 3. Make the implicit - explicit. Don't use naked primitive types in your code (similarly as you will not use naked literal constants, i.e. use PI instead of 3.1415...). Use a PL with the string static typing (preferably Algebraic Type System). Define type for UserID which cannot be mixed with the integers, i.e. type UserID = UserID int 4. in dynamic PLs you can use tagged tuples or tagged maps/structs, i.e. {:user_id, 1234} or {"_type": "UserID", "id": 1234} 5. validate all external inputs (even those comming from the DB or Message Broker): validateUserID: int -> UserID 6. The examples above assume integer representation of the user id, here's if we switch to UUIDv4: type UUIDv4 = UUIDv4 string type UserID = UserID UUIDv4 validateUserID: string -> UserID
- nivertech 4y agos/comming from/coming from/ s/string static typing/strict static typing/ as opposed to the weak static typing in languages like C/C++/etc.
- colonwqbang 4y agoThe type system of C is not as good as some later ones, but it's perfectly sufficient to prevent errors of the type shown in TFA. struct user_id { uint64_t val; }; void ban_user(struct user_id id);
- nivertech 4y agoyes, here's the correct code: #include <stdio.h> #include <stdint.h> typedef struct user_id { uint64_t val; } user_id_t; void ban_user(user_id_t id) { printf("user with id %lu was banned", id.val); } int main() { user_id_t someUser = {123}; ban_user(someUser); return 0; } output: > clang-7 -pthread -lm -o main main.c > ./main user with id 123 was banned And here's the code with the bug: #include <stdio.h> #include <stdint.h> typedef struct user_id { uint64_t val; } user_id_t; typedef struct group_id { uint64_t val; } group_id_t; void ban_user(user_id_t id) { printf("user with id %lu was banned", id.val); } int main() { user_id_t someUser = {123}; ban_user(someUser); group_id_t someGroup = {456}; ban_user(someGroup); return 0; } Compiler output: > clang-7 -pthread -lm -o main main.c main.c:22:12: error: passing 'group_id_t' (aka 'struct group_id') to parameter of incompatible type 'user_id_t' (aka 'struct user_id') ban_user(someGroup); ^~~~~~~~~ main.c:13:25: note: passing argument to parameter 'id' here void ban_user(user_id_t id) { ^ 1 error generated. exit status 1
- LorenPechtel 4y agoAnd if you use UUIDs for user IDs you're probably using them for other things. The same sort of mistake can happen. The only real defense here is language-level enforcement. Allow declaration of a subtype that is not assignment-compatible with the parent even though it's identical. I don't do any web-facing stuff but the only bits of code that know about things like IDs are the database stuff. All the logic works with classes that contain the ID and relevant data--you always pass the class, not the ID.
- 10000truths 4y agoIf you’re using randomly generated UUIDs, then collisions of any kind should be extremely unlikely, no matter how you’re using them. <0.2% chance in 2^60 random UUIDs.
- unscaled 4y agoUUID also suffers from a serious issue: there is no standard way to encode them as string and in binary format. In binary, UUID can be encoded as Big Endian or Little Endian or even Mixed Endian, and while the string encoding is usually lowercase with a certain hyphenization rules, I've seen variations of that. The (hexadecimal) text encoding is also quite inefficient compared to more modern standards like ULID or kSUID.
- masklinn 4y ago> In binary, UUID can be encoded as Big Endian or Little Endian or even Mixed Endian RFC 4122 section 4.1.2: > The fields are presented with the most significant one first. > while the string encoding is usually lowercase with a certain hyphenization rules, I've seen variations of that. RFC 4122 section 3: > The hexadecimal values "a" through "f" are output as lower case characters and are case insensitive on input. The hyphenation is normative, per the same section UUID = time-low "-" time-mid "-" time-high-and-version "-" clock-seq-and-reserved clock-seq-low "-" node
- thangngoc89 4y agoAre you familiar with Typescript? If it's possible, could you give me an example of 3 or 4 in Typescript?
- nivertech 4y agoI don't use TypeScript and I didn't found how to do proper ML-style wrapper types in TypeScript. But I guess one can use type aliases for that: type UserID = { userID: number; }; let validateUserID = function (x: number): UserID? { ... }; Note, that my original example was missing the fact that not every input value is a valid user ID, i.e. the function should return Maybe<UserID>, Option<UserID>, or Result<UserID,SomeErrorType>. In TypeScript one can use optional types xxx? instead.
- thangngoc89 4y agoI was deep into ReasonML/OCaml in the last coupe of years so yeah I missed option/result and the ability to define new type on top of the primitive ones. Your example: type UserID = { id: number; } Would result in more allocations. But I think it's the only way at the moment.
- culi 4y agoyour validation function would probably look more like ```ts const validateUserId = (x: number): x is UserId => {}; ``` for the `UserId` type declaration, I'm actually a little unsure of the best way to go about it. But I feel like using enums, typed string literals, or classes might be better approach. Would be interested if anyone has any thoughts on this
- nikeee 4y agoYou can sue Tagged/Branded types for that: https://medium.com/@KevinBGreene/surviving-the-typescript-ecosystem-branding-and-type-tagging-6cf6e516523d https://medium.com/@KevinBGreene/surviving-the-typescript-ec... It's a simple hack that takes a type and adds an additional field to it. You can then use a User-Defined Type-Guard to ensure a value is valid: https://levelup.gitconnected.com/user-defined-type-guards-in-typescript-fad639e4944f https://levelup.gitconnected.com/user-defined-type-guards-in...
- Hermitian909 4y agoMy one critique of this prescription is that UUIDs are not identifiable as user IDs. IMO you're better off with a format like "user{20 random characters}"
- camgunz 4y agoI'll get back on my "tests are better than types" hobby horse and say "or write a single test for this function". The problem here is that an engineer was able to push untested code to prod that irrevocably modifies the database. You need a test here because types won't tell you that a user was banned. And that test would also have caught this error.
- nivertech 4y agoIMO the developer need multiple safety nets from the wide net to the narrow one: - informal methods: - unstructured time to think about the problem (Hammock Driven Development) - write things down = writing is thinking - Rubber Duck Driven Development - design reviews - lightweight formal methods: - Decision Tables - FSMs - FP / Algebraic Type System with rich scalar datatypes - DbC - Design-by-Contract - PBT - Property-based Testing - TDD - example-based tests
- camgunz 4y agoNice, yeah totally. And I would add you shouldn't put all your risk eggs in one safety basket. If you only have types or tests not only are you not covering all your bases, you're asking way too much from a given technique.
- mardix 4y agoWhile UUID4 can be good for UserID. Sometimes, to avoid having a second index in the DB, ULID is another good option; As not only it supports some of the features of UUID4 of uniqueness, it is also sortable.
- fire 4y agoTIL about ULID[0]; It looks interesting, but the spec was last touched in 2019 and I haven't really heard of it before... is it actively used? Also curious because I don't actually know: If a format spec is GPL, does that encumber implementations of said spec? 0: https://github.com/ulid/spec https://github.com/ulid/spec
- unscaled 4y agoThe part that was last touched in 2019 is the spec + homepage. By this measure, UUID was last touched in 2005 (if we count RFC 4122). Most ULID implementations are still actively maintained and people are still using it in production. Most implementations also have liberal license like MIT so no, you should not worry about the GPL spec (unless you plan to distribute the spec with your app).
- fire 4y agoNice! I just hadn't seen it referenced anywhere before - the last big ID wave I can remember was the swath of content about moving to snowflake style IDs
- nivertech 4y agoULID leaks information about when the user was created (while serial IDs leaking the order in which users were created and their cardinality). I'm using ULIDs for the cases when the entity is publicly orderd by time: any timestamped event, e.g. a chat message, a log entry, or a sensor measurement/metric. But need to be careful that the timestamp resolution is detailed enough. Also, while ULID might be good for optimizing RDBMS indexes, it might create hotspots in NoSQL K/V stores (i.e. all entities will be created on the same node in the cluster).
- shagie 4y agoIf you're not going to do math on it, don't pass it around as a number.
- mjevans 4y agoOn one hand I agree. On the other; Legal Names change: for lots of reasons. It makes a LOT of sense to store a UID the way *NIX systems often do. An explicit indirect lookup table entry.
- shagie 4y agoThe question is "are you passing that ID around as an int? or as a byte array?" It may happen that the data type in the database is an int... because the database does math on it (or its source - seq.nextval type thing). However, when you get it, you don't need to have it be an int that you can do math with. Even if you keep the same underlying implementation of it, pass it around in a way that doesn't let you add two IDs together or mutate it. The easiest way to do that is to store it as a String. One of the advantages of taking it to a String in Java is that it becomes immutable and so can cleanly be used as a key to a Map and prevents the easy math operations on it without taking it from a String to a numeric type (and back). If you see someone doing `Integer.parseInt(someId)` it becomes clear that they're doing something wrong with that.
- mjevans 4y agoThe file format / interface API shouldn't make assumptions about the use case. A Display Name (what humans interface with) and the 'internal identity in the system' (be that a UUID, a bare number, an exact string, whatever) should BOTH be available. As an example, a backup / archive of a project is created. Years later it is restored, but now several employees have new names. Maybe some automation accounts got renamed too. Should the restoration fail when no user ID is present? Or should it instead restore the internal / native IDs? What if a user by the same name does exist, but they got re-added or the users migrated to a different authentication method and now have different internal / native IDs? So you store the lookup table. Maybe even use a third tiny integer column if there aren't that many users. Store the platform ID and also the name. Though also have an option to not store or flatten that information.
- _3u10 4y agoIn practice it works just fine. Lots of really big and small companies do it everyday and somehow manage not to collapse. In theory there are all sorts of negative consequences to using integers that you’ll run into once in a while and will require a google for a solution, however, everyday all of your queries and all of your inserts will be faster than using anything else. Also, it will be widely supported by any 3rd party code you use to build your project which is probably more important than anything else.
- bakugo 4y agoMore and more I'm noticing this narrative of "if I write awful code with obvious mistakes and somehow nobody else notices it in code review, it's never our fault, the language and its typing system or safety features should've stopped me" gaining popularity in the programming world and I really have to ask, what happened to programmers that actually knew what they were doing instead of expecting the computer to tell them what to do?
- yakshaving_jgt 4y agoThey never existed. It was all hubris. Mistakes inevitably happened and people lost money and/or died.
- kvdveer 4y agoSoftware systems have been able to grow mostly because developers have been able to delegate a lot of diligence to tools. Of course we could require a developer to "know what they are doing", many work environments do. However, you won't see many posts about that. First of all because it doesn't scale, and secondly because it doesn't make for interesting reading.
- bigDinosaur 4y agoBecause people are realising in general how good type systems can help at preventing such errors? The well was poisoned for a while with terrible type systems that provided either marginal safety or were just horrible to work with (e.g. complete lack of inference) but that is now seriously changing.
- preommr 4y agoWhat happened is we went from making pong to cyberpunk 2077.
- chrsig 4y agoEnough of the "hard problems" are solved where it's expected that developers use off-the-shelf type solutions. As a result, they're not getting hands on experience solving hard problems, or challenging each other to think more deeply about solutions.
- 4y ago
- janaagaard 4y agoI think is goes a long way to always use lang names for ID properties. So always call it userId, companyId etc. instead of simply id everywhere.
- makach 4y agoIs it just user-ids we should discuss or is it all kind of ids represented as an int? on one side you have the convenience and simplicity and on the other side you have a set of experience, best practice, security considerations to take into account. It can be overwhelming because the correct way seems obvious, listen to the experts and do as somebody think you should do - but it never is as easy as that. You need to balance your structure. Everything you do has a performance hit in some way or another and you need to consider the impact said practice will make in your environment.
- Semaphor 4y ago> Is it just user-ids we should discuss or is it all kind of ids represented as an int? Strongly typed ID’s are a thing advocated for in general [0], yes. And they aren’t even inconvenient in some languages. Modern C# for example makes it very easy to use them [1]. [0]: https://andrewlock.net/using-strongly-typed-entity-ids-to-avoid-primitive-obsession-part-1/ https://andrewlock.net/using-strongly-typed-entity-ids-to-av... [1]: https://github.com/andrewlock/StronglyTypedId https://github.com/andrewlock/StronglyTypedId
- alxmng 4y agoDisagree. UUIDs are a waste of space, and they aren’t sorted. Use 64-bit ints. Prefer not exposing them to users, but it’s not a problem for most software. If you ever get so big you need to shard, use Snowflake or a 64bit scheme that encodes the shard. Don’t overcomplicate things. You’re already using either SQLite or PostgreSQL and it already gives you auto-incrementing integer keys by default, without the need to encode/decode UUIDs in whatever software you’re writing to interface with it.
- onion2k 4y ago... and they aren’t sorted. I don't imagine there are many valid use cases where you want a list of users sorted by their database record ID though, and if you're suggesting an auto-incremented int then the creation timestamp will give you the same order anyway.
- tijsvd 4y agoTake incrementing int32. Extend to 64 bits. Multiply by large prime number (e.g. fnv32 prime). Mod 1 million. Add 1 million. End up with random looking, 7 digits, nicely sequenced, 32 bit integers. Write an exhaustive test to verify. When near 1 million users (yagni), reset sequence and do the same with 10 million (or one billion). Doesn't solve the upside of 128-bit random numbers (ala uuid): the ability to generate remotely and expect no collision.
- alxmng 4y agoThat’s a neat way of obscuring the id. I don’t see how remote generation is an upside. If you’re using UUID as a database key, you’re hitting the DB anyway, so it doesn’t save a trip to the DB. There are plenty of uses for UUID like peer-to-peer apps where that makes sense, but the article is talking about database keys.
- sonicgear1 4y agoUUIDv1 can be sorted by time of creation. That said UUIDs are pretty good, they are hard to guess and can be generated anywhere without a coordinator.
- 4y ago
- Mikhail_Edoshin 4y agoThe mistake has nothing to do with 'int' lacking something. 'int' is totally fine as an user ID or anything else. Everything in computer is essentially an 'int'. Every character in this text, for example, is an integer. The problem is that 'ban_account' changes data in a user record. Hence it should be a method of a 'User' class. And the 'Message' class should have a way to fetch an instance of 'User' for the sender. Here's the right way: ban_senders_of_messages(messages) { for (i = 0; i < messages.size; ++i) messages[i].sender.ban(); }
- actuallyalys 4y agoOne solution, available in most languages, is to not use manual indexing at all. Use a for-each loop, iterator or similar abstraction. Sometimes using an index or counter is unavoidable, of course, but hopefully seeing an indexed loop, especially combined with a comparatively risky operation like banning, would trigger more scrutiny during a code review, more testing, or both. Avoiding manual indexing doesn't prevent you from sticking other stray integers in place of user IDs (like specifying message IDs instead of the user ID that sent the message), but most of them are not biased toward the start like indexes are. Still, I think it's prudent to avoid these kinds of errors at all if you can. Perhaps a good reason to switch to UUIDs for all primary keys, even if the normal concerns about enumerability don't apply.
- dang 4y agoDiscussed at the time: User IDs probably shouldn't be passed around as ints - https://news.ycombinator.com/item?id=16946557 https://news.ycombinator.com/item?id=16946557 - April 2018 (84 comments)
- iancarroll 4y agoUnpredictable user IDs are also an important security feature. At every tech company, developers inevitably write an endpoint that accidentally operates on user data without authorization checks. It is much easier for an attacker to iterate over integers than to figure out a separate way to leak a list of user UUIDs.
- andyp-kw 4y agoA hacker knowing the user ID should not be a problem in itself if both authorization and authentication are used by the endpoint.
- de6u99er 4y agoI consider using (G)UIDs anyways best practice. They are super practical when merging databases.