9 ms·
Don't Be Afraid of Types
- parpfish 2y agomy code became much easier to maintain once i stopped thinking of it as writing "algorithms" and "processes" and started thinking of it as a series of type conversions. structuring what lives where became easier, naming things became systematic and consistent, and writing unit tests became simple.
- taberiand 2y agoI agree; in my experience, pretty much everything is ETL. We take data from one thing, change it a bit, and put it somewhere else. Sometimes, as a treat, we take data from two things, put them together, and then put that somewhere else. Frontend, backend, databases, services, reports, whatever - ETL. In that context, the types and transformations between types are the most important feature. (Everything is actually Category Theory)
- necovek 2y agoExcept that "ETL" is a terrible name for it: one of those generic acronyms, that even when expanded to "Extract, Transform, Load", needs explanation for at least the "E" and "L" parts. To me, looking at it as "functional" approach instead (data in, operate over that data, data out) is cognitively simpler.
- namaria 2y agoI remember working on an ETL pipeline in Airflow years ago and thinking: "we're defining and programming an abstract computer". I guess with general computers everything we do is basically defining nested specific computers. That's, I think, the insight behind SmallTalk and the original concept of objects it used: the objects were supposed to represent computers and the message passing was an abstract network layer.
- whattheheckheck 2y agohttps://rachit.pl/post/you-have-built-a-compiler/ https://rachit.pl/post/you-have-built-a-compiler/
- tomtom1337 2y agoThis resonates strongly! In Python, I often now find myself declaring Pydantic data structures, and then adding classmethods and regular methods on them to facilitate converting between them. It makes for great APIs (dot-chaining from one type to another), well-defined types (parse, don’t validate) and keeps the code associated with the type.
- whilenot-dev 2y agoExactly! class Profile: ... class User: @classmethod def from_profile(cls, profile: Profile) -> 'User': ... def to_profile(self) -> Profile: ... ...are about all the methods I need in my data records. Three simple rules though: 1. Keep isomorphisms to one class only: Don't put two def to_${OTHER_MODEL_NAME} in each class, instead (like you said) create one static mapping (@classmethod) and one instance mapping 2. Add a mapping to the one class that feels more generalized out of the two: A more generalized data model will probably be used a lot more throughout the application 3. The creation of instances should be pure: If a mapping has side effects and needs to await something then it isn't just a mapping - first resolve all necessary dependencies, then do the mapping
- salmonellaeater 2y ago> I found that there’s a slight aversion to creating new types in the codebases I work in. I've encountered the same phenomenon, and I too cannot explain why it happens. Some of the highest-value types are the small special-purpose types like the article's "CreateSubscriptionRequest". They make it much easier to test and maintain these kinds of code paths, like API handlers and DAO/ORM methods. One of the things that Typescript makes easy is that you can declare a type just to describe some values you're working with, independent of where they come from. So there's no need to e.g. implement a new interface when passing in arguments; if the input conforms to the type, then it's accepted by the compiler. I suspect part of the reason for not wanting to introduce a new type in other languages like Java is the extra friction of having to wrap values in a new class that implements the interface. But even in Typescript codebases I see reluctance to declare new types. They're completely free from the caller's perspective, and they help tremendously with preventing bugs and making refactoring easier. Why are so many engineers afraid to use them? Instead the codebase is littered with functions that take six positional arguments of type string and number. It's a recipe for bugs.
- re-thc 2y ago> But even in Typescript codebases I see reluctance to declare new types. The current Typescript hype / trend is to infer types. Problem is at some point it slow things down to a crawl and it can get really confusing. Instead of having a type mismatch between type A and type B you get an error report that looks like a huge json chain.
- motorest 2y ago> I've encountered the same phenomenon, and I too cannot explain why it happens. I think that some languages lead developers to think of types as architecture components. The cognitive cost and actual development work required to add a type to a project is not the one-liner that we see in TypeScript. As soon as you create a new class, you have a new component that is untested and unproven to work, which then requires developers to add test coverages, which then requires them to add the necessary behavior, etc. Before you know it, even though you started out by creating a class, you end up with 3 or 4 new files in your project and a PR that spans a dozen source files. Alternatively, you could instead pass an existing type, or even a primitive type? > But even in Typescript codebases I see reluctance to declare new types. Of course. Adding types is not free of cost. You're adding cognitive load to be able to understand what that symbol means and how it can and should be used, not to mention support infrastructure like all the type guards you need to have in place to nudge the compiler to help you write things the right way. Think about it for a second: one of the main uses of types is to prevent developers from misusing specific objects if they don't meet specific requirements. Once you define a type, you need to support the happy flows and also the other flows as well. The bulk of the complexity often lies in the non-happy flows.
- beders 2y agoThe issue is names. Every darn little thing in Java needs a name. If there is no good name, that's a hint that maybe you don't need a new type. Obligatory Clojure example: (defn full-name [{:keys [first-name last-name]}] (str first-name " " last-name)) This defines a function named `full-name`. The stuff between [] is the argument list. There's a single argument. The argument has no name. Instead it is using destructuring to access keys `:first-name` and `:last-name` of a map passed in (so the type of the unnamed argument is just Map) This function works for anything that has a key `:first-name` and `:last-name`. There's no need to declare a type ObjectWithFirstNameAndLastName. It would be quite silly.
- continuational 2y agoDid you forget to include `middle-name`? There's no way to tell.
- Spivak 2y agoIn Python you're describing a Protocol. It's actually super reasonable to have a ObjectWithFirstNameAndLastName noun like this. You don't ever need to construct one but you can use it in the type slot and objects you pass in will be checked to conform. You see all kinds of weird specific types floating around the standard lib like this for type hinting. Duck typing is great, what's even better is documenting when when they need to quack or waddle.
- wesselbindt 2y agoI think protocols have two major drawbacks regarding readability and safety. When I have a protocol, I cannot easily find its concrete implementations, so it becomes harder to see what the code is actually doing. As for safety, protocols have no way of distinguishing between class Command: def execute(self) -> None: # some implementation and class Prisoner: def execute(self) -> None: # some other implementation The implementor of the Prisoner class might not want the Prisoner class to be able to be slotted in where the Command class can be slotted in. Your type checker will be of no help here. If you use abstract base classes, your type checker can prevent such mistakes. So when it comes to your own code, the drawbacks of the structural Protocols in comparison the nominal ABCs are pretty big. The pros seem non-existent. The pro, I guess, is that you don't have to type the handful of characters "(Baseclass)" with every concrete implementation. But they do have one major advantage: if you have third party code that you have no control over, and you have some part of this codebase that you want to replace with your own, and there's no convenient way to do something like the adapter pattern, because it's somehow a bit deeply nested then a Protocol is a great solution.
- paulddraper 2y agoRelatedly, don’t be afraid of (database) tables. It’s okay, you really can have hundreds of tables, your DBMS can handle it. Obviously don’t create them for their own sake, but there’s no reason to force reuse or generic design for different things.
- necovek 2y agoI think with databases, it's natural to go for normalized forms for the benefits you get with a relational database. Unless you don't have anyone with any semblance of DB design, you usually have more friction when you want to de-normalize the DB instead. At least, that's been my experience, but I did have the luck of mostly working with experts in DB design.
- pcwelder 2y agoEncapsulation isn't the real reason why types help. Pass 5 variables individually, the only difference will be in verbosity. The real benefit is type narrowing. (Or declaring the space of all possible combinations of values.) Instead of id: null | string name: null | string ... You'd have user: null | User And User: { id: string, name: string, ..} declaring that all are not null together. Pushing validation to static checker instead of runtime.
- afc 2y agoAgree. I try hard to push the validation to static checks, but it can't always be done. For example: - a "probability" represented as a double must always be between 0 and 1 - A vector of values given to some group of functions must always be sorted (by some given order) But even in those cases, using types allows me to ensure that this validation always happens (when the type instance is created). It also let's me avoid having to explicitly validate this inefficiently (e.g. redundantly in preconditions in functions that receive these values). This doesn't change the fact that static validation is a better approach, but compliments it. I wrote a bit about it here: https://github.com/alefore/weblog/blob/master/edge/correctness.md#ghost-types-validators https://github.com/alefore/weblog/blob/master/edge/correctne...
- default-kramer 2y ago> Pass 5 variables individually, the only difference will be in verbosity. Which can be an extremely significant difference. If you're already using a single object and add a 6th field, you only need to update the places where the object is constructed and where the new field is consumed. If you're using individual variables, you also need to update all the code that those 5, now 6, variables simply pass through.
- noduerme 2y agoHuh. I don't see any conflict between loving OO-programming and also loving types. Isn't 9/10ths of OO just about consolidating your business logic into interfaces and types that you'd ideally want to re-use? I feel like if it's not, then people are using OO the wrong way.
- 9rx 2y agoYou can love conflicting things, but the basis of OO programming, that which makes it object-oriented and not just object-based, is message passing, which sees messages flow between objects for inspection. That is inherently runtime behaviour which is at odds with static type enforcement.
- pjmlp 2y agoAs shown by The Art of Metaobject Protocol that is the genesis of CLOS, not really.
- 9rx 2y agoThe Art of the Metaobject Protocol does not seem to go into any detail about senders sending blobs of data to receivers. What do you think it shows, exactly? That you don't need to send blobs of data? That should be obvious by the fact that you can count the number of languages that support that concept on one hand, but does not remove that message passing when employed is necessarily dependent on runtime.
- pjmlp 2y agoIt shows that message passing and types get along just fine.
- 9rx 2y agoIt is not like they are sworn enemies, but as messages can originate from outside the program, they cannot be typed statically. It is impossible to know what types will appear before runtime. For messages that originate from within the same codebase, you can, perhaps, start to layer static typing on top, but it can only be a partial solution as it remains that messages can come from other sources.
- salgernon 2y agoI was refactoring a 700 line recursive C function (!) - one of those with all variables declared at the outer scope while the function itself was mainly a one pass switch with goto’s for error handling. I created c++ classes for each case, hoisted them out and coalesced types that were otherwise identical. The new version was way smaller and and (imho) far more readable and maintainable. At some point I needed to change the types to capture a byte range from a buffer rather than just referring to the base+offset and length, and it was trivial to make that change and have it “just work”. These were no vtable classes with inline methods, within a single compilation unit - so they just poof go away in a stripped binary. ‘Tis better to create a class, than never to have class at all. Or curse the darkness.
- jongjong 2y agoI'm not a huge fan of types because they don't model the real world accurately. In the real world, most concepts which we describe with human language have many variations with optional properties and it's a pain and a waste of time to try to come up with labels to categorize each one... It induces people to believe that two similar concepts are distinct, when in fact, they are extremely similar and should share the same name. I hate codebases which have many different types for each concept... Just because they have a few properties that are different. It overcomplicates things and creates logical boundaries in the system before they are needed or well defined. It forces people to categorize concepts before they understand the business domain. People will argue that you can define types with optional properties, that's a fair point but you lose some type safety if you do that... If you can do away with some type safety (with regard to some properties), surely you can do away with it completely?
- myvoiceismypass 2y agoIf you flipped things a bit and think of types as more or less "allowed shapes of the things happening in my program", it might seem closer to the real-world comparison that you are thinking of.
- Kinrany 2y ago"Modeling the real world" only makes sense when you're building a simulator of some kind. Another similar approach is building a "digital twin" of something that is happening in the real world. But very often the software is the only implementation, there's nothing to simulate.
- necovek 2y agoPlease be afraid of types. On top of new cognitive load, introduction of redirection, and anti-patterns like pretend-simplification where you shuffle a bunch of unrelated parameters into a struct to make a function receive only a single argument (which quickly evolves into functions receiving parts of that struct they don't need, making use and testing much harder — I am surprised an article is really recommending this as a strategy), your types frequently get exposed to external users, and you need to start thinking about both backwards- and forwards-compatibility. Benefits of types should be carefully balanced against the cost of introducing them, and while calling that "fear" might not be appropriate, it should be a conscious cost-benefit analysis.
- NoahKAndrews 2y agoYou need to think about backwards and forwards compatibility either way, types help make that easier.
- necovek 2y agoNot when they are misused, which is quite common. The example of the article is a great one: replace all your arguments to a function with a single argument of a new "type". Soon, this new type gets used in another function, and it "just" needs another field added. Suddenly, you inadvertently changed the API for the previous function. So yes, you need to worry about backwards and forwards compat either way, but misusing types will make it harder and more error prone. Instead, if you carefully used types only when they really represent a new logical thing, you wouldn't replace unrelated set of arguments with a single argument of a new combo type, and code would be cleaner and saner.
- Hojojo 2y agoConversely, do you also consider the downsides and consequences of not having types? If you don't, then you're blind to whole categories of issues. As a good engineer, you need to be able to reason both ways to come to a reasonable solution, not just one that conforms to your particular ideology.
- 2y ago
- galaxyLogic 2y agoTypes are the dual of Functions. You could create very many functions which differ just a bit, or you could create functions which take arguments/parameters and do with a smaller set of functions. So how about using generic (=parameterized) types? Isn't that the answer?
- karparov 2y ago> That’s what the type system is for: a means of grouping similar bits of information into an easy-to-use whole. While types can be used for that, they are a much broader concept. I would say the general purpose of types is to tell apples from oranges.
- lurking_swe 2y agonot to mention having some actual confidence when making changes to a project! especially one you didn’t author.
- DaiPlusPlus 2y ago...even in a strictly-statically-typed language with a perfectly expressive type-system, it would be unwise to rely _only_ on a "Build succeeded!" message for having confidence in any changes to the system: there's no substitute for well-trodden unit and integration tests for any codebase of nontrivial importance or complexity.
- lurking_swe 2y agoit does eliminate an entire class of issues thougj. The typical “property not found” type of issues that are common in vanilla javascript, python, etc. Having types helps the IDE help YOU! That’s my favorite part about types and a strong IDE like Webstorm or IntelliJ. I agree it’s not a substitute for proper testing though.
- ninetyninenine 2y agoIn Java you can’t just create a type. You are creating a mini app. Classes in Java hold logic and internal mutating state. I don’t want to create this just because I need a type. Really you want to create a struct or an interface as a type.
- morcus 2y agoLuckily Java now has "Record" to partially handle the pain.
- jonasenordin 2y agoJava has interface types. https://en.wikipedia.org/wiki/Interface_(Java) https://en.wikipedia.org/wiki/Interface_(Java)
- ninetyninenine 2y agoI know but the article talks about classes and almost equate the concept of a class with a type in the same way java does.
- enriquto 2y agoAs somebody who is afraid of types (and also, who hates types, because we all hate what we fear), may my point of view serve as balance: you don't need a type system if everything is of the same type. Programming in a type-less style is an exhilarating and liberating experience: assembler : everything is a word C : everything is an array of bytes fortran/APL/matlab/octave : everything is a multi-dimensional array of floats lua : everything is a table tcl : everything is a string unix : everything is a file In some of these languages there are other types, OK, but it helps to treat these objects as awkward deviations from the appropriate thing, and to feel a bit guilty when you use them (e.g., strings in fortran).
- MrMcCall 2y agoBut once you get down to the unit data values inside any of those aggregates, you're still dealing with either characters, ints, floats, strings, arrays, and they each have their own individual access patterns and, more importantly, modification functions. You can't add a number to a string, only to another number. If you are dealing with a float, you better be careful how you check it for equality. If it's pure binary, what kind of byte is it? Ascii, unicode code point, unsigned byte, signed multi-byte int, ... whatever. There's no escaping the details, friend. And your saying "everything is a word" for assembler is just plain wrong.
- enriquto 2y ago> And your saying "everything is a word" for assembler is just plain wrong. I also love x87 registers, but they are becoming rarer these days.
- MrMcCall 2y agoYour profile says "just another C hacker". I learned C from reading K&R in the late 80s. Every C compiler I've worked with could output the code as assembler, so C is really a thin layer of abstraction that wraps assembler. Having programmed in pure assembler before, I understand the benefits of C's abstractions, which began with its minimal, but helpful, type system. Should I not be taking you seriously? We are not just talking with each other but sharing our expertise with those who may be reading. Sometimes I forget that other people can just be unpleasant on purpose. I find no other explanation for your response.
- DeathArrow 2y agoPeople might be afraid of types because in OOP land there's the idea that types aren't mere containers for data. You have to use encapsulation, inheritance and polymorphism. Fields and properties shouldn't have public setters. You assign a value only through a method, otherwise you make the gods angry. You have gazillions of constructors, static, public, protected, private and internal. And you have gazillions of methods, static, public, private, protected and internal. You inherit at least an abstract class and an interface. You have at least some other types composed in your type. Unless you do all this, some people might not consider them proper types. I had my engineering manager warn me that data classes are "anemic models". Yes, but why? "We should have logic and methods to set fields in classes". "Yes, but why?" "It's OOP and we do DDD and use encapsulation." "Yes, but why? Imagine we have immutable records that hold just data and static classes as function containers, and those functions just act on the records, return some new ones and change no state. This way we can reason with ease about our goddam software, especially if we don't encapsulate state and mutate it all over the place, Uncle Bob, be damned." He shook his head in horror. He probably thinks I am a kind of heretic.
- bluGill 2y agoThat is what happens when all too commonly people treat OOP as their religion. Prefer encapsulation to inheritance is good advice. Interfaces are very useful if you have more than one implementation (mocks/fakes do not count), but otherwise pointless. Having private data is always a good idea - no matter what your code style is you don't want to have to look over all 10+million lines of code anytime you need to change data. (there is no clear answer to how the size of a circle should be stores - radius, diameter, circumference, area - given one you can easially calculate the others, but there are implications to whatever choice you make and sometimes you will discover late that you made the wrong choice). > we [snip] use encapsulation." "Yes, but why?" Simple: because when your program gets large you cannot know everything needed to modify the code. You need places where you can say "something really complex happens here and I must trust it is right" (not to be confused with it is right, only that right or wrong is irrelevant) OOP is a great answer to the problems of large complex programs. It is not the only answer. It sometimes is the best and sometimes not. Applying it like a religion is wrong.
- teeray 2y agoMy favorite is when I come across a set of functions in Go with the same prefix that all take some struct treated as a bag of data. “validateUserRequest”, “processUserRequest”, “initializeUserRequest”, or something like that. You have written a type already, but made it annoying. Write methods for “Validate” and “Process”, then have a factory function “NewUserRequest” (consider whether you actually need it first though).
- jt2190 2y agoIf you ignore the weird rant about OOP (that references an article from 2006… haven’t we all moved on in the last twenty years?), the author’s main thesis lacks context: > But take it from someone that’s had do deal with codes passing through and returning several values of strings, ints, and bools through a series of function calls: a single struct value is much easier to work with. This presupposes that the code should be very strict with the data, which may or may not be desirable. For example, in many CRUD apps the client and the database enforce constraints, while the middle tier just needs to martial data between the two, and it’s questionable whether the middle tier should do type enforcement. As always: Challenge your assumptions, try to understand how a “bad” practice might actually be the right approach in certain contexts, etc, etc.
- darioush 2y agoKind of disagree with this article, when you add a "noun" (aka type), you're often introducing a new abstraction. Abstractions have a maintenance cost associated with it ie, another developer or possibly yourself must be able to recreate the "algebra" associated with that type (your thought process) at the time of making modifications. This creates some problems: 1. Since there's no requirement to create a cohesive algebra (API), there was probably never a cohesive abstraction to begin with. 2. Requirements may have changed since the inception of the abstraction, further breaking its cohesion. 3. Since we largely practice "PR (aka change) driven development", after a few substantial repetitions of step 2, now the abstraction has morphed into something that's actually very tied into the callsites (verbs), and is essentially now tech debt (more like a bespoke rube goldberg machine than a well-designed re-usable software component). You can introduce types if you follow the open/closed principle which means you don't change abstractions after their creation (instead create new ones and then delete old ones when they have no callsites).
- default-kramer 2y ago> Kind of disagree with this article, when you add a "noun" (aka type), you're often introducing a new abstraction. The article is talking about simple "bundle of values" types; the example is the CreateSubscriptionRequest. This is not an abstraction. It is simply a declaration of all the fields that must be provided if you want to create a subscription. And it is usually superior to passing those N fields around individually.
- darioush 2y agoyeah this is correct, somehow I jumped to a different conclusion :)
- yodsanklai 2y agoNot creating abstractions has a cost too. Let say you have a string which is a user_id, just define a user_id type. That'll document the code, helps avoid mistake thanks to type checking, reduce cognitive load, ease refactoring and so on. And if you need to change abstraction at some point, then refactor.
- boris_m 2y agoLanguages like Java are awful in that respect, as they make it super hard to declare new types. People expect for a type to contain some logic, but it doesn't have to. e.g. a configuration is a type that contains other types that contains yet other types. But I have never seen it done like that in languages like Java.
- joaonmatos 2y agoI think we're moving past this fear, thankfully. When you look at newer Java features (records, destructuring, value types) you see that even in the most stereotypical OOP language we are developing a language to describe both behavior (what we always had, with SOLID and domain classes and so on), but also a lightweight language to operate on pure data.
- deleted 2y ago[deleted]
- legulere 2y agoAlso known as primitive obsession