6 ms·
A corollary or generalized interpretation of this approach (and someone please specify if there’s a formal term for this) is: “fail locally, and immediately.”
by ramraj07 4y ago
A corollary or generalized interpretation of this approach (and someone please specify if there’s a formal term for this) is: “fail locally, and immediately.”
What I mean is that once something unexpected happens your code should ideally fail in that step itself.
The simplest most common example I’ve seen with python programmers is when they pass around dicts as arguments in complex code bases. Methods expect various keys to be present, and often methods also have fail safe defaults if some keys are absent. The defaults are written for the specification, sure, but often they also tolerate unexpected exceptions that happened upstream.
Now when an unexpected exception happens, your program fails somewhere else and the stacktrace is useless. The only way to figure out what went wrong is to debug it line by line.
With python there’s still no elegant solution. I’m now trying to ensure all my methods are typed and use dataclasses and pydantic classes to type and group these parameters but there’s still opportunities for these “fail later” errors. Solutions and suggestions would be appreciated!
- bzxcvbn 4y ago>Solutions and suggestions would be appreciated! Use a language with strong typing?
- aunderscored 4y agoPython is strongly typed. You want statically typed. (Instead of duck typed / dynamically typed)
- jahewson 4y agoFWIW “strong” typing is a colloquial term and lacks any precise meaning.
- bzxcvbn 4y agoCan you guess what this code does? class foo: pass obj = foo() obj.bar = "I thought Python was strongly typed?" print(obj.bar) And even better: class foo: a = 42 obj = foo() print(obj.a) del foo.a print(obj.a) Whatever your opinion on what the imprecise sentence "strongly typed language" should mean, these are definitely not features of one.
- JoshCole 4y agoYes, I can guess what the code does. But can you guess what this code will do? 1 + "1" Contrast Python (a strongly typed language): >>> 1 + "1" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'int' and 'str' >>> [] + 1 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate list (not "int") to list With Javascript (a weakly typed language): 1 + "1" "11" [] + 1 "1"
- tsimionescu 4y agoJS's typing is weaker than Python's, but Python still has a relatively weak type system - particularly in old-style classes as GP was showing (class foo(object)-style classes fix those problems, or at least some of them) EDIT: I was wrong about new-style classes fixing this. Other dynamic languages are stronger than both - for example Common Lisp.
- filmor 4y agoThere are no old-style classes in Python 3.
- tsimionescu 4y agoOops, you're right, and the problem isn't fixed by new-style classes. I must have misremembered something.
- dehrmann 4y ago> 1 + "1" Did someone say PHP?
- withinboredom 4y agoPHP has a strict mode.
- 4y ago
- bogdanoff_2 4y agoWell, it's not very strongly typed if you use dicts with default values for everything.
- anyfoo 4y agoYeah, that one is indeed obvious. That's what everyone advocating for static typing (which you mean, somebody commented already) has been shouting all the time. The problems coming from dynamic typing are just unnecessary, and the only thing that made python palpable for me again was mypy, which introduces static typing into python. But it's not the default, so it still only really works for small things.
- teaearlgraycold 4y agoIMO mypy is severely lacking compared to what TypeScript added to JavaScript. I'd always pick TS over Python given the choice.
- ramraj07 4y agoWe use pytype as well but nothing works runtime right? I implemented a “check_type” decorator I use in many places but am starting to think I should decorate all the methods in my code with this decorator while building it or something.
- MonkeyMalarky 4y ago>Solutions and suggestions would be appreciated! Ban the usage of default values or default parameters anywhere outside of top-level / public facing functions. Plus assert everything all the time. I've gotten into arguments with other developers over it but I'll take the inconvenience in developing now over tearing hair out over bugs later, anytime.
- bimboler38 4y ago>Plus assert everything all the time. This is where static checking comes in. Static tests should fail if it's assumed (and not asserted) that a key exists.
- BurningFrog 4y agoIn general, I think using real classes with a single central definition instead of raw manually created dicts is the solution. A thorough test suite is also needed, of course.
- jeshin 4y agodicts are just a little too easy to use. You just smear it down, pass it around, and you're in business. If you really want to shoot yourself in the foot, also modify its structure here and there along the way, it's just so convenient. Who needs all that hassle of declaring a data class for each little thing? It took me a little too long to realize that a data class represents a contract about the structure of your data, meaning that no matter how many calls deep you are passing it around, you will always know its structure without having to trace it back to the origin, and that's a powerful thing.
- FridgeSeal 4y agoI worked on a team once where a couple of co-workers were doing this and more in what is possibly the worst Python codebase I’ve had the misfortune of seeing. Highlights included: * DIY “json” logging function that did some obscene string concat work every time it was called and abused global vars; it also didn’t output valid JSON. Suggestions to just use a normal logging library were aggressively disregarded. * dozens of functions, all of which indirectly mutated this extremely nested dictionary of data. They all had slightly different names, and none of them took this dictionary as a parameter, they just abused global vars. All of them would do these insane checks to ensure that the specific keys they were looking for existed * none of it was in git properly; the 2 data engineers writing it passed the code back and forth using a google drive. * instead of importing functions from the Python files they wrote, they’d invoke the functions by shelling out, calling Python <other file.py>, string interpolating the values and then waiting for completion by *waiting for a file of a specific name to be written into the file system. Oh yeah and when they decided they wanted parallelism, instead of doing the sane thing and using something like joblib or multiprocessing to make stuff easy, they’d just shell out and invoke more Python processes via xargs…
- packetlost 4y agoAs someone working with an extremely large Python codebase, early on we made the call to never allow dictionaries as arguments to functions (with exceptions for if the dictionary is truly arbitrary and only gets logged/persisted for human reading). We rely heavily on type annotations and dataclasses. Type system weaknesses aside, the system is rather maintainable despite its size, complexity, and domain.
- strogonoff 4y agoComing back from TypeScript to Python, I found that most recent (3.10+) typing annotation shorthands are pretty succinct, and running mypy at all times really helps cut down on runtime errors. My recipe: — annotate variables, attributes, arguments and return values; — run a good type linter (we use mypy) at all times; — never pass around generic dictionaries: use dataclasses[0], TypedDicts, etc. instead. That way you define a subclass inheriting from, say, TypedDict and declare that your function only takes that subclass. After that, you’ll get a loud error if you pass any dictionary that doesn’t match the spec (missing keys, wrong values, etc.)—ideally, right in your IDE. (To reiterate, this would be a pointless exercise if you don’t lint all the time; most IDEs support this.) [0] You can additionally use them with Pydantic, which can validate data at runtime at a cost of some performance overhead.