12 ms·

Postmodern Error Handling in Python 3.6

• nicolaslem 10y ago
  Type annotations are also helpful without mypy. PyCharm uses them to provide hints and warnings during development. At first I was reluctant of the syntax, after a while I got used to it. Function definitions look a lot like Rust ones and do not require the endless docstrings just to document the type of the expected parameters anymore.
  • insertnickname 10y ago
    My question is this: What happened to all the duck-typing evangelism? "If it walks like a duck and quacks like a duck, it is a duck", "static typing only catches trivial errors", and so on.
    • nicolaslem 10y ago
      Not a definitive answer to the problem but mypy supports a limited sets of common "interfaces", for instance dict-like or list-like. http://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html#standard-duck-types http://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html#st...
      • raverbashing 10y ago
        Ah, that makes things better
      • throwaway_374 10y ago
        That's right, mirroring the built in (3.x) abc collections heirarchy. Which then allows you to do interesting things like delegation based polymorphism by registering with the appropriate metaclass so - just to come back to your parent's point - interrogating the type reveals it to be list-like or dict-like.
    • jsmeaton 10y ago
      Being able to lock down known interfaces can be beneficial, while leaving the rest open. Optional typing also lets you ignore the type hints when you really know what you want to do. > "static typing only catches trivial errors" To be honest I've not seen that sentiment very often, and it's definitely not something I agree with.
    • mattdeboard 10y ago
      Everything is cyclical. Pain is forgotten, same mistakes are made, pain is rediscovered, relief is reinvented. I had a high school teacher who said something like, "I haven't bought new ties for 25 years. Why? Because things always go in and out of fashion. A particular tie might be out of fashion for ten years but it comes back eventually."
    • evgen 10y ago
      I use typing at interface boundaries within code and sometimes between functions. Within a function I use duck typing. As others have mentioned, I use this more as an editor hint to warn me of questionable behavior and almost never run mypy. Yes, it only catches trivial errors, but a trivial error highlighted in the editor is easy to see/fix. The extra typing in the function sig is saved in not having to enter the same info in the docstring, so it feels like a wash to me.
    • deleted 10y ago
      [deleted]
    • kwhitefoot 10y ago
      > "static typing only catches trivial errors", That's partly because we tend to use the same types for everything. For instance in a spreadsheet you can subtract a column number from a row number and get no complaints even though the difference between the two quite likely makes no sense. If you have a separate type for row numbers and column numbers such mistakes can be caught.
      • throwawayish 10y ago
        In your example you'd just have two different types with no subtraction operation defined between them. That has nothing to do with type hints. What you probably meant: This is where being strongly typed helps. Python is and always has been strongly typed.
    • lmm 10y ago
      Back when static typing meant Java or worse, it was reasonable to think horribly verbose and clunky programming was inherent to static typing. But now we know better.
    • pmontra 10y ago
      It's still here. Types don't matter in many scenarios. Example: get a parameter from a POST, use it as a key to query the db, store it into the db. Is it a string, is it an int? It doesn't really matter to the application because the db driver can handle that. Don't slow me down by making me look at the docs of what I really receive from the web server to declare a type. Multiply by some hundred or thousand times. Strongly typed languages only raise costs for my customers. Given that the budget is constant they get more features with dynamically typed languages. I'm not happy seeing language designers introducing strong or optional typing in languages that didn't have it, but I understand that their goals are different from the goals of the developers that use their languages. I'd say leave the core of Python (and Ruby [1]) alone: if we picked those dynamically typed languages in the last 25 years (they're that old) it means we like them so. We can use plenty of other languages if we really need types. [1] https://blog.heroku.com/ruby-3-by-3 https://blog.heroku.com/ruby-3-by-3 "Matz: ... the third major goal of the Ruby 3 is adding some kind of static typing whille keeping the duck typing, so some kind of structure for soft-typing or something like that. The main goal of the type system is to detect errors early. So adding this kind of static type check or type interfaces does not affect runtime."
      • adwn 10y ago
        > Example: get a parameter from a POST, use it as a key to query the db, store it into the db. So you get a string, and you want to pass it unmodified to something that expects a string? The static typing answer is simple: Your variable is a string! How does that slow you down, or cost your customers a single cent?
        • pmontra 10y ago
          I really don't know if it's a string. AFAIK the framework could give my ints when it sees a number. I should check, but what for? I'll do the next time I need to do some math on those values and then forget again. They're not the things developers should waste time with.
          • alexvoda 10y ago
    • lngnmn 10y ago
      The both statements you have quoted are still valid. Moreover, if you would think a bit you would realize that the world around you is duck-typed. At least at the level of proteins. One more thing. Every ontological attempt to use naive categories would fail and end up in duck-typing-like approach, because it reflects how the mother nature is.
    • OJFord 10y ago
      There exist libraries that provide 'duck types' and other goodness for type hinting. typecheck [0] is one (which can also run-time enforce the check) used like: import typecheck as tc def logathing( dest: tc.hasattrs('write'), thing: tc.any(int, float, str), prefix: tc.optional(str)=None ): dest.write(prefix + str(thing) if prefix else str(thing)) [0] - https://github.com/prechelt/typecheck-decorator https://github.com/prechelt/typecheck-decorator
• raverbashing 10y ago
  "Mypy allows you to add type annotations and enforce them prior to running your program" Yes, and if I wanted type annotations to stop my program from working I wouldn't be using Python Enforcing types is exactly what Python IS NOT about. Because of Duck Typing and everything else So this is not "postmodern error handling" this is "let's code Java in something else and pat ourselves in the back" Do you want to check errors in "compile time"? Use Pylint. It does the right thing
  • rspeer 10y ago
    Duck typing is not the solution to every problem. Pylint has nothing to do with catching type errors. It catches problems with code style. I don't find the code in this blog post convincing, but there are reasons to write type-safe code.
    • raverbashing 10y ago
      > Pylint has nothing to do with catching type errors. No, it does catch type errors, for example accessing methods that are not present in the object > but there are reasons to write type-safe code. Yes, and there are lots of languages that allow you that
  • uweschmitt 10y ago
    I'm not a big fan of type annotations because the code easily turns unreadable which is not what I expect from Python code. But for larger projects type checking is valuable. Types introduce additional contracts between components to manage and control the overall complexity of the system. For small scripts and self contained applications I regard type checking as a burden. For larger projects not. Pylint is a valuable tool to detect typical errors but does not check types. Regard a function which returns the sum of two given arguments. It will work with both arguments being integers and for both being strings. PyLint will not complain here, but if your code uses this result you might trigger errors much later during progam execution which are hard to detect or understand without type checking.
    • 10y ago
• rspeer 10y ago
  There are cases where you could justify writing "except Exception as e" and having it return a value, like if you're the author of Flask or Raven or something that includes handling arbitrary errors in other people's code in its job description. But within your own code, you would not want to represent "an unforeseen error occurred" with a value, no matter how much you like enums. If you were parsing JSON and you got an unexpected error that isn't about parsing JSON, logging the error and continuing is not the right option. There is probably nothing reasonable your program can do. Raise the error so your broken code stops running.
  • nicolaslem 10y ago
    I often find myself using "except Exception", while I don't like it, I cannot find another way. For instance when making an HTTP request. The requests library can throw thousands of different errors (SSLError, BrokenPipe, socket errors, errors from urllib, errors from requests itself...). As far as I know, it is the only choice if you want to know if a request succeeded or not so you can deal with it without your view returning a 500. I would really appreciate if someone has another solution to this.
    • vitorbaptistaa 10y ago
      All of requests' exceptions inherit from requests.exceptions.RequestException [0], so you could catch it. [0]: http://docs.python-requests.org/en/master/_modules/requests/exceptions/ http://docs.python-requests.org/en/master/_modules/requests/...
    • nitely 10y ago
      Kennethreitz requests lib? Just use requests.exceptions.RequestException, that should handle every error, if not then report it. For python urllib IOError and OSError should suffice. At least in py3.
      • multipass 10y ago
        and in Python >=3.3, IOError is an alias for OSError.
• St-Clock 10y ago
  For our small team, we found that type annotations are useful as a substitute of documentation and it cut our docstrings at least by half. For most small and well-named API functions, we don't need docstrings anymore. We're big fans of type aliases also. For example, API functions do not return an int, they return a PK. They do not return a str (or Text), they return a URL. We're also heavy users of typing.Optional to mark parameters or return values that can accept and return None. We tried mypy but it failed miserably on our large codebase and frankly, with our large test suite, we never had a typing error found in production so I think it's not a good investment of our time. I'm eager to test type annotations with our next intern to see if it speed up the ramp up process.
  • lsh 10y ago
    Python type hinting and aliases in Python 3: https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html I've just finished porting a small application to Python 3 just because 2020 and deprecation is in sight already - there are so many things that have passed me by in the Py3 world.
    • kzrdude 10y ago
      What I missed first was this deal with provisional API. Nice that it's widely tested, but it should be clear that typing is provisional: Your program is not future compatible if you use it. Asyncio was provisional until Python 3.6, so it is effectively a Py 3.6 feature that has been backported to 3.5 in time and space.
  • Xophmeister 10y ago
    > we never had a typing error in production Python (currently) doesn't actually check actual types are per their annotations. Do you mean that you rely on, say, your IDE to do static analysis to ensure everything's lined up, or that just by virtue of having annotations, your team can reason about things more easily? That's one thing I don't really like about type annotations in Py3.5+, unless you run it through a static analyser or have an IDE that understands them, they're effectively just noise. As such, I have mypy running through Syntastic, in Vim, and while not perfect it's a vast improvement over non-annotated Python.
• dbrgn 10y ago
  If you just need a Rust-like Result type for Python: https://github.com/dbrgn/result https://github.com/dbrgn/result
• chimeracoder 10y ago
  > I’ll be the first person to admit I have no idea what postmodernism actually means Well, if we want to take the title literally: Error handling is a fundamentally Modernist idea - shoring up errors and "correcting" or at least containing them. Postmodern error handling would be anything that reasonably counts as a reaction against Modernist error handling. The most basic would be not handling errors at all, and instead embracing the chaos of an error-prone system, and designing a system in which errors propagate but converge to a desirable value.
  • adamcharnock 10y ago
    To add to this, AFAIK postmodernism is actually a critique of modernism rather than a being a substitute for it. This critique may/will eventually produce a concept which can replace modernism. I found that enlightening, but it is probably also splitting hairs with regards to the blog post.
    • bandrami 10y ago
      It's also important to remember that "postmodernism" is at this point about 130 years old.
      • chimeracoder 10y ago
        > It's also important to remember that "postmodernism" is at this point about 130 years old. You're thinking of Modernism. Postmodernism is more like 60 years old, depending on how you draw the line, and it's still an ongoing and developing set of philosophies.
        • bandrami 10y ago
          The term was used in the 1880s in painting circles; it probably varies by field
          • coldtea 10y ago
• jstimpfle 10y ago
  It's interesting to hear that Guido is now excited by types. A few years ago his opinion was still that types can't catch most problems. I think I will listen to that interview. Personally I think types are over-hyped for most (profane) code. The problem with complicated and/or specific types (which promise to catch more errors) is, in one word, coupling. They create serious dependencies across the whole project or even across project boundaries. The other problem is: one functions-guy's URL is the next function-girl's string. And all this up- and downcasting usually creates considerable noise. Often overzealous typing is painting oneself in a corner, creating more pain then relief. It's like creating these totally arbitrary object-oriented inheritance-based taxonomies which might make some sense in one place of the code but totally break down in the next place. But speaking about primitive types like int and str, or maybe list of int - most function arguments can accept only one of those for the code to make sense. So strategically placed type annotations can help reduce some boilerplate and put some useful barriers for error search there. While the simple-types cases are also the easy ones - the first thing you'll notice while testing is usually that you put a str where an int was expected.
  • deleted 10y ago
    [deleted]
  • adwn 10y ago
    > The problem with complicated and/or specific types (which promise to catch more errors) is, in one word, coupling. They create serious dependencies across the whole project or even across project boundaries. With dynamic typing, you'll still have this coupling, but now it's unchecked and (in practice) undocumented.
    • jstimpfle 10y ago
      Most parts of the code don't actually care about all aspects of the type (or the value), but only that it's an object that they can hand to the next function. Or they only care about a subset of the value's properties (like it being a string, not necessarily a URL). For example, in Haskell there is the filter function: filter :: (a -> Bool) -> [a] -> [a] filter _ [] = [] filter pred (x:xs) | pred x = x : filter pred xs | otherwise = filter pred xs It doesn't actually care what (the type of) a is. This is one of the showcase examples of the power of Haskell's typesystem. (And admittedly it works nicely in this case). But you can have that easily with no typing at all. def filter(pred, xs): return [x for x in xs if pred(x)] So, dependencies avoided and it's not a huge source of bugs. While in the typed case, a system is needed to prove that the properties that get input come out again (or possibly a function of these properties). That quickly gets quite involved for more advanced cases. Just look what the average Haskell code looks like, how many language extensions are typically needed, how crippled the code typically is (libraries dictating exactly what properties the client must be able to statically precompute). Most of the Haskell community seems to agree that the end-goal would be a practical system for dependent types (the unification of types and values, i.e. no types at all ;->). For example, there are some libraries trying to capture the construction and execution of valid SQL queries in the type system. To understand these libraries without doubt a genius-level IQ is required. But the client code is still basically an unreadable mess. And the libraries are not reusable for cases where the database schema isn't known at compile time. The static type system actually prevents code reuse -- it makes the code really inflexible.
• gcb0 10y ago
  kinda of a bad pattern. lots of abuse of error handling apparently just to have a fetch and parser in the same method? why not have one promise of a fetch and then a promise of a successful parse?
• miloshadzic 10y ago
  Another reading of this paragraph is that it's essentially douchebaggery.
  • minikites 10y ago
    I'll never understand the techie disdain for the humanities and liberal arts.
    • plafl 10y ago
      Most tech people I know have an above average knowledge and respect for the humanities. Maybe you cannot excel in both but you can enjoy both. There is always a subset of the population, intersecting also technical people, that increase their self perceived value by decreasing the value of the others.
    • true_religion 10y ago
      Isn't computer science a liberal arts? At least it is a feature of lib art schools on the east coast, not necessarily always an engineering degree.
      • minikites 10y ago
        It depends on your definition of "liberal arts". The classical liberal arts (i.e. arts of a liberated person, as opposed to the servile arts) are music, arithmetic, geometry, astronomy, grammar, logic, and rhetoric. So if you boil down computer science to its mathematical principles, you could make a decent case for it. These days, the liberal arts generally relate to the humanities: language (e.g. English degree in the USA), fine arts, history, anthropology, sociology, philosophy, etc. It's the study of people and groups of people, instead of the physical world and the laws that operate on the physical world independently of the people that occupy it.
    • xkxx 10y ago
      Maybe it's partly because of anti-intellectualism, the idea that being very smart is bad. I had never heard about the idea, until recently. It was some article about American anti-intellectualism and why it is so popular. Reading that article was a very weird experience for me. I believe this concept doesn't exist in the culture of the country where I am from.
• adamcharnock 10y ago
  Am I correct in thinking that duck tying and type hinting are not mutually exclusive? Type hinting is not enforced at runtime, so we are still free to pass around whatever we like as long as it behaves as expected.
  • shawabawa3 10y ago
    Correct, although it kind of defeats the point of typing if you ignore type errors. There is however an Any type which you can use for things you don't need enforced
• demetri 10y ago
  I am mostly confused by this. The author started out with a friend's question about an Either implementation and ended up with an enumeration and an option. Enumerations are useful for a number of reasons in typed, compiled languages (I guess they provide readability in Python), but they are not the same as monads (of which Either is one type). I am not a Python developer, but as far as I can tell, this implementation gets you none of the real benefits that an a true Either implementation would get you in terms of functional goodness. Am I missing something here? Is there some kind of Python magic that makes this better than just propagating the exception?
  • kortex 10y ago
    I thought the same thing. There are so many other options in the Python toolbox, Enum is far from the first I would reach for. Not sure what the benefit is.
• gaius 10y ago
  I’ll be the first person to admit I have no idea what postmodernism actually means It means it means there's no right or wrong way to do it, everything's subjective. Which is actually the exact opposite of the Python philosophy!
• mixedCase 10y ago
  Postmodern error handling would probably be treating any error as a successful result.
• leshow 10y ago
  I'm a bit confused by this post, the 'enums' in rust are actually 'algebraic data types' and not enumerations like what the author is showing in python. the code here isn't really very equivalent and if anything just shows how much less expressive python is than a language which has proper algebraic types.
• mythrwy 10y ago
  There must be something I don't get. What is the advantage of creating an Enum class to store relationships rather than just using a dictionary or namedtuple? (Arguably) readability? I mean, I see it works but fail to grasp the advantage.
• Animats 10y ago
  This looks like an attempt to use Rust-type error handling in Python. This is pounding a screw. Rust does it that way because Rust doesn't have exceptions. (Although, as with Go, all the heavy machinery for exception unwinding has been retrofitted so "recover" will work. Rust and Go just don't have the syntax for exceptions.) Python doesn't need to become Rust, or Haskell. Writing except Exception as e: is clueless Python. If you're checking for something that went wrong in the outside world, you catch EnvironmentError. Catching Exception catches far too much, including syntax errors. It's sometimes useful to catch Exception at the top level of a program, log an error, and restart, but not in interior code. Note how useless the examples in the article are. It's not like someone is trying to handle all the things that can go wrong in a network connection, or in nested code where you have a network connection and a database transaction open at the same time, either can fail, and you have to unwind properly. Typed language are fine, but this semi-checked addition of types to Python is a mess. Python doesn't need this. I'm in favor of design by contract, typing, and assertions, but not in this broken semi-optional way. Stop Guido before he kills again.
  • quotemstr 10y ago
    When you have a generation of people brought up to believe that exceptions are somehow evil, what do you expect?
    • Animats 10y ago
      I know. Python is one of the few languages to get exceptions right. There's a reasonably sane exception hierarchy, and if you catch something, you get everything in its subtree. The "with" clause and exceptions interact properly, so if you hit a problem closing a resource in a "with", the right stuff happens. Since Python is reference counted/garbage collected, you don't have the ownership problem of exception objects you have in C++. Python isn't big on RAII, so you don't have the exception-in-a-destructor problem (or worse, the exception during GC problem) unless you do something to force that. Python is probably the best type declaration free language around. Trying to make it into something else damages the language. Python has become uncool, though, and so "cool features" are being added that don't quite fit. These changes to Python are so big that this should have been called Python 4. But if Guido had done that, the reply from the big users would have been "Hell, no, we won't go!".
• diminoten 10y ago
  What is a "maybe implementation" and an "either"?