9 ms·

Beartype: Fast runtime type checking in Python

117 points by azhenley 5y ago

riofoxx 5y ago
I can help you earn $ 6.0,0.0.0 with $ 2.5.0.0 in a w.e.e.k .t.h.r.o.u.g.h mr Harland w.h.o h.e.l.p.e.d m.e g.e.t the b.e.s.t in c.r.y.p.t.o t.r.a.d.i.n.g.. Contact via WhatsApp +13052395906)
topoloneuron 5y ago
Can I use this on my full codebase all at once, without inserting "from beartype import beartype" into each file and decorating every single function and method?
- Joker_vD 5y ago
  I guess you could, near the beginning of your program's entry point, decorate the importlib's functions to decorate the top-level functions of imported modules with "@beartype" decorator. Or there is always find+grep+sed, if you prefer a different style of madness.
zohch 5y ago
What always annoys me to no end is when people complain that python type hints are not runtime enforced, when this is quite normal for most statically typed languages: C++, go, rust. Python already had runtime type checking mechanisms and the use of these does inform the behavior of static type checkers for python like mypy. All that being said, Beartype is really nifty and seems quite useful.
- mjburgess 5y ago
  The spirit of a type-system is that invalid code never runs. The compilation step in static languages type-checks such that the compiled-instructions wont run if the source-code is invalid. Since this isnt how python works, one would reasonably expect the spirit of a type system to be preserved -- ie., to enforce runtime checks.
  - zohch 5y ago
    > The spirit of a type-system is that invalid code never runs. > The compilation step in static languages type-checks such that the compiled-instructions wont run if the source-code is invalid. Nothing is forcing anyone to run code that does not pass type checking with `mypy --strict`. > Since this isnt how python works, one would reasonably expect the spirit of a type system to be preserved -- ie., to enforce runtime checks. Python already had mechanisms for this, which again, as I said, already informs the static type checking mechanisms. Also, no, this is not what I expect from the type hints, this is not what I felt was missing from python before type hints. What I want from type hints and in python in general is static type checking, I want to know, without executing one single line of code, if I have type errors, that is the key selling point of mypy and type hints to me and the key deficiency that python had without type hints and mypy. I already had every ability to do runtime type checking in python before type hints and mypy, and they were woefully inadequate in comparison to static type checking.
    - lozenge 5y ago
      Is the standard library type checked? Are there still mistakes in the typeshed like 2**-1 is int?
      - zohch 5y ago
        > Is the standard library type checked? There are type hints for the standard library and using `--strict` works just fine if that is all you are using. This does not preclude bugs from the standard library or errors in types but bugs in a statically typed language's compiler also does not defeat the purpose of static typing either and having a static typed language does not eliminate the possibility of bugs in the compiler or standard library either. It seems people forget that reinterpret_cast, void * (c++), Object (java), (interface{}) (golang) exists. If all you are using is the standard library then you will be able to write much safer code in python than in golang, given it passes static type validation. At least python's type system has generics. > Are there still mistakes in the typeshed like 2*-1 is int? I'm sure there are still mistakes, just like I'm sure there are still bugs in most compilers.
- guenthert 5y ago
  Am I unreasonable in hoping for optional type hints to be taken advantage of by a compiler to produce correct, optimized code, similar to what (sufficiently smart) Common Lisp compilers do?
- darthrupert 5y ago
  This comment is in the top #10 of the stupidest things I've read on this site. Congratulations! As personal advice, I suggest not being annoyed about a thing you're totally wrong about.
a-dub 5y ago
> When decorated by typeguard, every call to that function checks every integer nested in that list. > When decorated by beartype, every call to the same function checks only a single random integer contained in a single random nested list contained in a single random nested list contained in that parent list. This is what we mean by the quaint phrase "one-way random walk over the expected data structure." so it's not actually faster, it just doesn't actually check the types or even the complete shape of the data structure. seems like it would catch obvious problems, but make more subtle ones way harder to find as it creates a sense that "oh, the types are checked" when they aren't really.
- eru 5y ago
  Yes, though as eg QuickCheck shows, randomised testing is unreasonably effective in practice.
  - a-dub 5y ago
    assuming you're sampling the same distribution. every call to a function can have different arguments. this seems to make the assumption that the types and shape don't change so you can amortize the cost of type checking over multiple function calls... but they can, and do, sometimes with low probability and those are the hard bugs to solve. especially when you think types are being checked.
    - eru 5y ago
      I suspect you are reading different things into my comment than I wanted to convey. Your argument is perfectly valid, if you try to steelman my argument into something like a mathematical argument. But it wasn't meant that way. I meant exactly what I said: empirically, in practice, randomised checking is unreasonably effective. You illustrated the 'unreasonable' part.
      - a-dub 5y ago
        i'd never seen this quickcheck thing before... it's basically fuzzing built into unit tests, neat... it is, however, quite different from a type checker though. where a fuzz tester generates random inputs to try and break a function... a type checker enforces assertions on data that is actually flowing through the system. i think my big beef with this library is that it does so probabilistically, while marketing itself as something that does complete checks and has been highly optimized. it's not a fast assertion checker, it is one that employs random sampling... that's fine, but it should be extremely upfront about that as this misunderstanding has the potential to cause a lot of pain on top of already painful situations.
dwohnitmok 5y ago
I don't understand the section on the coupon collector's problem and its relevance to beartype. The coupon collector problem is talking about the expected number of random beartype-style checks it would take to fully check every element in a collection (basically how long on average it would take to do the same thing typeguard does). But I fail to see how that's relevant to beartype. Beartype's constant-time runtime has nothing to do with the coupon collector problem. It has to do with the simple fact that no matter how large the container beartype will only sample one random element from that container. Likewise, when the README mentions > Since h ≥ 1, beartype samples at least as many items each call as assumed in the usual coupon collector's problem and thus paradoxically takes a fewer number of calls on average to check all items of a container containing arbitrarily many subcontainers as it does to check all items of a container containing no subcontainers. that's just because beartype happens to check one element from each level of nesting of a container so a deeply nested container just gets more checks in a single call than a non-nested container right? If I compare a list with 50 elements and a list with 50 levels of nesting in beartype but a single element underneath all that nesting, clearly the latter only requires a single call to fully type-check while the former takes 225 calls to fully check on average. But that's just because beartype is doing a different amount of work in each case, since in the latter case beartype samples all 50 levels all at once. Unless I'm missing something the entire discussion on the coupon collector problem seems irrelevant.
- spicybright 5y ago
  Oof, if it really picked elements at random to type check, I'd actually say that'd be "unbearable"
  - zbentley 5y ago
    That's what it does.
SjorsVG 5y ago
Runtime type checking.. What a dreadful idea
- tirpen 5y ago
  Much worse than static type checking. But infinitely better than no type checking.
- nhumrich 5y ago
  Most places I see runtime type checking, is when checking the type of incoming data, aka, Json, or if the function is a library and you are helping the callers use the correct types. Go/java, etc all do this by default. You read an int from a in memory untyped object, and you get an error if it's not actually an int.
ReleaseCandidat 5y ago
> Basically, beartype type-checks randomly sampled data. So, it doesn't really type check and the chance that l: list[str] = ["good", 1] typechecks without errors is 50%?
- ddalex 5y ago
  Pretty much, yes; however this would flag up as a flaky integration test, prompted you to fix if you have a hard rule about no flakyness in tests
MichaelMoser123 5y ago
Are there any code editors/editor plugins that make effective use of python type hints? The python plugins for visual studio code didn't make much use of the type hints, last time that I checked. i was hoping that type hints would be used for code completion, but that doesn't seem to be the case.
- aldanor 5y ago
  PyCharm has its own engine and while not perfect, it's fairly good.
  - quietbritishjim 5y ago
    And when it fails to infer the type of something you can add `assert isinstance(foo, T)`, and PyCharm will make use of that.
    - ramraj07 5y ago
      This will throw an error in runtime for bad types, which might or might not be your intention (unless of course you build the package, which only makes it all the more harder to debug if this is where your program fails at this step). So, just don’t use assert outside of tests?
      - quietbritishjim 5y ago
        For sure, adding that line of code will affect the runtime as well as type hinting in the IDE. Certainly don't use it unless you're sure that the asserted condition is actually always true (or not sure and want to know if it isn't!).
  - willvarfar 5y ago
    I’m not a huge fan of pycharm’s inference and autocomplete. It’s not that it’s bad, I was really just that I was hoping for more. There is so much that could be done eg IDEs could enable proper exhaustive signature checks when running code in debuggers. Pypy could probably do some really good cheap checks in debug mode too. And if not checking while debugging, they could at least highlight failed hints in the call stack when the debugger stops. A long long time ago before MyPy I made a runtime checker called Obiwan. I always preferred its syntax to MyPy, eg [int] instead of list[int]. I then got into playing with SSA within functions and that really helped type checking. So there’s loads more I think could be done to do what typescript did for JavaScript but to python.
joncp 5y ago
Whatever your opinion of the library or runtime type checking in general, you've got to admit that's a helluva README they've put together.
- dec0dedab0de 5y ago
  absolutely, I don't like type checking at all, and especially not runtime type checking, but the README has made me want to try it.
nawgz 5y ago
So does it truly take the level of resources of Microsoft to make a good type system? It’s strange to see TypeScript infect the whole JS world while Python remains fractured at best with regards to type systems
- qsort 5y ago
  ? This isn't a fork of the type system. Python's type annotations are formally part of the language, a program correctly typing on mypy will also typecheck on Pylance etc. The OP is a different implementation of type checking, runtime as opposed to static. Python's problem is that there is no equivalent of @types/*, which is in fact a notable issue but also entirely orthogonal.
  - nawgz 5y ago
    > Python's type annotations are formally part of the language > Python's problem is that there is no equivalent of @types/* Yes, this is exactly why I asked: > Microsoft to make a good type system Operative word "good". A type system that loses types for libraries has, to say the least, extremely limited value. More to the point, whatever typing Python has is wildly unpopular amongst Python engineers, and when I talk to my Python-using colleagues they seem to have no clue they can even use types. Perhaps it's also a community problem in this way.
Olreich 5y ago
> emphasizing efficiency, portability, and thrilling puns. Results: Runs 1000x slower than not using it, but 200x faster than the competition! So only use this in correctness-critical paths, or use a language that supports strong typing at compile time.
exabrial 5y ago
I've yet to be unable to teach someone what types are and how to use them, including some 10 year olds. I'm not really sure why this is still a thing at all.
- guenthert 5y ago
  Neither plain Python nor beartype-enhanced Python is type-less like FORTH, BCPL or BLISS, so I don't quite grok your comment.
dec0dedab0de 5y ago
Could someone explain the benefit of runtime type checking? I thought the whole point of type checking was to catch errors before runtime. If it's already running, wouldn't it be better to just let it keep running and see what happens? Is the idea that you would run this during your tests and it would give you more insight than a static type checker? EDIT: The answer is in the FAQ https://github.com/beartype/beartype#frequently-asked-questions-faq https://github.com/beartype/beartype#frequently-asked-questi... It seems like it would never cause an error, and it's really for type checking dynamic code. I'm still trying to wrap my head around it, but it seems promising so far.
- dharmab 5y ago
  Type hints aren't actually enforced in Python. So for example a third party can import your code and call functions with the wrong type. Because of duck typing, sometimes this won't result in an error but could still cause a bug. You can explicitly check types with isinstance(), this library automates that.
  - dec0dedab0de 5y ago
    Type hints aren't actually enforced in Python. So for example a third party can import your code and call functions with the wrong type. Because of duck typing, sometimes this won't result in an error but could still cause a bug. Sure, but it also might work, if you've already made it through testing and deployment, surely you don't want to cause a new production bug because there might be another bug. What I would really want at runtime is something that automatically casts and validates. As in, I don't care if it's a string, I care that if I wrap it in str it matches this regex.
    - mixmastamyk 5y ago
      Finding bugs as early as possible, rather than soldiering on, is most often what you want. The reason is that the farther the code executes, the details and error messages of what went wrong in the past can get fuzzier and harder to diagnose. If you're doing ducktyping... it should be specified explicitly, believe Python typing calls it a "protocol."
esjeon 5y ago
From my guesses, this is mainly designed for data engineering, where you do run the same operation on a lot of entries. N is very very big, so it will negate the probabilistic nature of the checker. Other than that, this is simply useless. Note that, if your function arguments are primitives (int, str, etc), you can design your function to fail early when the type is incorrect. Things can be trickier with objects, but still doable. So, no, this won't be useful for anything w/ deep logic. Also, half serious half joke, but you should reconsider your language choice if you need stuffs like this, because Python isn't good at both performance and type safety.

117 points by azhenley 5y ago

I can help you earn $ 6.0,0.0.0 with $ 2.5.0.0 in a w.e.e.k .t.h.r.o.u.g.h mr Harland w.h.o h.e.l.p.e.d m.e g.e.t the b.e.s.t in c.r.y.p.t.o t.r.a.d.i.n.g.. Contact via WhatsApp +13052395906)

topoloneuron 5y ago

Can I use this on my full codebase all at once, without inserting "from beartype import beartype" into each file and decorating every single function and method?

Joker_vD 5y ago
I guess you could, near the beginning of your program's entry point, decorate the importlib's functions to decorate the top-level functions of imported modules with "@beartype" decorator. Or there is always find+grep+sed, if you prefer a different style of madness.

zohch 5y ago

What always annoys me to no end is when people complain that python type hints are not runtime enforced, when this is quite normal for most statically typed languages: C++, go, rust. Python already had runtime type checking mechanisms and the use of these does inform the behavior of static type checkers for python like mypy. All that being said, Beartype is really nifty and seems quite useful.

mjburgess 5y ago
The spirit of a type-system is that invalid code never runs. The compilation step in static languages type-checks such that the compiled-instructions wont run if the source-code is invalid. Since this isnt how python works, one would reasonably expect the spirit of a type system to be preserved -- ie., to enforce runtime checks.
- zohch 5y ago
  > The spirit of a type-system is that invalid code never runs. > The compilation step in static languages type-checks such that the compiled-instructions wont run if the source-code is invalid. Nothing is forcing anyone to run code that does not pass type checking with `mypy --strict`. > Since this isnt how python works, one would reasonably expect the spirit of a type system to be preserved -- ie., to enforce runtime checks. Python already had mechanisms for this, which again, as I said, already informs the static type checking mechanisms. Also, no, this is not what I expect from the type hints, this is not what I felt was missing from python before type hints. What I want from type hints and in python in general is static type checking, I want to know, without executing one single line of code, if I have type errors, that is the key selling point of mypy and type hints to me and the key deficiency that python had without type hints and mypy. I already had every ability to do runtime type checking in python before type hints and mypy, and they were woefully inadequate in comparison to static type checking.
  - lozenge 5y ago
    Is the standard library type checked? Are there still mistakes in the typeshed like 2**-1 is int?
    - zohch 5y ago
      > Is the standard library type checked? There are type hints for the standard library and using `--strict` works just fine if that is all you are using. This does not preclude bugs from the standard library or errors in types but bugs in a statically typed language's compiler also does not defeat the purpose of static typing either and having a static typed language does not eliminate the possibility of bugs in the compiler or standard library either. It seems people forget that reinterpret_cast, void * (c++), Object (java), (interface{}) (golang) exists. If all you are using is the standard library then you will be able to write much safer code in python than in golang, given it passes static type validation. At least python's type system has generics. > Are there still mistakes in the typeshed like 2*-1 is int? I'm sure there are still mistakes, just like I'm sure there are still bugs in most compilers.
guenthert 5y ago
Am I unreasonable in hoping for optional type hints to be taken advantage of by a compiler to produce correct, optimized code, similar to what (sufficiently smart) Common Lisp compilers do?
darthrupert 5y ago
This comment is in the top #10 of the stupidest things I've read on this site. Congratulations! As personal advice, I suggest not being annoyed about a thing you're totally wrong about.

a-dub 5y ago

> When decorated by typeguard, every call to that function checks every integer nested in that list. > When decorated by beartype, every call to the same function checks only a single random integer contained in a single random nested list contained in a single random nested list contained in that parent list. This is what we mean by the quaint phrase "one-way random walk over the expected data structure." so it's not actually faster, it just doesn't actually check the types or even the complete shape of the data structure. seems like it would catch obvious problems, but make more subtle ones way harder to find as it creates a sense that "oh, the types are checked" when they aren't really.

eru 5y ago
Yes, though as eg QuickCheck shows, randomised testing is unreasonably effective in practice.
- a-dub 5y ago
  assuming you're sampling the same distribution. every call to a function can have different arguments. this seems to make the assumption that the types and shape don't change so you can amortize the cost of type checking over multiple function calls... but they can, and do, sometimes with low probability and those are the hard bugs to solve. especially when you think types are being checked.
  - eru 5y ago
    I suspect you are reading different things into my comment than I wanted to convey. Your argument is perfectly valid, if you try to steelman my argument into something like a mathematical argument. But it wasn't meant that way. I meant exactly what I said: empirically, in practice, randomised checking is unreasonably effective. You illustrated the 'unreasonable' part.
    - a-dub 5y ago
      i'd never seen this quickcheck thing before... it's basically fuzzing built into unit tests, neat... it is, however, quite different from a type checker though. where a fuzz tester generates random inputs to try and break a function... a type checker enforces assertions on data that is actually flowing through the system. i think my big beef with this library is that it does so probabilistically, while marketing itself as something that does complete checks and has been highly optimized. it's not a fast assertion checker, it is one that employs random sampling... that's fine, but it should be extremely upfront about that as this misunderstanding has the potential to cause a lot of pain on top of already painful situations.

dwohnitmok 5y ago

I don't understand the section on the coupon collector's problem and its relevance to beartype. The coupon collector problem is talking about the expected number of random beartype-style checks it would take to fully check every element in a collection (basically how long on average it would take to do the same thing typeguard does). But I fail to see how that's relevant to beartype. Beartype's constant-time runtime has nothing to do with the coupon collector problem. It has to do with the simple fact that no matter how large the container beartype will only sample one random element from that container. Likewise, when the README mentions > Since h ≥ 1, beartype samples at least as many items each call as assumed in the usual coupon collector's problem and thus paradoxically takes a fewer number of calls on average to check all items of a container containing arbitrarily many subcontainers as it does to check all items of a container containing no subcontainers. that's just because beartype happens to check one element from each level of nesting of a container so a deeply nested container just gets more checks in a single call than a non-nested container right? If I compare a list with 50 elements and a list with 50 levels of nesting in beartype but a single element underneath all that nesting, clearly the latter only requires a single call to fully type-check while the former takes 225 calls to fully check on average. But that's just because beartype is doing a different amount of work in each case, since in the latter case beartype samples all 50 levels all at once. Unless I'm missing something the entire discussion on the coupon collector problem seems irrelevant.

spicybright 5y ago
Oof, if it really picked elements at random to type check, I'd actually say that'd be "unbearable"
- zbentley 5y ago
  That's what it does.

SjorsVG 5y ago

Runtime type checking.. What a dreadful idea

tirpen 5y ago
Much worse than static type checking. But infinitely better than no type checking.
nhumrich 5y ago
Most places I see runtime type checking, is when checking the type of incoming data, aka, Json, or if the function is a library and you are helping the callers use the correct types. Go/java, etc all do this by default. You read an int from a in memory untyped object, and you get an error if it's not actually an int.

ReleaseCandidat 5y ago

> Basically, beartype type-checks randomly sampled data. So, it doesn't really type check and the chance that l: list[str] = ["good", 1] typechecks without errors is 50%?

ddalex 5y ago
Pretty much, yes; however this would flag up as a flaky integration test, prompted you to fix if you have a hard rule about no flakyness in tests

MichaelMoser123 5y ago

Are there any code editors/editor plugins that make effective use of python type hints? The python plugins for visual studio code didn't make much use of the type hints, last time that I checked. i was hoping that type hints would be used for code completion, but that doesn't seem to be the case.

aldanor 5y ago
PyCharm has its own engine and while not perfect, it's fairly good.
- quietbritishjim 5y ago
  And when it fails to infer the type of something you can add `assert isinstance(foo, T)`, and PyCharm will make use of that.
  - ramraj07 5y ago
    This will throw an error in runtime for bad types, which might or might not be your intention (unless of course you build the package, which only makes it all the more harder to debug if this is where your program fails at this step). So, just don’t use assert outside of tests?
    - quietbritishjim 5y ago
      For sure, adding that line of code will affect the runtime as well as type hinting in the IDE. Certainly don't use it unless you're sure that the asserted condition is actually always true (or not sure and want to know if it isn't!).
- willvarfar 5y ago
  I’m not a huge fan of pycharm’s inference and autocomplete. It’s not that it’s bad, I was really just that I was hoping for more. There is so much that could be done eg IDEs could enable proper exhaustive signature checks when running code in debuggers. Pypy could probably do some really good cheap checks in debug mode too. And if not checking while debugging, they could at least highlight failed hints in the call stack when the debugger stops. A long long time ago before MyPy I made a runtime checker called Obiwan. I always preferred its syntax to MyPy, eg [int] instead of list[int]. I then got into playing with SSA within functions and that really helped type checking. So there’s loads more I think could be done to do what typescript did for JavaScript but to python.

joncp 5y ago

Whatever your opinion of the library or runtime type checking in general, you've got to admit that's a helluva README they've put together.

dec0dedab0de 5y ago
absolutely, I don't like type checking at all, and especially not runtime type checking, but the README has made me want to try it.

nawgz 5y ago

So does it truly take the level of resources of Microsoft to make a good type system? It’s strange to see TypeScript infect the whole JS world while Python remains fractured at best with regards to type systems

qsort 5y ago
? This isn't a fork of the type system. Python's type annotations are formally part of the language, a program correctly typing on mypy will also typecheck on Pylance etc. The OP is a different implementation of type checking, runtime as opposed to static. Python's problem is that there is no equivalent of @types/*, which is in fact a notable issue but also entirely orthogonal.
- nawgz 5y ago
  > Python's type annotations are formally part of the language > Python's problem is that there is no equivalent of @types/* Yes, this is exactly why I asked: > Microsoft to make a good type system Operative word "good". A type system that loses types for libraries has, to say the least, extremely limited value. More to the point, whatever typing Python has is wildly unpopular amongst Python engineers, and when I talk to my Python-using colleagues they seem to have no clue they can even use types. Perhaps it's also a community problem in this way.

Olreich 5y ago

> emphasizing efficiency, portability, and thrilling puns. Results: Runs 1000x slower than not using it, but 200x faster than the competition! So only use this in correctness-critical paths, or use a language that supports strong typing at compile time.

exabrial 5y ago

I've yet to be unable to teach someone what types are and how to use them, including some 10 year olds. I'm not really sure why this is still a thing at all.

guenthert 5y ago
Neither plain Python nor beartype-enhanced Python is type-less like FORTH, BCPL or BLISS, so I don't quite grok your comment.

dec0dedab0de 5y ago

Could someone explain the benefit of runtime type checking? I thought the whole point of type checking was to catch errors before runtime. If it's already running, wouldn't it be better to just let it keep running and see what happens? Is the idea that you would run this during your tests and it would give you more insight than a static type checker? EDIT: The answer is in the FAQ https://github.com/beartype/beartype#frequently-asked-questions-faq https://github.com/beartype/beartype#frequently-asked-questi... It seems like it would never cause an error, and it's really for type checking dynamic code. I'm still trying to wrap my head around it, but it seems promising so far.

dharmab 5y ago
Type hints aren't actually enforced in Python. So for example a third party can import your code and call functions with the wrong type. Because of duck typing, sometimes this won't result in an error but could still cause a bug. You can explicitly check types with isinstance(), this library automates that.
- dec0dedab0de 5y ago
  Type hints aren't actually enforced in Python. So for example a third party can import your code and call functions with the wrong type. Because of duck typing, sometimes this won't result in an error but could still cause a bug. Sure, but it also might work, if you've already made it through testing and deployment, surely you don't want to cause a new production bug because there might be another bug. What I would really want at runtime is something that automatically casts and validates. As in, I don't care if it's a string, I care that if I wrap it in str it matches this regex.
  - mixmastamyk 5y ago
    Finding bugs as early as possible, rather than soldiering on, is most often what you want. The reason is that the farther the code executes, the details and error messages of what went wrong in the past can get fuzzier and harder to diagnose. If you're doing ducktyping... it should be specified explicitly, believe Python typing calls it a "protocol."

esjeon 5y ago

From my guesses, this is mainly designed for data engineering, where you do run the same operation on a lot of entries. N is very very big, so it will negate the probabilistic nature of the checker. Other than that, this is simply useless. Note that, if your function arguments are primitives (int, str, etc), you can design your function to fail early when the type is incorrect. Things can be trickier with objects, but still doable. So, no, this won't be useful for anything w/ deep logic. Also, half serious half joke, but you should reconsider your language choice if you need stuffs like this, because Python isn't good at both performance and type safety.