8 ms·

Python as a declarative programming language (2017)

139 points by akashtndn 8y ago

qop 8y ago
Where does this get off on saying torch didn't have traction until pytorch? Are you high? Twitter? NYU? Facebook? Python is a bad technology. Modern python like what's in the article is all about figuring out how to make python code that makes C do all the work, and its pathetic to then take credit for the success that fortune 100 companies and world renowned universities were using before python attached itself to it. Give me a break. If you want to write fast code and interop with C, write Lua and use Torch. If you're doing technical computing or extreme scale numerics, go with Julia. But never python.
mlthoughts2018 8y ago
I was sad that Cython was only given a fleeting nod at the end and numba wasn’t even mentioned. I might even go so far as to say that Cython is the most user-friendly way to write new pure C programs that there is. If you’re about to write some greenfield project in C, seriously consider using Cython, even if you have no desire for any Python entrypoints in the final product. Numba as well is quite impressive: between jitclass, deferred_type, and facilities to target GPUs, it’s crazy how much speedup you can get with little extra code. Overall I think people don’t clearly state the speed trade-off Python presents. Everyone focuses on the “40 times slower” part and nobody points out that “equivalent program in C++” is a glib false equivalence, because a truly equivalent program would have to offer all the same run-time dynamic type manipulation and nearly arbitrarily modifiable object instances involved. What Python really proposes is this: “I’ll give you 100x more flexibility in exchange for 40x worse performance for certain classes of operations.” Don’t want the 40x slowdown? Das cool. Code around it by using extension modules that let you specify homogeneously typed contiguous memory arrays (or bare metal structs like with ctypes or numba’s jitclass), or let you bypass CPython overhead because e.g. you know you don’t need the flexibility offered by the data model looking up a generic python function to call for an __add__ operation, and instead you can arrange for a GIL-releasing call to a pure C function. And so on. I guess my point is that the story should be “look what Python gives you” and then “this flexibility had a cost.” But instead it always feels more like, “this C pseudocode for this tight algorithm looks superficially similar to Python syntax for that algorithm ... but oh boy look how much slower Python is!” —- which, apart from a possible surprise when you have never looked into Python before, should be of interest to badically no one.
- SlowRobotAhead 8y ago
  That’s certainly a fair way to put it. I guess I don’t encounter the type who really get upset that ”Python is too slow!! I’d rather use C++” because that takes a certain type of person who is technically able but practically challenged. I personally find myself surrounded by the opposite peoples.
  - mlthoughts2018 8y ago
    This is the beauty of Cython & numba for me. It lets you selectively target just the small sections of the code that need a speed-up for an understood reason. You can use nice Python profiling tools to gather evidence about which program hotspots truly are too slow, and then just target those for specialization either as extension module functions/classes in Cython (or even a manually written extension module, or a plain C library used via cffi), or as a JIT function or class with numba. You can use the annotation features of Cython & numba to further profile locations that have avoidable CPython API overhead, and it becomes a very evidence-based and iterative optimization cycle. At the end, for 90% of the Python code that was more than fast enough already and didn’t touch performance critical sections, you just got to save all the time & trouble of needing to rewrite it (usually painfully) and debug it and re-test it all in C or C++. This is why, for me, the entire idea that you should start a project out in C++ or C is a giant type of premature optimization in a lot of cases (not all cases but a lot). Did you prototype the boilerplate parts in Python and actually measure the sort of performance hotspots you’ll have to optimize, to a degree that you know for sure that some light sprinkling of Cython & numba certainly won’t work? If not, it’s probably premature optimization to presume you’ll need C or C++ as the primary implementation language.
    - SlowRobotAhead 8y ago
      I use C day in day out for embedded. So you’ll get no complaints from me! What I’ll find interesting is if the CPyrhon idea is ever pushed to embedded. Because right now the state of microPython or Zerynth trying to be a one stop shops for micros but in Python isn’t attractive. Then again, I don’t want or need pythons standard libs - what I really want is all C code with an option to ultra lightweight run scripting in Python. So 90% C, 10% Py. No option exists yet.
orf 8y ago
Python is a great language for this kind of super-glue code. The pure tensorflow snippets are rarely run in isolation, and python makes retrieving, loading and munging data pretty simple when you tap into its rich ecosystem of scientific and utility packages. Python has always been about dipping down to C when needed - just look at the various stdlib features like namedtuple and lru_cache that had optimized C versions introduced once the python versions were deemed useful and performance critical. A commenter below seems outraged about the torch->pytorch change, but really the GitHub contributions and adoptions for each speak for themselves
- GyYZTfWBfQw 8y ago
  > A commenter below seems outraged about the torch->pytorch change, but really the GitHub contributions and adoptions for each speak for themselves What are you trying to say exactly? Could you be more explicit?
andybak 8y ago
I've always been a fan of Django's "Declarative for most stuff. Procedural when you need it" approach. Models and forms are a good example but it permeates the whole framework. It's largely an case of favouring composition over inheritance combined with sensible defaults but it feels declarative for the common cases.
- dkersten 8y ago
  I agree, I think this is a good approach. Common things should be super quick and easy, but when you need to go further, it should get out of the way.
Animats 8y ago
Where's the "declarative" part here? It's common to do number crunching in NumPy with Python as the control layer. But the Python code is imperative.
- dfox 8y ago
  It is declarative in the sense that the imperative code builds up some kind of object tree that declaratively describes the operation that is performed as late as possible. Numpy does not really work this way, but TensorFlow does and viewed from the right PoV this is also how SQLAchemy works.
- joshuamorton 8y ago
  Numpy is declarative in the sense that it is vectorized.
  - Animats 8y ago
    ?
    - joshuamorton 8y ago
      Instead of explicitly writing loops, you decalre an operation like add, and it is looped over a vector for you.
      - Animats 8y ago
        That's just application. "Do this to all that stuff". Imperative, but possibly parallel.
plg 8y ago
tldr: Vectorize your code. Good advice not just for Python but other interpreted languages like MATLAB and R.
myWindoonn 8y ago
For faster Python, try PyPy. It's very compatible, very fast for many workloads, and has been constantly receiving interesting new work for over a decade.
magicalhippo 8y ago
First I saw of this was about 9 years ago, with https://fenicsproject.org/ https://fenicsproject.org/ The Python "front-end" allows you to write something which looks very much like the math you're trying to solve. When evaluated, the Python code generates specialized C++ code for a Python module which is in turn compiled and linked back in, and then the C++ code is executed producing the results. Guess this is old news now but I was pretty blown away by it back then.

139 points by akashtndn 8y ago

qop 8y ago

Where does this get off on saying torch didn't have traction until pytorch? Are you high? Twitter? NYU? Facebook? Python is a bad technology. Modern python like what's in the article is all about figuring out how to make python code that makes C do all the work, and its pathetic to then take credit for the success that fortune 100 companies and world renowned universities were using before python attached itself to it. Give me a break. If you want to write fast code and interop with C, write Lua and use Torch. If you're doing technical computing or extreme scale numerics, go with Julia. But never python.

mlthoughts2018 8y ago

I was sad that Cython was only given a fleeting nod at the end and numba wasn’t even mentioned. I might even go so far as to say that Cython is the most user-friendly way to write new pure C programs that there is. If you’re about to write some greenfield project in C, seriously consider using Cython, even if you have no desire for any Python entrypoints in the final product. Numba as well is quite impressive: between jitclass, deferred_type, and facilities to target GPUs, it’s crazy how much speedup you can get with little extra code. Overall I think people don’t clearly state the speed trade-off Python presents. Everyone focuses on the “40 times slower” part and nobody points out that “equivalent program in C++” is a glib false equivalence, because a truly equivalent program would have to offer all the same run-time dynamic type manipulation and nearly arbitrarily modifiable object instances involved. What Python really proposes is this: “I’ll give you 100x more flexibility in exchange for 40x worse performance for certain classes of operations.” Don’t want the 40x slowdown? Das cool. Code around it by using extension modules that let you specify homogeneously typed contiguous memory arrays (or bare metal structs like with ctypes or numba’s jitclass), or let you bypass CPython overhead because e.g. you know you don’t need the flexibility offered by the data model looking up a generic python function to call for an __add__ operation, and instead you can arrange for a GIL-releasing call to a pure C function. And so on. I guess my point is that the story should be “look what Python gives you” and then “this flexibility had a cost.” But instead it always feels more like, “this C pseudocode for this tight algorithm looks superficially similar to Python syntax for that algorithm ... but oh boy look how much slower Python is!” —- which, apart from a possible surprise when you have never looked into Python before, should be of interest to badically no one.

SlowRobotAhead 8y ago
That’s certainly a fair way to put it. I guess I don’t encounter the type who really get upset that ”Python is too slow!! I’d rather use C++” because that takes a certain type of person who is technically able but practically challenged. I personally find myself surrounded by the opposite peoples.
- mlthoughts2018 8y ago
  This is the beauty of Cython & numba for me. It lets you selectively target just the small sections of the code that need a speed-up for an understood reason. You can use nice Python profiling tools to gather evidence about which program hotspots truly are too slow, and then just target those for specialization either as extension module functions/classes in Cython (or even a manually written extension module, or a plain C library used via cffi), or as a JIT function or class with numba. You can use the annotation features of Cython & numba to further profile locations that have avoidable CPython API overhead, and it becomes a very evidence-based and iterative optimization cycle. At the end, for 90% of the Python code that was more than fast enough already and didn’t touch performance critical sections, you just got to save all the time & trouble of needing to rewrite it (usually painfully) and debug it and re-test it all in C or C++. This is why, for me, the entire idea that you should start a project out in C++ or C is a giant type of premature optimization in a lot of cases (not all cases but a lot). Did you prototype the boilerplate parts in Python and actually measure the sort of performance hotspots you’ll have to optimize, to a degree that you know for sure that some light sprinkling of Cython & numba certainly won’t work? If not, it’s probably premature optimization to presume you’ll need C or C++ as the primary implementation language.
  - SlowRobotAhead 8y ago
    I use C day in day out for embedded. So you’ll get no complaints from me! What I’ll find interesting is if the CPyrhon idea is ever pushed to embedded. Because right now the state of microPython or Zerynth trying to be a one stop shops for micros but in Python isn’t attractive. Then again, I don’t want or need pythons standard libs - what I really want is all C code with an option to ultra lightweight run scripting in Python. So 90% C, 10% Py. No option exists yet.

orf 8y ago

Python is a great language for this kind of super-glue code. The pure tensorflow snippets are rarely run in isolation, and python makes retrieving, loading and munging data pretty simple when you tap into its rich ecosystem of scientific and utility packages. Python has always been about dipping down to C when needed - just look at the various stdlib features like namedtuple and lru_cache that had optimized C versions introduced once the python versions were deemed useful and performance critical. A commenter below seems outraged about the torch->pytorch change, but really the GitHub contributions and adoptions for each speak for themselves

GyYZTfWBfQw 8y ago
> A commenter below seems outraged about the torch->pytorch change, but really the GitHub contributions and adoptions for each speak for themselves What are you trying to say exactly? Could you be more explicit?

andybak 8y ago

I've always been a fan of Django's "Declarative for most stuff. Procedural when you need it" approach. Models and forms are a good example but it permeates the whole framework. It's largely an case of favouring composition over inheritance combined with sensible defaults but it feels declarative for the common cases.

dkersten 8y ago
I agree, I think this is a good approach. Common things should be super quick and easy, but when you need to go further, it should get out of the way.

Animats 8y ago

Where's the "declarative" part here? It's common to do number crunching in NumPy with Python as the control layer. But the Python code is imperative.

dfox 8y ago
It is declarative in the sense that the imperative code builds up some kind of object tree that declaratively describes the operation that is performed as late as possible. Numpy does not really work this way, but TensorFlow does and viewed from the right PoV this is also how SQLAchemy works.
joshuamorton 8y ago
Numpy is declarative in the sense that it is vectorized.
- Animats 8y ago
  ?
  - joshuamorton 8y ago
    Instead of explicitly writing loops, you decalre an operation like add, and it is looped over a vector for you.
    - Animats 8y ago
      That's just application. "Do this to all that stuff". Imperative, but possibly parallel.

plg 8y ago

tldr: Vectorize your code. Good advice not just for Python but other interpreted languages like MATLAB and R.

myWindoonn 8y ago

For faster Python, try PyPy. It's very compatible, very fast for many workloads, and has been constantly receiving interesting new work for over a decade.

magicalhippo 8y ago

First I saw of this was about 9 years ago, with https://fenicsproject.org/ https://fenicsproject.org/ The Python "front-end" allows you to write something which looks very much like the math you're trying to solve. When evaluated, the Python code generates specialized C++ code for a Python module which is in turn compiled and linked back in, and then the C++ code is executed producing the results. Guess this is old news now but I was pretty blown away by it back then.