10 ms·

Python Programming and Numerical Methods: A Guide for Engineers and Scientists

267 points by happy-go-lucky 6y ago

mud_dauber 6y ago
Thank you for posting this.
hahahahe 6y ago
I would like to see a polished Python package for data, math, and science, similar to Mathematica. Or perhaps Wolfram can start supporting Python? I’d actually use that over Pandas/Conda.
- leephillips 6y ago
  Do you know about Sage? Or are you thinking about something different?
- gatestore 6y ago
  It is now possible to use Python inside Mathematica. Take a look at: https://blog.wolfram.com/2019/05/16/announcing-the-wolfram-client-library-for-python/ https://blog.wolfram.com/2019/05/16/announcing-the-wolfram-c... https://reference.wolfram.com/language/WolframClientForPython/ https://reference.wolfram.com/language/WolframClientForPytho... It is also possible to run the Wolfram Kernel inside a Jupyter Notebook: https://github.com/WolframResearch/WolframLanguageForJupyter https://github.com/WolframResearch/WolframLanguageForJupyter
unnouinceput 6y ago
I love Python, it gives me the best jobs of my life. Usually is like this. Customer happens to stumble some easy 3 lines program that solves the problem in an easy subset and after that it will create a full program around that one with all the blings. Then goes into production after a month and 15k USD lighter and its users are using it for full scale and the solving time now is in hours. Then it will search for an expert to solve the impossible problem of speeding up Python and curses at all the Indians that are scamming him and all experts who ask for another 15k USD to actually implement the program correctly. I shit you not, last one was a data scientist who paid like $3/h some Indian dude that got the job done in 2 days, took the money and contract was closed under those terms. Then this data scientist was all over his job posting crying that, while the initial matrix was in dozens of columns/rows now the same Python program would take 3 days when throwing at it a matrix that was having millions of columns/rows in size. I mean, it's several orders of magnitude higher, I was amazed that the program actually finished the job after 3 days instead of just crashing. So I had to drill into his thick skull the idea that while he initially went to war against Lichtenstein and won, definitely cannot win against US army using the same weapon. Only after that he agreed to scrap the Python program altogether and finally go with C++, because speed does matter in the end. Like I said, I love Python, especially for anything math related.
- alokrai 6y ago
  I wonder why it is necessary to mention "Indians that are scamming" him as if scamming is something only Indians do? Should I be careful only when dealing with Indian consultants and presume others won't lie or cheat? edit: grammar
  - unnouinceput 6y ago
    Probably because Indians are the ones that are leaders in scamming? Also, if you paid attention, I've mentioned the initial Python dev was an honest Indian, he did his job within terms of contract. The data scientist, which in this case was a Canadian, is solely at fault for not disclosing full scale to the Indian one.
    - alokrai 6y ago
      >Probably because Indians are the ones that are leaders in scamming? I could respond to it but there is not much to be said. I think you have made my point.
    - rramadass 6y ago
      You really seem to have a way with words where, while the point of your original premise (i.e. lack of requirements, technical competence) is valid you have managed to bury that under completely unrelated negative and racist stereotyping.
  - mlN90 6y ago
    >as if [bad thing] is something only [group] do? This is such a nonsense statement and it comes up everytime someone describes someones nationality in correlation with something awful. Honest question, when you read the line "[...]Chinese that are good at math [...]" do you read that all Chinese people are math-wizards? Because that is not what it says. What about "[...] black people that are fast runners [...]"? all 3 examples, including the one that turned on your torch of virtue, describes a sub-set of a group, the primary attribute of said group and nothing more. If anything the implication that the guy you were replied to is somehow biased and "racist" against the absolute-plague-tier of disproportional scammers coming out of India is based on nothing but your inability to differentiate between "broadspectrum-racism" and "critism of a subset of a group"
deleted 6y ago
[deleted]
amitport 6y ago
This looks like a great resouce, thanks! Note that in practice you can teach part II (numerical methods) without 90% of part I (python programming) given that your audience has some minimal programming experience (not necessarily with python).
physicsguy 6y ago
Not sure if this is more a difference between UK/US universities as this says it's targeted at "serious undergraduates or graduate students", but we covered all of this material in the 1st and 2nd year undergraduate courses at the Universities I've studied at or taught at here.
- dataflow 6y ago
  It matters what you're majoring in and focusing on. There's only so much time to teach so much material in CS, for example, and there's not much desire from the faculty to teach numerical diff eq solving (and certainly none from most CS students to learn it). The ones that do go to grad school tend to need these more, but they haven't seen them in undergrad so they end up learning what they're lacking in grad school (or late undergrad). That said, do you guys really learn/teach the shooting method, Lagrange interpolation, predictor-corrector methods, QR factorization, etc. as 1st/2nd-year undergrads? For what majors? I feel like this would come at the cost of deferring other topics to later years.
  - physicsguy 6y ago
    We don't generally have majors/minors in the UK, so perhaps this is part of it. You do a degree in for e.g. Maths/Physics/Mech Eng, and all of your courses are in that. Yes, we also covered other stuff like splines, finite difference method, too. I think Finite Element was third year where I last taught.
    - dataflow 6y ago
      > You do a degree in for e.g. Maths/Physics/Mech Eng, and all of your courses are in that. "Major" refers to the same things you mentioned (physics, math, EE, etc.), it's not any different in the US. > Yes, we also covered other stuff like splines, finite difference method, too. I think Finite Element was third year where I last taught. Sorry to repeat the question, but again -- in what major (or "for what degree" in your terminology) did you study this? (CS? MechE? Physics? EE?)
gostsamo 6y ago
Time to first "use julia instead" comment: 3, 2, 1...
- Mauricebranagh 6y ago
  If its a pure numerical problem just use Fortran and buy the NAG librarys. oooh just checked and NAG do python library's now as well so that might be teh best solution.
  - gostsamo 6y ago
    I'm actually a python developer, but I might be the only one appreciating the identity investment that people on both sides put in their chosen language.
- deleted 6y ago
  [deleted]
- eternauta3k 6y ago
  You'll have to wait a bit longer, the interpreter is loading :)
- cambalache 6y ago
  Now that people are "writing games and web servers" in Julia (as per some commenters) and that the use case list for Rust grows exponentially I cannot wait for the most epic of collisions, Julia vs Rust, astronomers must be rubbing their hands for this one.
  - gostsamo 6y ago
    It is funny, because those kinds of arguments are totally meaningless. Languages are tools and a true craftsman would master many of them and use them when they are the right tool for the job. Instead, people who know only one tool try to convince themselves and everyone else that the tool they have is the only one they need and thus have it all backward.
ngcc_hk 6y ago
Ch 15 I tried the code. Forget to do plt.close() and pythonista has warning. Btw, not sure how to get the label work.
reikonomusha 6y ago
I’ve been a sort of oddball in the numerical/scientific computing world in the past 10 or so years in that I mostly reject using Python for serious numerical programming. This seems to be against the common wisdom of my PhD scientist colleagues who grab for Python any and every chance they get. The Python community has done an amazing job making a suite of scientific software. Numerical libraries and plotting are excellent in Python. Sympy and Sage are both very ambitious, and in my opinion, quite successful projects. So what’s the problem? Longevity and stability. Make fun of it all you want, but old crusty FORTRAN code, and in the past 20 years C and C++, are the bedrock of serious scientific applications. Python usually isn’t. I’m not confident enough to say why with certainty, but maybe it has to do with the fact that Python APIs and type signatures (whether written out or not) are too handwavy and easy to change without detection. In addition to this, I rarely find that people, scientists especially, are writing applications in Python. Python is too hard to deploy as an application sane people can use, is a mess to configure, and difficult to make efficient if your slow parts aren’t rewritten. Most scientists slop code around in Jupyter notebooks which bitrot and are forgotten about in 6 months. The world is clearly in need of good numerical code and environments, even with Python’s amazing ecosystem. Julia has been popular among my savvier programming colleagues and it has attracted very high quality contributions, rivaling and exceeding Python in some cases. Back to me: what do I do? I write my numerical applications in Common Lisp. It’s probably as unpopular of a choice as it gets in the scientific world. I use Lisp because: - The language is standardized and my code won’t break when the compiler updates - I can write very high performance floating point code, using SIMD/AVX, and readily inspect the assembly code at the function level - I can easily bind to C and Fortran libraries - I can deploy applications as static executables - I can avoid garbage collection and heap allocations with careful programming - I can write generic, extensible interfaces - I can debug algorithms live—in the depths of stack traces—without the compile-printf-edit cycle, which is insanely useful when encountering tricky floating point accuracy bugs. Clearly I’m a “programmer’s programmer” though, which may be a hint as to why Lisp remains unsuitable for scientist use. Most scientists don’t know or don’t care what a stack allocation is, and that’s fine, because the tools have evolved around them in such a way they can be hugely successful without that knowledge. It’s also remiss to not mention that scientists typically have 8+ years of training in their (non-software) discipline, and “ain’t nobody got time for [programming],” despite its growing importance and necessity. The biggest negative of using Lisp is that a lot of work you’d take for granted in Python or Julia is not done. It took me several hours to get a certain complex 2^n dimensional matrix factorization routine working from LAPACK in Lisp, whereas in Python it would take me 30 seconds flat to look it up, read the documentation, import it, and call it. So it’s definitely the case that Lisp’s “startup overhead” is high. But at the end of the day, if I’m building an electromagnetic simulator, I feel some pre-baked math routines are the least of my concerns, and the overall code structure and quality is of the highest concern. Python is suboptimal in that category, in my opinion. One good thing about college courses like these is that they (slowly) displace numerical programming courses that are MATLAB tutorials in disguise.
saruken 6y ago
This looks like a great resource, but the vast number of grammatical mistakes is really distracting. Some issues are to be expected with a self-published book, but at least in the sections I read, a majority of sentences are grammatically incorrect. Maybe I'm just naive, but that surprises me, especially with this being affiliated with an institution like Berkeley.
tralarpa 6y ago
Some of the chapters look a bit superficial. I have seen other books (with the same target audience) with deeper discussions on the impact of algorithmic/implementation choices on the accuracy of the results etc. Here, they sometimes just refer to a python function with the link to its documentation.
- Scipio_Afri 6y ago
  Which books?
sireat 6y ago
This looks like a nice introductory text (for non-programmers). Is there a Github link to notebooks? I've been teaching Python to adult non-programmers for a few years now. One issue is that of order on what to teach. What do you teach after you teach variables and basic data types? There is no one perfect solution. This particular book teaches dictionaries, lists, sets (and NumPy arrays!) before branching and iteration. Good news - the examples in this book do not use L or list as variable names, which even some seemingly respectable books do. Personally I feel that branching and iteration (over strings) should come first before progressing to more complex data types. If you learn of lists before you know of iteration you can't really do much with lists.
musingsole 6y ago
Looks like a solid textbook. Skimming the ToC, I thought it was mostly basic stuff...and then I kept going. It goes deeper into many topics than I would've guessed and that's before getting to the numerical analysis stuff. The code examples look solid for referring back to. tl;dr: I bought a copy
gabehcoud 6y ago
just use matlab smh
pantsforbirds 6y ago
Was a little disappointed to see the root finding section didn't include Brent's Method: https://en.wikipedia.org/wiki/Brent%27s_method https://en.wikipedia.org/wiki/Brent%27s_method I think it's one of the more interesting algorithms in general.
fcatus 6y ago
fyi there is a small bug: clicking on "source code doesn't open github, it just redirect you to the same page.
maidenstock 6y ago
Interesting! Thanks of the book.

Thank you for posting this.

I would like to see a polished Python package for data, math, and science, similar to Mathematica. Or perhaps Wolfram can start supporting Python? I’d actually use that over Pandas/Conda.

leephillips 6y ago
Do you know about Sage? Or are you thinking about something different?
gatestore 6y ago
It is now possible to use Python inside Mathematica. Take a look at: https://blog.wolfram.com/2019/05/16/announcing-the-wolfram-client-library-for-python/ https://blog.wolfram.com/2019/05/16/announcing-the-wolfram-c... https://reference.wolfram.com/language/WolframClientForPython/ https://reference.wolfram.com/language/WolframClientForPytho... It is also possible to run the Wolfram Kernel inside a Jupyter Notebook: https://github.com/WolframResearch/WolframLanguageForJupyter https://github.com/WolframResearch/WolframLanguageForJupyter

unnouinceput 6y ago

I love Python, it gives me the best jobs of my life. Usually is like this. Customer happens to stumble some easy 3 lines program that solves the problem in an easy subset and after that it will create a full program around that one with all the blings. Then goes into production after a month and 15k USD lighter and its users are using it for full scale and the solving time now is in hours. Then it will search for an expert to solve the impossible problem of speeding up Python and curses at all the Indians that are scamming him and all experts who ask for another 15k USD to actually implement the program correctly. I shit you not, last one was a data scientist who paid like $3/h some Indian dude that got the job done in 2 days, took the money and contract was closed under those terms. Then this data scientist was all over his job posting crying that, while the initial matrix was in dozens of columns/rows now the same Python program would take 3 days when throwing at it a matrix that was having millions of columns/rows in size. I mean, it's several orders of magnitude higher, I was amazed that the program actually finished the job after 3 days instead of just crashing. So I had to drill into his thick skull the idea that while he initially went to war against Lichtenstein and won, definitely cannot win against US army using the same weapon. Only after that he agreed to scrap the Python program altogether and finally go with C++, because speed does matter in the end. Like I said, I love Python, especially for anything math related.

alokrai 6y ago
I wonder why it is necessary to mention "Indians that are scamming" him as if scamming is something only Indians do? Should I be careful only when dealing with Indian consultants and presume others won't lie or cheat? edit: grammar
- unnouinceput 6y ago
  Probably because Indians are the ones that are leaders in scamming? Also, if you paid attention, I've mentioned the initial Python dev was an honest Indian, he did his job within terms of contract. The data scientist, which in this case was a Canadian, is solely at fault for not disclosing full scale to the Indian one.
  - alokrai 6y ago
    >Probably because Indians are the ones that are leaders in scamming? I could respond to it but there is not much to be said. I think you have made my point.
  - rramadass 6y ago
    You really seem to have a way with words where, while the point of your original premise (i.e. lack of requirements, technical competence) is valid you have managed to bury that under completely unrelated negative and racist stereotyping.
- mlN90 6y ago
  >as if [bad thing] is something only [group] do? This is such a nonsense statement and it comes up everytime someone describes someones nationality in correlation with something awful. Honest question, when you read the line "[...]Chinese that are good at math [...]" do you read that all Chinese people are math-wizards? Because that is not what it says. What about "[...] black people that are fast runners [...]"? all 3 examples, including the one that turned on your torch of virtue, describes a sub-set of a group, the primary attribute of said group and nothing more. If anything the implication that the guy you were replied to is somehow biased and "racist" against the absolute-plague-tier of disproportional scammers coming out of India is based on nothing but your inability to differentiate between "broadspectrum-racism" and "critism of a subset of a group"

deleted 6y ago

[deleted]

amitport 6y ago

This looks like a great resouce, thanks! Note that in practice you can teach part II (numerical methods) without 90% of part I (python programming) given that your audience has some minimal programming experience (not necessarily with python).

physicsguy 6y ago

Not sure if this is more a difference between UK/US universities as this says it's targeted at "serious undergraduates or graduate students", but we covered all of this material in the 1st and 2nd year undergraduate courses at the Universities I've studied at or taught at here.

dataflow 6y ago
It matters what you're majoring in and focusing on. There's only so much time to teach so much material in CS, for example, and there's not much desire from the faculty to teach numerical diff eq solving (and certainly none from most CS students to learn it). The ones that do go to grad school tend to need these more, but they haven't seen them in undergrad so they end up learning what they're lacking in grad school (or late undergrad). That said, do you guys really learn/teach the shooting method, Lagrange interpolation, predictor-corrector methods, QR factorization, etc. as 1st/2nd-year undergrads? For what majors? I feel like this would come at the cost of deferring other topics to later years.
- physicsguy 6y ago
  We don't generally have majors/minors in the UK, so perhaps this is part of it. You do a degree in for e.g. Maths/Physics/Mech Eng, and all of your courses are in that. Yes, we also covered other stuff like splines, finite difference method, too. I think Finite Element was third year where I last taught.
  - dataflow 6y ago
    > You do a degree in for e.g. Maths/Physics/Mech Eng, and all of your courses are in that. "Major" refers to the same things you mentioned (physics, math, EE, etc.), it's not any different in the US. > Yes, we also covered other stuff like splines, finite difference method, too. I think Finite Element was third year where I last taught. Sorry to repeat the question, but again -- in what major (or "for what degree" in your terminology) did you study this? (CS? MechE? Physics? EE?)

gostsamo 6y ago

Time to first "use julia instead" comment: 3, 2, 1...

Mauricebranagh 6y ago
If its a pure numerical problem just use Fortran and buy the NAG librarys. oooh just checked and NAG do python library's now as well so that might be teh best solution.
- gostsamo 6y ago
  I'm actually a python developer, but I might be the only one appreciating the identity investment that people on both sides put in their chosen language.
deleted 6y ago
[deleted]
eternauta3k 6y ago
You'll have to wait a bit longer, the interpreter is loading :)
cambalache 6y ago
Now that people are "writing games and web servers" in Julia (as per some commenters) and that the use case list for Rust grows exponentially I cannot wait for the most epic of collisions, Julia vs Rust, astronomers must be rubbing their hands for this one.
- gostsamo 6y ago
  It is funny, because those kinds of arguments are totally meaningless. Languages are tools and a true craftsman would master many of them and use them when they are the right tool for the job. Instead, people who know only one tool try to convince themselves and everyone else that the tool they have is the only one they need and thus have it all backward.

ngcc_hk 6y ago

Ch 15 I tried the code. Forget to do plt.close() and pythonista has warning. Btw, not sure how to get the label work.

reikonomusha 6y ago

I’ve been a sort of oddball in the numerical/scientific computing world in the past 10 or so years in that I mostly reject using Python for serious numerical programming. This seems to be against the common wisdom of my PhD scientist colleagues who grab for Python any and every chance they get. The Python community has done an amazing job making a suite of scientific software. Numerical libraries and plotting are excellent in Python. Sympy and Sage are both very ambitious, and in my opinion, quite successful projects. So what’s the problem? Longevity and stability. Make fun of it all you want, but old crusty FORTRAN code, and in the past 20 years C and C++, are the bedrock of serious scientific applications. Python usually isn’t. I’m not confident enough to say why with certainty, but maybe it has to do with the fact that Python APIs and type signatures (whether written out or not) are too handwavy and easy to change without detection. In addition to this, I rarely find that people, scientists especially, are writing applications in Python. Python is too hard to deploy as an application sane people can use, is a mess to configure, and difficult to make efficient if your slow parts aren’t rewritten. Most scientists slop code around in Jupyter notebooks which bitrot and are forgotten about in 6 months. The world is clearly in need of good numerical code and environments, even with Python’s amazing ecosystem. Julia has been popular among my savvier programming colleagues and it has attracted very high quality contributions, rivaling and exceeding Python in some cases. Back to me: what do I do? I write my numerical applications in Common Lisp. It’s probably as unpopular of a choice as it gets in the scientific world. I use Lisp because: - The language is standardized and my code won’t break when the compiler updates - I can write very high performance floating point code, using SIMD/AVX, and readily inspect the assembly code at the function level - I can easily bind to C and Fortran libraries - I can deploy applications as static executables - I can avoid garbage collection and heap allocations with careful programming - I can write generic, extensible interfaces - I can debug algorithms live—in the depths of stack traces—without the compile-printf-edit cycle, which is insanely useful when encountering tricky floating point accuracy bugs. Clearly I’m a “programmer’s programmer” though, which may be a hint as to why Lisp remains unsuitable for scientist use. Most scientists don’t know or don’t care what a stack allocation is, and that’s fine, because the tools have evolved around them in such a way they can be hugely successful without that knowledge. It’s also remiss to not mention that scientists typically have 8+ years of training in their (non-software) discipline, and “ain’t nobody got time for [programming],” despite its growing importance and necessity. The biggest negative of using Lisp is that a lot of work you’d take for granted in Python or Julia is not done. It took me several hours to get a certain complex 2^n dimensional matrix factorization routine working from LAPACK in Lisp, whereas in Python it would take me 30 seconds flat to look it up, read the documentation, import it, and call it. So it’s definitely the case that Lisp’s “startup overhead” is high. But at the end of the day, if I’m building an electromagnetic simulator, I feel some pre-baked math routines are the least of my concerns, and the overall code structure and quality is of the highest concern. Python is suboptimal in that category, in my opinion. One good thing about college courses like these is that they (slowly) displace numerical programming courses that are MATLAB tutorials in disguise.

saruken 6y ago

This looks like a great resource, but the vast number of grammatical mistakes is really distracting. Some issues are to be expected with a self-published book, but at least in the sections I read, a majority of sentences are grammatically incorrect. Maybe I'm just naive, but that surprises me, especially with this being affiliated with an institution like Berkeley.

tralarpa 6y ago

Some of the chapters look a bit superficial. I have seen other books (with the same target audience) with deeper discussions on the impact of algorithmic/implementation choices on the accuracy of the results etc. Here, they sometimes just refer to a python function with the link to its documentation.

Scipio_Afri 6y ago
Which books?

sireat 6y ago

This looks like a nice introductory text (for non-programmers). Is there a Github link to notebooks? I've been teaching Python to adult non-programmers for a few years now. One issue is that of order on what to teach. What do you teach after you teach variables and basic data types? There is no one perfect solution. This particular book teaches dictionaries, lists, sets (and NumPy arrays!) before branching and iteration. Good news - the examples in this book do not use L or list as variable names, which even some seemingly respectable books do. Personally I feel that branching and iteration (over strings) should come first before progressing to more complex data types. If you learn of lists before you know of iteration you can't really do much with lists.

musingsole 6y ago

Looks like a solid textbook. Skimming the ToC, I thought it was mostly basic stuff...and then I kept going. It goes deeper into many topics than I would've guessed and that's before getting to the numerical analysis stuff. The code examples look solid for referring back to. tl;dr: I bought a copy

gabehcoud 6y ago

just use matlab smh

pantsforbirds 6y ago

Was a little disappointed to see the root finding section didn't include Brent's Method: https://en.wikipedia.org/wiki/Brent%27s_method https://en.wikipedia.org/wiki/Brent%27s_method I think it's one of the more interesting algorithms in general.

fcatus 6y ago

fyi there is a small bug: clicking on "source code doesn't open github, it just redirect you to the same page.

maidenstock 6y ago

Interesting! Thanks of the book.