7 ms·

Exploring Mathematics with Matplotlib and Python

156 points by acangiano 7y ago

enriquto 7y ago
As much as I love the Python language, it is shameful how it has become a sort of "schtrumpf"-like addition to any computer-related stuff. Such introductory tutorials are great (but this one, specifically, would benefit much by having the code that generates each figure next to it). However, it is really not necessary to specify that your particular thing is "with Python" as if it really meant anything fundamental. What kind of shitty reasoning leads to this? "Oh, let's introduce this elementary mathematics to the illiterate masses by writing it as a Python script. Now everyone will understand it!" This is a lack of respect for the agency of the readers.
- jorge-fundido 7y ago
  Perhaps it's just a factual element. Or perhaps the intention is `s/python/pragmatic/`. Yeah, python sucks in many ways but is it really so horrible? I know! Let's ask the reader to get intimate with a static compiler. If they're "smart" enough to satisfy (or trick) the compiler, then they've earned the "reward" of being able to execute their program. There is value is static typing, but there are many instances where that cost is not worth the reward.
  - jacobolus 7y ago
    I think the grandparent poster was just pointing out that this post has very little to do with python or matplotlib, with the code to generate the plots just thrown in as an afterthought at the end. Which makes it weird to have them in the title. (Whether python is a good tool or not is unrelated to this observation.) I would expect a post entitled “exploring mathematics with python” to have a whole lot more python code (inline with the text and better explained instead of an uncommented blob at the end) and a whole lot more mathematics. A more accurately descriptive title for this post might be “counting the repetitions among randomly chosen positive integers”.... which of course isn’t going to get as many clicks or as many reflexive upvotes from non-readers as a post promising “exploring mathematics with python” because it doesn’t sound (and frankly isn’t) all that interesting to most readers. (It might make a decent short project for middle school students though.) Personally I flagged the post for its misleading title.
    - lonelappde 7y ago
      Every in the title is a major part of the article. The author used Python's matplotlib to investigate mathematics, and share both the math and the Python code. It's not the author's fault if you bring your own baggage to the words.
    - inimino 7y ago
      What I got out of the post was that exploratory and experimental mathematics is fun and worthwhile and if you haven't tried it, you should, and by the way, this experiment uses Python and matplotlib (which some readers may already know). I think you missed the point of the article.
  - reificator 7y ago
    > If they're "smart" enough to satisfy (or trick) the compiler, then they've earned the "reward" of being able to execute their program. This is needlessly dismissive and frankly offensive. I prefer having errors from a compiler(or static analysis, or...) because it helps me. Not because I am a better programmer but because it helps me be a better programmer. > Compiler: Hey that type doesn't work there Oh! Thank you! I meant to use this type instead. > Compiler: This value is freed here but used right afterward here I meant to clone it. Whoops. That would have been embarrassing to debug in production! And so on. I absolutely recognize that it's a barrier to entry, but it's not one erected to keep people out, it's there to catch your mistakes for you so that you spend less time debugging and more time writing your actual application.
    - inimino 7y ago
      It's also needlessly dismissive to write as if everyone using Python is making a mistake.
      - reificator 7y ago
        > It's also needlessly dismissive to write as if everyone using Python is making a mistake. That's true. If I start doing that, please call me out on it. From the HN Guidelines: > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.[0] I'm very much not a fan of meaningful whitespace, but I use Python occasionally, and regularly help my friends who are learning it grasp this or that topic. Except where pointing them to a specific library or other tool, I've never said "You shouldn't do that in Python, do it in this other language instead". [0]: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
        inimino 7y ago
        I apologize if I misinterpreted what you wrote. It seems you just meant to criticize the title, or...? I'm actually not sure what your above comment was getting at. Sorry if I missed your point.
- ShineThu 7y ago
  ShineThu
- svat 7y ago
  While your comment (“let's introduce this elementary mathematics to the illiterate masses by writing it as a Python script. Now everyone will understand it”) may apply to some articles, what relevance does it have on this one? This article is fundamentally about how, starting even with a set of random numbers, simply visualizing it can lead to further explorations that can be carried out again with computers — sure you could replace “Matplotlib and Python” with “a graphing library and a programming language”, but fundamentally the article is not trying to teach any elementary mathematics and in fact it does not even reach the point of proving any theorems (for that matter despite a fair bit of trying I haven't actually figured out yet how to prove the conjecture the article ends with); it's just about the process of exploration (generating hypotheses, verifying them, etc) using computers.
svat 7y ago
This is a nice article. For those who have not yet read it (it's short, read it!), a one-paragraph summary: the author starts with a list of random numbers. Visualizing it (plotting the numbers, with the list index on the x axis) suggests / leads to (for the author) curiosity about how often numbers repeat. Plotting that leads to the question of what the maximum frequency would be, as a size of the input list. This can lead to a hypothesis, which one can explore with larger runs. And then after some musings about this process, the post suddenly ends (leaving the rest to the reader), and gives the code that was used for plotting. This article is essentially an encouragement and a reminder of our ability to do experimental mathematics (https://en.wikipedia.org/w/index.php?title=Experimental_mathematics&oldid=899918382 https://en.wikipedia.org/w/index.php?title=Experimental_math...): there's even a journal for it, and the Wikipedia article on it is worth reading (https://en.wikipedia.org/w/index.php?title=Experimental_Mathematics_(journal)&oldid=894684677 https://en.wikipedia.org/w/index.php?title=Experimental_Math...). See also (I guess I'm just reproducing the first page of search results here) this article (https://www.maa.org/external_archive/devlin/devlin_03_09.html https://www.maa.org/external_archive/devlin/devlin_03_09.htm...), these two in the Notices of the AMS (https://www.ams.org/notices/200505/fea-borwein.pdf https://www.ams.org/notices/200505/fea-borwein.pdf, http://www.ams.org/notices/199506/levy.pdf http://www.ams.org/notices/199506/levy.pdf), this website (https://www.experimentalmath.info https://www.experimentalmath.info), this post by Wolfram (https://blog.stephenwolfram.com/2017/03/two-hours-of-experimental-mathematics/ https://blog.stephenwolfram.com/2017/03/two-hours-of-experim...), and there's even book by V. I. Arnold (besides a couple by Borwein and Bailey, and others). Especially in number theory and probability, simple explorations with a computer can suggest deep conjectures that are yet to be proved.
- svat 7y ago
  By the way: the post ends with a conjecture that the maximum frequency is likely to be log n + 1 (with log here denoting the logarithm to base 10), but the more precise result seems to be that the largest frequency is: ((ln n)/(ln ln n))(1 + o(1)) with high probability (i.e., probability 1 - o(1)), where ln denotes the natural logarithm (log to base e). Empirically, the maximum frequency for n=1000, 10000, 100000 often seems to be, respectively, 5, 6 or 7, and 7 or 8. This problem has applications in studying hash tables etc., and can be found under terms like "maximum load" with balls in bins, and proving this doesn't seem to be very easy. As the post says “the solution is likely not as trivial as it first looks”. The analysis may be hard, but these days if faced with a problem like this in the real world (e.g. we have a hash table of size M that will receive N entries in it, and we're curious about the likely maximum load), we can likely just experiment to find out. Even when the numbers are too large to run simulations directly, an in-between solution is to get a tractable expression (a recurrence relation using dynamic programming or whatever) for the closed form, and write a program to compute it.
juggernaut390 7y ago
I’m embarrassed to see such a post upvoted. Also, matplotlib is outdated. If you want a good visualization tool, it should leverage as many features as it can to present the most information possible. This includes not just color but interactive tools like hover tools. A library like bokeh makes this extremely easy for example. I’m a bit sad to see such posts whose purpose is to demonstrate how to leverage visualization tools to improve our understanding of a phenomenon by people holding on to legacy outdated tools. It sends the wrong message.
- b52_ 7y ago
  Nobody cares if matplotlib is outdated. It served its purpose here, which was to show how plotting data can lead to further understanding data. You don't need a fully featured tool to do so.
  - goerz 7y ago
    As far as "fully featured" goes, there is nothing more fully featured than matplotlib. It just tends to take a lot more code to generate a plot, compared to some higher-level libraries, so for simple visualizations, matplotlib may not be worth the tradeoff of power vs ease of use.
    - enriquto 7y ago
      Nothing really beats "plot sin(x)" in gnuplot.
- goerz 7y ago
  How is matplotlib outdated? It's just relatively low-level, so it might be quicker to use high-level tools like seaborn or altair (which use matplotlib as their backend), if that fits the situation. For heavily customized publication-ready plots (that means print), there is no alternative to matplotlib. The other two low-level Python visualization libs, plotly and bokeh (see https://pyviz.org/tools.html https://pyviz.org/tools.html) focus on interactive plots, which is an entirely different use case! Bokeh just does not generate visualizations for print. Thus, it does NOT make matplotlib obsolete. On top of that, last time I checked, bokeh was far less flexible than matplotlib.
r34 7y ago
1. How many people do there need to be in a room so the there is a greater than 50% chance of at least two of them sharing the same birthday? 2. How many numbers we have to draw from 365 so the there is a greater than 50% chance of at least two of them are the same? 3. How many numbers we have to draw from X so the there is a greater than Y% chance of at least Z of them are the same? I think X,Y,Z are enough parameters: Drawing from X=1000 numbers, what is the chance Y that Z=(5,6,7,8...) is the same? Sorry, I'm not a matematician, just some breakfast ideas ;) Edit: Inspired by http://datagenetics.com/blog/february72019/index.html http://datagenetics.com/blog/february72019/index.html

156 points by acangiano 7y ago

As much as I love the Python language, it is shameful how it has become a sort of "schtrumpf"-like addition to any computer-related stuff. Such introductory tutorials are great (but this one, specifically, would benefit much by having the code that generates each figure next to it). However, it is really not necessary to specify that your particular thing is "with Python" as if it really meant anything fundamental. What kind of shitty reasoning leads to this? "Oh, let's introduce this elementary mathematics to the illiterate masses by writing it as a Python script. Now everyone will understand it!" This is a lack of respect for the agency of the readers.

jorge-fundido 7y ago
Perhaps it's just a factual element. Or perhaps the intention is `s/python/pragmatic/`. Yeah, python sucks in many ways but is it really so horrible? I know! Let's ask the reader to get intimate with a static compiler. If they're "smart" enough to satisfy (or trick) the compiler, then they've earned the "reward" of being able to execute their program. There is value is static typing, but there are many instances where that cost is not worth the reward.
- jacobolus 7y ago
  I think the grandparent poster was just pointing out that this post has very little to do with python or matplotlib, with the code to generate the plots just thrown in as an afterthought at the end. Which makes it weird to have them in the title. (Whether python is a good tool or not is unrelated to this observation.) I would expect a post entitled “exploring mathematics with python” to have a whole lot more python code (inline with the text and better explained instead of an uncommented blob at the end) and a whole lot more mathematics. A more accurately descriptive title for this post might be “counting the repetitions among randomly chosen positive integers”.... which of course isn’t going to get as many clicks or as many reflexive upvotes from non-readers as a post promising “exploring mathematics with python” because it doesn’t sound (and frankly isn’t) all that interesting to most readers. (It might make a decent short project for middle school students though.) Personally I flagged the post for its misleading title.
  - lonelappde 7y ago
    Every in the title is a major part of the article. The author used Python's matplotlib to investigate mathematics, and share both the math and the Python code. It's not the author's fault if you bring your own baggage to the words.
  - inimino 7y ago
    What I got out of the post was that exploratory and experimental mathematics is fun and worthwhile and if you haven't tried it, you should, and by the way, this experiment uses Python and matplotlib (which some readers may already know). I think you missed the point of the article.
- reificator 7y ago
  > If they're "smart" enough to satisfy (or trick) the compiler, then they've earned the "reward" of being able to execute their program. This is needlessly dismissive and frankly offensive. I prefer having errors from a compiler(or static analysis, or...) because it helps me. Not because I am a better programmer but because it helps me be a better programmer. > Compiler: Hey that type doesn't work there Oh! Thank you! I meant to use this type instead. > Compiler: This value is freed here but used right afterward here I meant to clone it. Whoops. That would have been embarrassing to debug in production! And so on. I absolutely recognize that it's a barrier to entry, but it's not one erected to keep people out, it's there to catch your mistakes for you so that you spend less time debugging and more time writing your actual application.
  - inimino 7y ago
    It's also needlessly dismissive to write as if everyone using Python is making a mistake.
    - reificator 7y ago
      > It's also needlessly dismissive to write as if everyone using Python is making a mistake. That's true. If I start doing that, please call me out on it. From the HN Guidelines: > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.[0] I'm very much not a fan of meaningful whitespace, but I use Python occasionally, and regularly help my friends who are learning it grasp this or that topic. Except where pointing them to a specific library or other tool, I've never said "You shouldn't do that in Python, do it in this other language instead". [0]: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      - inimino 7y ago
        I apologize if I misinterpreted what you wrote. It seems you just meant to criticize the title, or...? I'm actually not sure what your above comment was getting at. Sorry if I missed your point.
ShineThu 7y ago
ShineThu
svat 7y ago
While your comment (“let's introduce this elementary mathematics to the illiterate masses by writing it as a Python script. Now everyone will understand it”) may apply to some articles, what relevance does it have on this one? This article is fundamentally about how, starting even with a set of random numbers, simply visualizing it can lead to further explorations that can be carried out again with computers — sure you could replace “Matplotlib and Python” with “a graphing library and a programming language”, but fundamentally the article is not trying to teach any elementary mathematics and in fact it does not even reach the point of proving any theorems (for that matter despite a fair bit of trying I haven't actually figured out yet how to prove the conjecture the article ends with); it's just about the process of exploration (generating hypotheses, verifying them, etc) using computers.

svat 7y ago

This is a nice article. For those who have not yet read it (it's short, read it!), a one-paragraph summary: the author starts with a list of random numbers. Visualizing it (plotting the numbers, with the list index on the x axis) suggests / leads to (for the author) curiosity about how often numbers repeat. Plotting that leads to the question of what the maximum frequency would be, as a size of the input list. This can lead to a hypothesis, which one can explore with larger runs. And then after some musings about this process, the post suddenly ends (leaving the rest to the reader), and gives the code that was used for plotting. This article is essentially an encouragement and a reminder of our ability to do experimental mathematics (https://en.wikipedia.org/w/index.php?title=Experimental_mathematics&oldid=899918382 https://en.wikipedia.org/w/index.php?title=Experimental_math...): there's even a journal for it, and the Wikipedia article on it is worth reading (https://en.wikipedia.org/w/index.php?title=Experimental_Mathematics_(journal)&oldid=894684677 https://en.wikipedia.org/w/index.php?title=Experimental_Math...). See also (I guess I'm just reproducing the first page of search results here) this article (https://www.maa.org/external_archive/devlin/devlin_03_09.html https://www.maa.org/external_archive/devlin/devlin_03_09.htm...), these two in the Notices of the AMS (https://www.ams.org/notices/200505/fea-borwein.pdf https://www.ams.org/notices/200505/fea-borwein.pdf, http://www.ams.org/notices/199506/levy.pdf http://www.ams.org/notices/199506/levy.pdf), this website (https://www.experimentalmath.info https://www.experimentalmath.info), this post by Wolfram (https://blog.stephenwolfram.com/2017/03/two-hours-of-experimental-mathematics/ https://blog.stephenwolfram.com/2017/03/two-hours-of-experim...), and there's even book by V. I. Arnold (besides a couple by Borwein and Bailey, and others). Especially in number theory and probability, simple explorations with a computer can suggest deep conjectures that are yet to be proved.

svat 7y ago
By the way: the post ends with a conjecture that the maximum frequency is likely to be log n + 1 (with log here denoting the logarithm to base 10), but the more precise result seems to be that the largest frequency is: ((ln n)/(ln ln n))(1 + o(1)) with high probability (i.e., probability 1 - o(1)), where ln denotes the natural logarithm (log to base e). Empirically, the maximum frequency for n=1000, 10000, 100000 often seems to be, respectively, 5, 6 or 7, and 7 or 8. This problem has applications in studying hash tables etc., and can be found under terms like "maximum load" with balls in bins, and proving this doesn't seem to be very easy. As the post says “the solution is likely not as trivial as it first looks”. The analysis may be hard, but these days if faced with a problem like this in the real world (e.g. we have a hash table of size M that will receive N entries in it, and we're curious about the likely maximum load), we can likely just experiment to find out. Even when the numbers are too large to run simulations directly, an in-between solution is to get a tractable expression (a recurrence relation using dynamic programming or whatever) for the closed form, and write a program to compute it.

juggernaut390 7y ago

I’m embarrassed to see such a post upvoted. Also, matplotlib is outdated. If you want a good visualization tool, it should leverage as many features as it can to present the most information possible. This includes not just color but interactive tools like hover tools. A library like bokeh makes this extremely easy for example. I’m a bit sad to see such posts whose purpose is to demonstrate how to leverage visualization tools to improve our understanding of a phenomenon by people holding on to legacy outdated tools. It sends the wrong message.

b52_ 7y ago
Nobody cares if matplotlib is outdated. It served its purpose here, which was to show how plotting data can lead to further understanding data. You don't need a fully featured tool to do so.
- goerz 7y ago
  As far as "fully featured" goes, there is nothing more fully featured than matplotlib. It just tends to take a lot more code to generate a plot, compared to some higher-level libraries, so for simple visualizations, matplotlib may not be worth the tradeoff of power vs ease of use.
  - enriquto 7y ago
    Nothing really beats "plot sin(x)" in gnuplot.
goerz 7y ago
How is matplotlib outdated? It's just relatively low-level, so it might be quicker to use high-level tools like seaborn or altair (which use matplotlib as their backend), if that fits the situation. For heavily customized publication-ready plots (that means print), there is no alternative to matplotlib. The other two low-level Python visualization libs, plotly and bokeh (see https://pyviz.org/tools.html https://pyviz.org/tools.html) focus on interactive plots, which is an entirely different use case! Bokeh just does not generate visualizations for print. Thus, it does NOT make matplotlib obsolete. On top of that, last time I checked, bokeh was far less flexible than matplotlib.

r34 7y ago

1. How many people do there need to be in a room so the there is a greater than 50% chance of at least two of them sharing the same birthday? 2. How many numbers we have to draw from 365 so the there is a greater than 50% chance of at least two of them are the same? 3. How many numbers we have to draw from X so the there is a greater than Y% chance of at least Z of them are the same? I think X,Y,Z are enough parameters: Drawing from X=1000 numbers, what is the chance Y that Z=(5,6,7,8...) is the same? Sorry, I'm not a matematician, just some breakfast ideas ;) Edit: Inspired by http://datagenetics.com/blog/february72019/index.html http://datagenetics.com/blog/february72019/index.html