8 ms·

SciPy: Interested in adopting PRIMA, but little appetite for more Fortran code

126 points by zaikunzhang 3y ago

zaikunzhang 3y ago
see also Optimization Without Derivatives: PRIMA Fortran Version and Inclusion in SciPy, https://news.ycombinator.com/item?id=35959991 https://news.ycombinator.com/item?id=35959991 SciPy enhancement: The Fortran 77 implementation of COBYLA is buggy and challenging to maintain. Switch to the PRIMA implementation? #18118, https://github.com/scipy/scipy/issues/18118#issuecomment-1552057125 https://github.com/scipy/scipy/issues/18118#issuecomment-155...
bouchard 3y ago
I understand the lack of appetite for old FORTRAN 77 code, but PRIMA is a modern Fortran implementation. You only need Python's ctypes module to load the compiled Fortran library thanks to Fortran's iso_c_binding module. Using f2c as suggested seems like a worse alternative, not a solution.
- wirrbel 3y ago
  > Using f2c as suggested seems like a worse alternative, not a solution. essentially madness. Modern fortran is quite nice, much nicer than C from a Python programmer's perspective.
  - zaikunzhang 3y ago
    > Modern fortran is quite nice, much nicer than C from a Python programmer's perspective. Totally agree. I hope everyone making comments on Fortran can spend a few minitues reading what modern Fortran is, e.g., by checking https://fortran-lang.org/en/ https://fortran-lang.org/en/
  - drbaba 3y ago
    I used Fortran 2008 a lot some years ago; initially I wanted to write a C++ wrapper around a Fortran library that seemed useful, but ended up liking the language better than C++ and just did the whole project in it. I think a lot of people don’t realize that modern Fortran actually supports object-oriented programming and Matlab-style array slicing, and comes with built-in support for e.g. matrices and complex numbers. It even has some features inspired by functional languages, like being able to mark functions as pure (side-effect free). Combined with its general speed, this makes it a pretty nice language for numerical code. Although for most new users, I think Julia might be a better option than Fortran these days.
- noobermin 3y ago
  This reads like the maintainer, mind you a maintainer of scipy, the premier foundational library in numerical python, does not even understand what PRIMA is or what modern fortran is like. I get people are busy, but this shakes my opinion of scipy a little bit. It doesn't take that long to read the readme... It does not reflect well on them.
  - deleted 3y ago
    [deleted]
  - fluidcruft 3y ago
    I've been trying to debug some of scipy's statistics and it's... incomprehensible. I am not impressed. I'll happily take fortran over its growing morass of wierdo boost wrappers.
tomr_stargazer 3y ago
I thought that the PRIMA developer’s comment in this thread was really mature and thoughtful. Reproduced below (minus some formatting): > My goal is to make Professor Powell 12’s solvers as accessible as possible to scientists, engineers, and algorithm researchers. I am not particularly in favor of or against any language. I hope that everyone can easily use Powell’s solvers in her/his favorite languages. > The first implementation of PRIMA is in modern Fortran simply because Powell’s implementation was in Fortran 77. Using Fortran, I can systematically verify the bit-to-bit faithfulness of the modernized implementation (not only “faithful up to an epsilon”). In addition, the intrinsic support for matrix-vector calculations is a strong advantage when developing reference implementations (or templates) of numerical solvers — most numerical algorithms are combinations of such calculations anyway. > The major motivation for developing the modern Fortran version is to provide a reference for the implementation in other languages, namely Python, MATLAB, C++, Julia, and R. A reference implementation must be structured, modularized, readable, understandable, and extendable. The original Fortran 77 code is a true masterpiece, but it is not proper at all for being used as a reference implementation. You do not want to use a spaghetti-style codebase with 244 GOTOs as a reference, or your implementation will be of the same style. > Putting it more straightforwardly, I implemented the modern-Fortran version of PRIMA in order to develop versions that are entirely Fortran-free 3. > Coming back to the point, I fully understand why the SciPy community has “little appetite for taking on more Fortran code”. The reputation of Fortran has been damaged over the years. I do not agree with the damaged reputation, and I feel sorry for those who do not have a chance to know (or refuse to know) modern Fortran due to this false reputation, but I do not blame them. It is the responsibility of the Fortran community to re-establish the reputation. I do not regard it as a bully to request non-Fortran implementations. It is not a question of surrendering or not. > I do consider myself a member of the Fortran community. Taking my share of the aforementioned responsibilty, I will try to promote the usage of modern Fortran via the PRIMA project. There is nothing more convincing than a successful real-life project. > For the inclusion of PRIMA in SciPy, I will keep communicating with both the SciPy and the Fortran community (e.g., those on this discourse), trying to find the best route. As pointed out by others, f2c is not an option due to its incapability of handling modern Fortran. Official C++ and Python implementations are being planned, but they will not be delivered in the near future. The most probable and practical solution, as suggested here and under the other thread, is to wrap the modern Fortran implementation of PRIMA using iso_c_bindings + ctypes or similar facilities. I hope the SciPy maintainers will accept this solution.
radarsat1 3y ago
I wonder why not just try to use more Pythonic solutions such as numba within scipy. I realize that maybe not in all cases can the same performance as Fortran be achieved but it strikes me as much more hackable for Python programmers than having to learn an new (old) language just for certain core algorithms. One of the principal reasons for switching to C implementations is because more people understand C than Fortran. This seems doubly true for Python-based jit solutions like numba or jax, so why not just skip C altogether and go for a more native port?
- adastra22 3y ago
  Numpy is Fortran btw.
  - remram 3y ago
    From numpy.org: > The core of NumPy is well-optimized C code.
    - taeric 3y ago
      Amusingly, "well-optimized C code" includes calling out to fortran libraries. :D https://numpy.org/doc/1.13/user/building.html https://numpy.org/doc/1.13/user/building.html
      - cfcf14 3y ago
        It's turtles all the way down, except for the final turtle, which is Fortran...
        bee_rider 3y ago
        The last turtle is probably assembly, or at least intrinsics, in most good BLAS implementations. Probably called from C, because that’s easier in C. The nice thing about Fortran is that you get something almost as good, but a normal engineer can write it. There are more C programmers than Fortran programmers, but there are more Fortran programmers than there are C programmers who can write really good assembly kernels. And all the C programmers in that last group are already working on vendor BLAS implementations.
adastra22 3y ago
This is crazy. Get over yourself! Whether you like it or not, Fortran is still THE industry standard for scientific codes, and the fastest compiled language on many architectures.
- raoulj 3y ago
  In the defense of the SciPy response, while there may be some programming language zealotry behind the position, it seems to be from a lack of expertise in SciPy to maintain its Fortran code that's pushing SciPy to avoid new Fortran additions. Hopefully OP gets PRIMA through! Been admiring this effort, Dr. Zhang.
  - ok123456 3y ago
    F2py, which is already a standard part of numpy, does the heaving lifting and generates idiomatic python wrappers for Fortan code. It's just zealotry of them going "eeew fortran that's like really really old". Fortran got multidimensional dense numeric arrays right. Modern Fortran allows you to do elemental/vectorized operations on them. C does not even come close, and requires tons of preprocessor macros if you try. Every other 'modern' language introduces a bunch of tooling and dependencies just to get where Fortran is, because they just don't have a focus on scientific computing.
    - michaelt 3y ago
      > It's just zealotry of them going "eeew fortran that's like really really old". The developers of SciPy are maintaining a codebase that is 18% fortran, according to [1]. The only language that makes up a larger part of the codebase is python. Don't you think it's possible their opinion on fortran is actually informed by quite a bit of experience? [1] https://github.com/scipy/scipy/ https://github.com/scipy/scipy/
      - bouchard 3y ago
        I don't believe anybody would argue against their reluctance for old FORTRAN 77 code; it's a PITA to deal with. The central issue is that their experience with the old Fortran doesn't translate to modern Fortran; they are two different beasts.
travisporter 3y ago
Who is this insane developer who links twitter to every text selection on this page!@
cozzyd 3y ago
Looking at the original issue, scipy maintainers don't want to use LGPL code (why?!?) but they use code that appears in Numerical Recipes (which has super onerous licensing)?!?
- jabl 3y ago
  First, if you read through that scipy issue (https://github.com/scipy/scipy/issues/18118 https://github.com/scipy/scipy/issues/18118 ) the author was willing and able to relicense PRIMA under a 3-clause BSD license which is perfectly acceptable for scipy. For the numerical recipes reference, there is a mention that scipy uses a slightly improved version of Powell's algorithm that is originally due to Forman Acton and presumably published in his popular book on numerical analysis, and that also happens to be described & included in numerical recipes. That is, unless the code scipy uses is copied from numerical recipes, which I presume it isn't, NR having the same algorithm doesn't mean that every other independent implementation of that algorithm falls under NR copyright.
  - cozzyd 3y ago
    thanks, I misunderstood the NR reference. (In my field... it is common for people to copy and paste terrible NR code without understanding how bad it is or the licensing terms...).
domcoltd 3y ago
Hopefully, the SciPy community can stay open-minded about modern Fortran libraries. Modern Fortran is quite different from Fortran 77, while being as powerful, if not more. In addition, there has been a significant community effort on improving and modernising the legacy packages, the ecosystem, and the language itself. With projects like LFortran (https://lfortran.org/ https://lfortran.org/), fpm (https://github.com/fortran-lang/fpm https://github.com/fortran-lang/fpm), and stdlib (https://github.com/fortran-lang/stdlib https://github.com/fortran-lang/stdlib), I believe that Fortran will enjoy prosperity again.
- zaikunzhang 3y ago
  Each of LFortran, fpm, and stdlib deserve a HN front-page story :)
- nvy 3y ago
  Any resources for learning Fortran in 202X that don't bother going over the old cruft? Last time I looked (admittedly, years ago), most resources seemed to be in the vein of "here's Fortran 1990, and once you've learned that here are a few chapters on F03 and F08, treated separately". It'd be great to learn there exists a "Fortran: the good parts" sort of thing that just completely elides the outdated stuff.
  - bouchard 3y ago
    I found "Modern Fortran: Building Efficient Parallel Applications" to be a pretty good read. See https://milancurcic.com/book/ https://milancurcic.com/book/ Also https://fortran-lang.org/en/ https://fortran-lang.org/en/ is a good hub to find more information, packages, etc.
  - deleted 3y ago
    [deleted]
  - nunuvit 3y ago
    https://fortran-lang.org/en/learn/quickstart/ https://fortran-lang.org/en/learn/quickstart/
  - 3y ago
whinvik 3y ago
Even though it's not great but I understand why this maybe the direction SciPy maintainers want to go in. They have limited capacity and I think focusing on a single language makes sense especially if there are way more developers in that language, C. One way to solve this would be of course, to have more people from the Fortran community being part of the SciPy community. But I don't know how big the Fortran community really is to be able to do that.
- nunuvit 3y ago
  There just isn't much interest. HPC focuses on distributed and product doesn't care about open source. That mostly leaves just professors and oddballs.
ChrisRackauckas 3y ago
Interesting response. I develop the Julia SciML organization https://sciml.ai/ https://sciml.ai/ and we'd be more than happy to work with you to get wrappers for PRIMA into Optimization.jl's general interface (https://docs.sciml.ai/Optimization/stable/ https://docs.sciml.ai/Optimization/stable/). Please get in touch and we can figure out how to set this all up. I personally would be curious to try this out and do some benchmarks against nlopt methods.

126 points by zaikunzhang 3y ago

see also Optimization Without Derivatives: PRIMA Fortran Version and Inclusion in SciPy, https://news.ycombinator.com/item?id=35959991 https://news.ycombinator.com/item?id=35959991 SciPy enhancement: The Fortran 77 implementation of COBYLA is buggy and challenging to maintain. Switch to the PRIMA implementation? #18118, https://github.com/scipy/scipy/issues/18118#issuecomment-1552057125 https://github.com/scipy/scipy/issues/18118#issuecomment-155...

bouchard 3y ago

I understand the lack of appetite for old FORTRAN 77 code, but PRIMA is a modern Fortran implementation. You only need Python's ctypes module to load the compiled Fortran library thanks to Fortran's iso_c_binding module. Using f2c as suggested seems like a worse alternative, not a solution.

wirrbel 3y ago
> Using f2c as suggested seems like a worse alternative, not a solution. essentially madness. Modern fortran is quite nice, much nicer than C from a Python programmer's perspective.
- zaikunzhang 3y ago
  > Modern fortran is quite nice, much nicer than C from a Python programmer's perspective. Totally agree. I hope everyone making comments on Fortran can spend a few minitues reading what modern Fortran is, e.g., by checking https://fortran-lang.org/en/ https://fortran-lang.org/en/
- drbaba 3y ago
  I used Fortran 2008 a lot some years ago; initially I wanted to write a C++ wrapper around a Fortran library that seemed useful, but ended up liking the language better than C++ and just did the whole project in it. I think a lot of people don’t realize that modern Fortran actually supports object-oriented programming and Matlab-style array slicing, and comes with built-in support for e.g. matrices and complex numbers. It even has some features inspired by functional languages, like being able to mark functions as pure (side-effect free). Combined with its general speed, this makes it a pretty nice language for numerical code. Although for most new users, I think Julia might be a better option than Fortran these days.
noobermin 3y ago
This reads like the maintainer, mind you a maintainer of scipy, the premier foundational library in numerical python, does not even understand what PRIMA is or what modern fortran is like. I get people are busy, but this shakes my opinion of scipy a little bit. It doesn't take that long to read the readme... It does not reflect well on them.
- deleted 3y ago
  [deleted]
- fluidcruft 3y ago
  I've been trying to debug some of scipy's statistics and it's... incomprehensible. I am not impressed. I'll happily take fortran over its growing morass of wierdo boost wrappers.

tomr_stargazer 3y ago

I thought that the PRIMA developer’s comment in this thread was really mature and thoughtful. Reproduced below (minus some formatting): > My goal is to make Professor Powell 12’s solvers as accessible as possible to scientists, engineers, and algorithm researchers. I am not particularly in favor of or against any language. I hope that everyone can easily use Powell’s solvers in her/his favorite languages. > The first implementation of PRIMA is in modern Fortran simply because Powell’s implementation was in Fortran 77. Using Fortran, I can systematically verify the bit-to-bit faithfulness of the modernized implementation (not only “faithful up to an epsilon”). In addition, the intrinsic support for matrix-vector calculations is a strong advantage when developing reference implementations (or templates) of numerical solvers — most numerical algorithms are combinations of such calculations anyway. > The major motivation for developing the modern Fortran version is to provide a reference for the implementation in other languages, namely Python, MATLAB, C++, Julia, and R. A reference implementation must be structured, modularized, readable, understandable, and extendable. The original Fortran 77 code is a true masterpiece, but it is not proper at all for being used as a reference implementation. You do not want to use a spaghetti-style codebase with 244 GOTOs as a reference, or your implementation will be of the same style. > Putting it more straightforwardly, I implemented the modern-Fortran version of PRIMA in order to develop versions that are entirely Fortran-free 3. > Coming back to the point, I fully understand why the SciPy community has “little appetite for taking on more Fortran code”. The reputation of Fortran has been damaged over the years. I do not agree with the damaged reputation, and I feel sorry for those who do not have a chance to know (or refuse to know) modern Fortran due to this false reputation, but I do not blame them. It is the responsibility of the Fortran community to re-establish the reputation. I do not regard it as a bully to request non-Fortran implementations. It is not a question of surrendering or not. > I do consider myself a member of the Fortran community. Taking my share of the aforementioned responsibilty, I will try to promote the usage of modern Fortran via the PRIMA project. There is nothing more convincing than a successful real-life project. > For the inclusion of PRIMA in SciPy, I will keep communicating with both the SciPy and the Fortran community (e.g., those on this discourse), trying to find the best route. As pointed out by others, f2c is not an option due to its incapability of handling modern Fortran. Official C++ and Python implementations are being planned, but they will not be delivered in the near future. The most probable and practical solution, as suggested here and under the other thread, is to wrap the modern Fortran implementation of PRIMA using iso_c_bindings + ctypes or similar facilities. I hope the SciPy maintainers will accept this solution.

radarsat1 3y ago

I wonder why not just try to use more Pythonic solutions such as numba within scipy. I realize that maybe not in all cases can the same performance as Fortran be achieved but it strikes me as much more hackable for Python programmers than having to learn an new (old) language just for certain core algorithms. One of the principal reasons for switching to C implementations is because more people understand C than Fortran. This seems doubly true for Python-based jit solutions like numba or jax, so why not just skip C altogether and go for a more native port?

adastra22 3y ago
Numpy is Fortran btw.
- remram 3y ago
  From numpy.org: > The core of NumPy is well-optimized C code.
  - taeric 3y ago
    Amusingly, "well-optimized C code" includes calling out to fortran libraries. :D https://numpy.org/doc/1.13/user/building.html https://numpy.org/doc/1.13/user/building.html
    - cfcf14 3y ago
      It's turtles all the way down, except for the final turtle, which is Fortran...
      - bee_rider 3y ago
        The last turtle is probably assembly, or at least intrinsics, in most good BLAS implementations. Probably called from C, because that’s easier in C. The nice thing about Fortran is that you get something almost as good, but a normal engineer can write it. There are more C programmers than Fortran programmers, but there are more Fortran programmers than there are C programmers who can write really good assembly kernels. And all the C programmers in that last group are already working on vendor BLAS implementations.

adastra22 3y ago

This is crazy. Get over yourself! Whether you like it or not, Fortran is still THE industry standard for scientific codes, and the fastest compiled language on many architectures.

raoulj 3y ago
In the defense of the SciPy response, while there may be some programming language zealotry behind the position, it seems to be from a lack of expertise in SciPy to maintain its Fortran code that's pushing SciPy to avoid new Fortran additions. Hopefully OP gets PRIMA through! Been admiring this effort, Dr. Zhang.
- ok123456 3y ago
  F2py, which is already a standard part of numpy, does the heaving lifting and generates idiomatic python wrappers for Fortan code. It's just zealotry of them going "eeew fortran that's like really really old". Fortran got multidimensional dense numeric arrays right. Modern Fortran allows you to do elemental/vectorized operations on them. C does not even come close, and requires tons of preprocessor macros if you try. Every other 'modern' language introduces a bunch of tooling and dependencies just to get where Fortran is, because they just don't have a focus on scientific computing.
  - michaelt 3y ago
    > It's just zealotry of them going "eeew fortran that's like really really old". The developers of SciPy are maintaining a codebase that is 18% fortran, according to [1]. The only language that makes up a larger part of the codebase is python. Don't you think it's possible their opinion on fortran is actually informed by quite a bit of experience? [1] https://github.com/scipy/scipy/ https://github.com/scipy/scipy/
    - bouchard 3y ago
      I don't believe anybody would argue against their reluctance for old FORTRAN 77 code; it's a PITA to deal with. The central issue is that their experience with the old Fortran doesn't translate to modern Fortran; they are two different beasts.

travisporter 3y ago

Who is this insane developer who links twitter to every text selection on this page!@

cozzyd 3y ago

Looking at the original issue, scipy maintainers don't want to use LGPL code (why?!?) but they use code that appears in Numerical Recipes (which has super onerous licensing)?!?

jabl 3y ago
First, if you read through that scipy issue (https://github.com/scipy/scipy/issues/18118 https://github.com/scipy/scipy/issues/18118 ) the author was willing and able to relicense PRIMA under a 3-clause BSD license which is perfectly acceptable for scipy. For the numerical recipes reference, there is a mention that scipy uses a slightly improved version of Powell's algorithm that is originally due to Forman Acton and presumably published in his popular book on numerical analysis, and that also happens to be described & included in numerical recipes. That is, unless the code scipy uses is copied from numerical recipes, which I presume it isn't, NR having the same algorithm doesn't mean that every other independent implementation of that algorithm falls under NR copyright.
- cozzyd 3y ago
  thanks, I misunderstood the NR reference. (In my field... it is common for people to copy and paste terrible NR code without understanding how bad it is or the licensing terms...).

domcoltd 3y ago

Hopefully, the SciPy community can stay open-minded about modern Fortran libraries. Modern Fortran is quite different from Fortran 77, while being as powerful, if not more. In addition, there has been a significant community effort on improving and modernising the legacy packages, the ecosystem, and the language itself. With projects like LFortran (https://lfortran.org/ https://lfortran.org/), fpm (https://github.com/fortran-lang/fpm https://github.com/fortran-lang/fpm), and stdlib (https://github.com/fortran-lang/stdlib https://github.com/fortran-lang/stdlib), I believe that Fortran will enjoy prosperity again.

zaikunzhang 3y ago
Each of LFortran, fpm, and stdlib deserve a HN front-page story :)
nvy 3y ago
Any resources for learning Fortran in 202X that don't bother going over the old cruft? Last time I looked (admittedly, years ago), most resources seemed to be in the vein of "here's Fortran 1990, and once you've learned that here are a few chapters on F03 and F08, treated separately". It'd be great to learn there exists a "Fortran: the good parts" sort of thing that just completely elides the outdated stuff.
- bouchard 3y ago
  I found "Modern Fortran: Building Efficient Parallel Applications" to be a pretty good read. See https://milancurcic.com/book/ https://milancurcic.com/book/ Also https://fortran-lang.org/en/ https://fortran-lang.org/en/ is a good hub to find more information, packages, etc.
- deleted 3y ago
  [deleted]
- nunuvit 3y ago
  https://fortran-lang.org/en/learn/quickstart/ https://fortran-lang.org/en/learn/quickstart/
- 3y ago

whinvik 3y ago

Even though it's not great but I understand why this maybe the direction SciPy maintainers want to go in. They have limited capacity and I think focusing on a single language makes sense especially if there are way more developers in that language, C. One way to solve this would be of course, to have more people from the Fortran community being part of the SciPy community. But I don't know how big the Fortran community really is to be able to do that.

nunuvit 3y ago
There just isn't much interest. HPC focuses on distributed and product doesn't care about open source. That mostly leaves just professors and oddballs.

ChrisRackauckas 3y ago

Interesting response. I develop the Julia SciML organization https://sciml.ai/ https://sciml.ai/ and we'd be more than happy to work with you to get wrappers for PRIMA into Optimization.jl's general interface (https://docs.sciml.ai/Optimization/stable/ https://docs.sciml.ai/Optimization/stable/). Please get in touch and we can figure out how to set this all up. I personally would be curious to try this out and do some benchmarks against nlopt methods.