8 ms·

Show HN: Veros, a high-performance ocean simulator in pure Python

41 points by dionhaefner 9y ago

dionhaefner 9y ago
Veros is a port of a traditional, MPI-parallelized, Fortran ocean model to Python / NumPy. All parallelism is handled implicitly by Bohrium (https://github.com/bh107/bohrium https://github.com/bh107/bohrium), so all core routines just consist of basic NumPy-like array operations. The best part: Turns out we actually perform half as good as the Fortran reference on 24 CPUs, and as good on a GPU.
imauld 9y ago
> Currently, the only officially supported Python version for Veros is Python 2.7 (64-bit), which should be the standard Python interpreter on most systems. Couple questions: - Who cares if it's the standard Python on most systems? You shouldn't be installing things in to the system Python anyway. - Why start a project like this in Python 2? It's going to be EOL in a couple of years. New projects should absolutely be started in Python 3. I can't see a reason for starting anything new, intended for distribution, in Py2.
- profquail 9y ago
  I agree that new projects should, in general, be targeting Python 3.x by default. In this specific case though, it looks like the Bohrium library being used by Veros for parallelization has some issues with Python 3 so that may be what’s keeping this project on Python 2 (at least for now).
  - dionhaefner 9y ago
    This. I don't like recommending Python 2 either, but that's the way it is for now. Our tests pass on Python 3, too[1], but Bohrium isn't reliable enough on Python 3 to recommend it to the average user. I should probably change to wording on that in the docs. [1] https://travis-ci.org/dionhaefner/veros https://travis-ci.org/dionhaefner/veros
dionhaefner 9y ago
If you're curious how NumPy and Bohrium stack up against pure Fortran, I'd recommend you have a look at our benchmarks: http://veros.readthedocs.io/more/benchmarks.html http://veros.readthedocs.io/more/benchmarks.html
hobofan 9y ago
What does someone use a ocean simulator for? It seems to be for meteorology, but what specific problems would a meteorologist use this for?
- dionhaefner 9y ago
  > It seems to be for meteorology Not quite - meteorolgists study the atmosphere, while the ocean is studied by oceanographers. > What does someone use a ocean simulator for? Ocean models are used to further our understanding of the dynamical system that is, well, the ocean. Ocean dynamics are governed by a set of partial differential equations (usually a simplified form of the more general Navier-Stokes Equations), which are coupled and nonlinear in the general case. Analytical results are only available for highly simplified setups, and many of the large-scale phenomena in the ocean are inherently unmeasurable (even more so since we are not able to observe the ocean in isolation, i.e., without atmospherical fluctuations). So we need numerical models to be able to setup more sophisticated experiments. A classical study is e.g. Kawase, 1987 [1], who showed in a simple model how the global large-scale ocean circulation is setup and maintained by only a couple dynamical effects. So there are two kinds of ocean models: One is used to study the ocean in isolation, either using idealized or realistic models (which are both supported by Veros). The other kind are hyper-realistic coupled models, which are one crucial component of climate or earth system models like CESM[2]. Those are used to make projections of the world climate (and they're a major data source for the IPCC reports). Those are usually huge setups that run on the world's largest supercomputers, so only extremely optimized Fortran ocean models like POP[3] are used for that purpose. Overall, earth system modelling is still a Fortran bastion, and Veros is the only higher-level programming language simulator I know of. [1] https://journals.ametsoc.org/doi/abs/10.1175/1520-0485(1987)017%3C2294%3AEODOCD%3E2.0.CO%3B2 https://journals.ametsoc.org/doi/abs/10.1175/1520-0485(1987)... [2] http://www.cesm.ucar.edu/ http://www.cesm.ucar.edu/ [3] http://www.cesm.ucar.edu/models/cesm1.0/pop2/ http://www.cesm.ucar.edu/models/cesm1.0/pop2/
  - hobofan 9y ago
    Very interesting! Thanks for the short introduction! Might I suggest adding an introduction along those lines to your README or docs? I think right now, when someone stumbles upon the project, unless they already have a very good understanding of the topic it is really hard to get a grip of it. From my experience, adding some more basic introductions to a scientific field (or links to such resources) to the documentation of a domain-specific software can go a long way in getting people interested in the field.

41 points by dionhaefner 9y ago

Veros is a port of a traditional, MPI-parallelized, Fortran ocean model to Python / NumPy. All parallelism is handled implicitly by Bohrium (https://github.com/bh107/bohrium https://github.com/bh107/bohrium), so all core routines just consist of basic NumPy-like array operations. The best part: Turns out we actually perform half as good as the Fortran reference on 24 CPUs, and as good on a GPU.

imauld 9y ago

> Currently, the only officially supported Python version for Veros is Python 2.7 (64-bit), which should be the standard Python interpreter on most systems. Couple questions: - Who cares if it's the standard Python on most systems? You shouldn't be installing things in to the system Python anyway. - Why start a project like this in Python 2? It's going to be EOL in a couple of years. New projects should absolutely be started in Python 3. I can't see a reason for starting anything new, intended for distribution, in Py2.

profquail 9y ago
I agree that new projects should, in general, be targeting Python 3.x by default. In this specific case though, it looks like the Bohrium library being used by Veros for parallelization has some issues with Python 3 so that may be what’s keeping this project on Python 2 (at least for now).
- dionhaefner 9y ago
  This. I don't like recommending Python 2 either, but that's the way it is for now. Our tests pass on Python 3, too[1], but Bohrium isn't reliable enough on Python 3 to recommend it to the average user. I should probably change to wording on that in the docs. [1] https://travis-ci.org/dionhaefner/veros https://travis-ci.org/dionhaefner/veros

dionhaefner 9y ago

If you're curious how NumPy and Bohrium stack up against pure Fortran, I'd recommend you have a look at our benchmarks: http://veros.readthedocs.io/more/benchmarks.html http://veros.readthedocs.io/more/benchmarks.html

hobofan 9y ago

What does someone use a ocean simulator for? It seems to be for meteorology, but what specific problems would a meteorologist use this for?

dionhaefner 9y ago
> It seems to be for meteorology Not quite - meteorolgists study the atmosphere, while the ocean is studied by oceanographers. > What does someone use a ocean simulator for? Ocean models are used to further our understanding of the dynamical system that is, well, the ocean. Ocean dynamics are governed by a set of partial differential equations (usually a simplified form of the more general Navier-Stokes Equations), which are coupled and nonlinear in the general case. Analytical results are only available for highly simplified setups, and many of the large-scale phenomena in the ocean are inherently unmeasurable (even more so since we are not able to observe the ocean in isolation, i.e., without atmospherical fluctuations). So we need numerical models to be able to setup more sophisticated experiments. A classical study is e.g. Kawase, 1987 [1], who showed in a simple model how the global large-scale ocean circulation is setup and maintained by only a couple dynamical effects. So there are two kinds of ocean models: One is used to study the ocean in isolation, either using idealized or realistic models (which are both supported by Veros). The other kind are hyper-realistic coupled models, which are one crucial component of climate or earth system models like CESM[2]. Those are used to make projections of the world climate (and they're a major data source for the IPCC reports). Those are usually huge setups that run on the world's largest supercomputers, so only extremely optimized Fortran ocean models like POP[3] are used for that purpose. Overall, earth system modelling is still a Fortran bastion, and Veros is the only higher-level programming language simulator I know of. [1] https://journals.ametsoc.org/doi/abs/10.1175/1520-0485(1987)017%3C2294%3AEODOCD%3E2.0.CO%3B2 https://journals.ametsoc.org/doi/abs/10.1175/1520-0485(1987)... [2] http://www.cesm.ucar.edu/ http://www.cesm.ucar.edu/ [3] http://www.cesm.ucar.edu/models/cesm1.0/pop2/ http://www.cesm.ucar.edu/models/cesm1.0/pop2/
- hobofan 9y ago
  Very interesting! Thanks for the short introduction! Might I suggest adding an introduction along those lines to your README or docs? I think right now, when someone stumbles upon the project, unless they already have a very good understanding of the topic it is really hard to get a grip of it. From my experience, adding some more basic introductions to a scientific field (or links to such resources) to the documentation of a domain-specific software can go a long way in getting people interested in the field.