6 ms·

Another take: There's value in using a compiled language here. I'm an amateur dabbling in computational chemistry, and my weapon of choice is Rust. (After start

Another take: There's value in using a compiled language here. I'm an amateur dabbling in computational chemistry, and my weapon of choice is Rust. (After starting using Python). Why: - Fast, in an area where speed is important - Can be made faster for repetitive tasks by communicating with a GPU using Vulkan, CUDA etc - Can incorporate a GUI, and 3D graphics, which helps for exploring things visually, tweaking parameters, doing time simulations etc. - Can share. Ie, I showed my brother yesterday by sharing a 10Mb executable. Matlab has license complications and you need it installed. Sharing Python programs is a disaster. (Docker? Pip? Pyenv? Poetry? Virtualenv? System Python? System dependencies for the C and Fortran dependencies?)

wodenokoto 4y ago
Maybe it is not part of computational chemistry, but how do you do explorative analysis? More than half my time working with data is spend grouping and summarising data, typically in Pandas and drawing interactive graphs in plotly.
- the__alchemist 4y ago
  Plots and visualizations in 3D. (Vulkan / WGPU) For example, to compare wave function solutions, I use surface plots of a 2D slice. I have a UI that can change the slice of the non-mapped axis. Plots of psi'', the potential etc too. Or for simulating molecular motion, draw it out in 3D, with 6DOF FPS-style camera controls etc. I imagine Pandas would be OOM too slow for this. Compared to Numpy, Pandas is, in my experience far too slow. Another speed limitation is Matplotlib's 3D plots make my computer crawl when rotating them etc; and they're not very interactive, at least on their own. When dealing with these systems, a particular computational challenge is that they're 3D, so computation scales with precision^3. (This also adds complications to visualization, as I alluded to above)
  - legerdemain 4y ago
    Did you write your own visualizations using wgpu? I don't think its goals include providing visualization out of the box.
    - the__alchemist 4y ago
      Yep! So, took some work upfront. Basic rendering/navigation engine. `egui` for the GUI.
akuro 4y ago
Interesting. You might be the first computational chemist I know who actually uses Rust. I know a lot of computational chemists! Python is the big one, all of the aforementioned chemists are either intermediate or advanced in that. The runner-up seems to be Julia, which I personally have no experience with. The big guys are Fortran and C++. I prefer C for tasks of this nature, but I also shill Scheme so don't listen to my opinions on programming languages. Best of luck on your computational chemistry endeavours!
- doubleg72 4y ago
  [flagged]
  - akuro 4y ago
    Don't listen to my opinions, sure. But pay heed to the large number of computational chemists who do use these languages. There is a reason that professionals employ the tools that they do.
    - freehorse 4y ago
      Historical reasons, most often than not.
    - tialaramex 4y ago
      One reason we have lots of people in numerate disciplines using Python is that we taught them Python. Python is very teachable. Given a class of average 19 year old students from a numerate discipline, my colleagues will teach most of them Python to a reasonable standard in a single module (so e.g 2 hours of teaching per week over 16 weeks). The same would not be true if we were supposed to teach them C++ for example. It's a huge, sprawling language, full of difficult concepts that are unfriendly to teach, and equally filled with foot guns that if you don't teach them will definitely maim your students, but if you do teach them take up yet more precious time on minutiae. Safe Rust wouldn't be as hard to teach as C++ but it's no picnic. So even if the Chemists decided that ideally they'd like their undergraduates to learn Rust instead of Python, I think the argument would be that it can't be done on the existing timeline.
BaculumMeumEst 4y ago
Python can be very fast. Library ecosystem and network effects by far outweigh anything you listed. Mindless Rust evangelism is tiresome.
- deleted 4y ago
  [deleted]
- patrick451 4y ago
  One thing is for sure, rust has excellent marketing.
  - pdimitar 4y ago
    Or it's actually good and you're bemoaning it due to bias. It's worth exploring the areas outside of our comfort zone.
    - patrick451 4y ago
      I've tried it, and it seems interesting. But I distrust anything that appears to have a lot of propaganda behind it, and rust definitely fits that bill. The hype will need to die down and the dust settle before I trust anything important to it.
      - pdimitar 4y ago
        I think you are being unfair here. Literally every community has the 1% of toxic, loud and insufferable people in it. Point me at your hobbies and I'll find these kinds of people there, I can bet my neck on it. Many people conflate their own irritation from hearing about the thing X often and against their will, with the quality of the thing X. IMHO critical thinking and maturity demands that we make that distinction. I have worked several times in Rust companies and their devs are extremely hardcore and very quiet. You couldn't tell these people are responsible for the technical operations and code of worldwide financial services. So again, be a bit more charitable. Every community has its rotten apples. That says nothing about the quality of the thing that the community stands for.
einpoklum 4y ago
Given your criteria, you might want to consider (modern) C++. * Fast - in many cases faster than Rust, although the difference is inconsequential relative to Python-to-Rust improvement I guess. * _Really_ utilize CUDA, OpenCL, Vulcan etc. Specifically, Rust GPU is limited in its supported features, see: https://github.com/Rust-GPU/Rust-CUDA/blob/master/guide/src/features.md https://github.com/Rust-GPU/Rust-CUDA/blob/master/guide/src/... ... * Host-side use of CUDA is at least as nice, and probably nicer, than what you'll get with Rust. That is, provided you use my own Modern C++ wrappers for the CUDA APIs: https://github.com/eyalroz/cuda-api-wrappers/ https://github.com/eyalroz/cuda-api-wrappers/ :-) ... sorry for the shameless self-plug. * ... which brings me to another point: Richer offering of libraries for various needs than Rust, for you to possibly utilize. * Easier to share than Rust. A target system is less likely to have an appropriate version of Rust and the surrounding ecosystem. There are downsides, of course, but I was just applying your criteria.
- the__alchemist 4y ago
  I want to get into that for CUDA. I've been using Vulkan via WGPU. But there, everything has to pass between CPU and GPU as byte arrays, when I've heard CUDA lets you maintain the same data structures. On the to-do list. Would also def benefit from the larger selection of C++ libs, especially for GUI and graphics. Given the main computation issue is doing the same operation on many values in a 3D array, using the GPU more would be great. And my current setup for that is clumsy. I disagree on easier to share. Binaries work the same either way. Compiling from source on rust is generally install Rustc, and do `cargo b --release`. Compiling someone else's code in C++ is... complicated.
  - einpoklum 4y ago
    > when I've heard CUDA lets you maintain the same data structure Yes, that's possible with a combination of: * Unified address space for pointers (and C++ references are basically pointers) * Paged virtual memory on GPUs ... however, remember that many/most data structures which make sense on the CPU may not make sense to use on a GPU. And - NVIDIA very often implements and promotes certain features because they sound good in marketing, not because they're actually that useful. Or because they let you make a super-slow program into a meh-speed program, not because you would use them in a really-fast program. > Compiling from source on rust is generally install Rustc, and do `cargo b --release`. Well, I guess you may have a point, but let me still make a couple more arguments. First, installing Rust (and related libraries/tools/whatever) is not trivial for people who don't know rust, and some OS distributions only offer you an older version of rust using their distro-package-managers. It's more likely that a C++ environment is already set up on someone's machine... TBH, though, if the distro is older, that might not be a perfect fit for what you're building. Second C++ package managers becoming more popular, e.g. Conan: https://conan.io/ https://conan.io/ ; and that helps for when your OS distro doesn't cover you. > Compiling someone else's code in C++ is... complicated. This has actually improved a whole lot over the past... say, decade or so! * Unzip/untar * Configure the build with CMake (cross-platform build system generator - almost ubiquitous these days in C++ projects): `cmake -B build_dir/` * Assuming you have the dependencies set up - it should "just build": `cmake --build build_dir` * Run the thing from the build directory or install it with `cmake --install build_dir`. ... and if you want to tweak package options more easily than via the command-line, you can use a TUI (ccmake) or GUI (cmake-gui). The problems start when you're missing dependencies, or if the author didn't make their code platform-independent / multi-platform.
flatiron 4y ago
If sharing is a problem I’m unsure why you didn’t think docker was the solution. I can’t think of an easier way for both the provider and the consumer than docker.
- Thiez 4y ago
  They did mention licenses as a complicating factor.

wodenokoto 4y ago

Maybe it is not part of computational chemistry, but how do you do explorative analysis? More than half my time working with data is spend grouping and summarising data, typically in Pandas and drawing interactive graphs in plotly.

the__alchemist 4y ago
Plots and visualizations in 3D. (Vulkan / WGPU) For example, to compare wave function solutions, I use surface plots of a 2D slice. I have a UI that can change the slice of the non-mapped axis. Plots of psi'', the potential etc too. Or for simulating molecular motion, draw it out in 3D, with 6DOF FPS-style camera controls etc. I imagine Pandas would be OOM too slow for this. Compared to Numpy, Pandas is, in my experience far too slow. Another speed limitation is Matplotlib's 3D plots make my computer crawl when rotating them etc; and they're not very interactive, at least on their own. When dealing with these systems, a particular computational challenge is that they're 3D, so computation scales with precision^3. (This also adds complications to visualization, as I alluded to above)
- legerdemain 4y ago
  Did you write your own visualizations using wgpu? I don't think its goals include providing visualization out of the box.
  - the__alchemist 4y ago
    Yep! So, took some work upfront. Basic rendering/navigation engine. `egui` for the GUI.

akuro 4y ago

Interesting. You might be the first computational chemist I know who actually uses Rust. I know a lot of computational chemists! Python is the big one, all of the aforementioned chemists are either intermediate or advanced in that. The runner-up seems to be Julia, which I personally have no experience with. The big guys are Fortran and C++. I prefer C for tasks of this nature, but I also shill Scheme so don't listen to my opinions on programming languages. Best of luck on your computational chemistry endeavours!

doubleg72 4y ago
[flagged]
- akuro 4y ago
  Don't listen to my opinions, sure. But pay heed to the large number of computational chemists who do use these languages. There is a reason that professionals employ the tools that they do.
  - freehorse 4y ago
    Historical reasons, most often than not.
  - tialaramex 4y ago
    One reason we have lots of people in numerate disciplines using Python is that we taught them Python. Python is very teachable. Given a class of average 19 year old students from a numerate discipline, my colleagues will teach most of them Python to a reasonable standard in a single module (so e.g 2 hours of teaching per week over 16 weeks). The same would not be true if we were supposed to teach them C++ for example. It's a huge, sprawling language, full of difficult concepts that are unfriendly to teach, and equally filled with foot guns that if you don't teach them will definitely maim your students, but if you do teach them take up yet more precious time on minutiae. Safe Rust wouldn't be as hard to teach as C++ but it's no picnic. So even if the Chemists decided that ideally they'd like their undergraduates to learn Rust instead of Python, I think the argument would be that it can't be done on the existing timeline.

BaculumMeumEst 4y ago

Python can be very fast. Library ecosystem and network effects by far outweigh anything you listed. Mindless Rust evangelism is tiresome.

deleted 4y ago
[deleted]
patrick451 4y ago
One thing is for sure, rust has excellent marketing.
- pdimitar 4y ago
  Or it's actually good and you're bemoaning it due to bias. It's worth exploring the areas outside of our comfort zone.
  - patrick451 4y ago
    I've tried it, and it seems interesting. But I distrust anything that appears to have a lot of propaganda behind it, and rust definitely fits that bill. The hype will need to die down and the dust settle before I trust anything important to it.
    - pdimitar 4y ago
      I think you are being unfair here. Literally every community has the 1% of toxic, loud and insufferable people in it. Point me at your hobbies and I'll find these kinds of people there, I can bet my neck on it. Many people conflate their own irritation from hearing about the thing X often and against their will, with the quality of the thing X. IMHO critical thinking and maturity demands that we make that distinction. I have worked several times in Rust companies and their devs are extremely hardcore and very quiet. You couldn't tell these people are responsible for the technical operations and code of worldwide financial services. So again, be a bit more charitable. Every community has its rotten apples. That says nothing about the quality of the thing that the community stands for.

einpoklum 4y ago

Given your criteria, you might want to consider (modern) C++. * Fast - in many cases faster than Rust, although the difference is inconsequential relative to Python-to-Rust improvement I guess. * _Really_ utilize CUDA, OpenCL, Vulcan etc. Specifically, Rust GPU is limited in its supported features, see: https://github.com/Rust-GPU/Rust-CUDA/blob/master/guide/src/features.md https://github.com/Rust-GPU/Rust-CUDA/blob/master/guide/src/... ... * Host-side use of CUDA is at least as nice, and probably nicer, than what you'll get with Rust. That is, provided you use my own Modern C++ wrappers for the CUDA APIs: https://github.com/eyalroz/cuda-api-wrappers/ https://github.com/eyalroz/cuda-api-wrappers/ :-) ... sorry for the shameless self-plug. * ... which brings me to another point: Richer offering of libraries for various needs than Rust, for you to possibly utilize. * Easier to share than Rust. A target system is less likely to have an appropriate version of Rust and the surrounding ecosystem. There are downsides, of course, but I was just applying your criteria.

the__alchemist 4y ago
I want to get into that for CUDA. I've been using Vulkan via WGPU. But there, everything has to pass between CPU and GPU as byte arrays, when I've heard CUDA lets you maintain the same data structures. On the to-do list. Would also def benefit from the larger selection of C++ libs, especially for GUI and graphics. Given the main computation issue is doing the same operation on many values in a 3D array, using the GPU more would be great. And my current setup for that is clumsy. I disagree on easier to share. Binaries work the same either way. Compiling from source on rust is generally install Rustc, and do `cargo b --release`. Compiling someone else's code in C++ is... complicated.
- einpoklum 4y ago
  > when I've heard CUDA lets you maintain the same data structure Yes, that's possible with a combination of: * Unified address space for pointers (and C++ references are basically pointers) * Paged virtual memory on GPUs ... however, remember that many/most data structures which make sense on the CPU may not make sense to use on a GPU. And - NVIDIA very often implements and promotes certain features because they sound good in marketing, not because they're actually that useful. Or because they let you make a super-slow program into a meh-speed program, not because you would use them in a really-fast program. > Compiling from source on rust is generally install Rustc, and do `cargo b --release`. Well, I guess you may have a point, but let me still make a couple more arguments. First, installing Rust (and related libraries/tools/whatever) is not trivial for people who don't know rust, and some OS distributions only offer you an older version of rust using their distro-package-managers. It's more likely that a C++ environment is already set up on someone's machine... TBH, though, if the distro is older, that might not be a perfect fit for what you're building. Second C++ package managers becoming more popular, e.g. Conan: https://conan.io/ https://conan.io/ ; and that helps for when your OS distro doesn't cover you. > Compiling someone else's code in C++ is... complicated. This has actually improved a whole lot over the past... say, decade or so! * Unzip/untar * Configure the build with CMake (cross-platform build system generator - almost ubiquitous these days in C++ projects): `cmake -B build_dir/` * Assuming you have the dependencies set up - it should "just build": `cmake --build build_dir` * Run the thing from the build directory or install it with `cmake --install build_dir`. ... and if you want to tweak package options more easily than via the command-line, you can use a TUI (ccmake) or GUI (cmake-gui). The problems start when you're missing dependencies, or if the author didn't make their code platform-independent / multi-platform.

flatiron 4y ago

If sharing is a problem I’m unsure why you didn’t think docker was the solution. I can’t think of an easier way for both the provider and the consumer than docker.

Thiez 4y ago
They did mention licenses as a complicating factor.