4 ms·

I've written quite a bit of production code in C++, Python and Rust, and currently work on a hybrid Rust/Python system. Here's my experience: - C++ is an unusu

by ekidd 4y ago

I've written quite a bit of production code in C++, Python and Rust, and currently work on a hybrid Rust/Python system. Here's my experience: - C++ is an unusually large language. And it has many historic footguns, requiring a higher level of vigilance and code review. If I were starting a brand new project today, I wouldn't try to build a team of C++ programmers. - Untyped Python becomes more difficult to refactor and maintain once you reach 50k to 80k lines on a group project. Typed Python, however, scales nicely beyond this size. - Rust is a "medium-sized" language. It requires developers to learn more than Go or Python does, but less than C++. And Rust has far fewer traps for the unwary and the reckless than C++. Rust's tooling is also very good in many areas. - It's tempting to split a project into a fast "core" language, and high-level "glue" language. There are real advantages to this. (Which is why I've done it on one recent project!) But this also comes with costs: everyone needs to be fairly good at two languages, and switch back and forth. And you pay a tax at the boundary. If I were building a brand new database (and a team to maintain it), I'd actually be strongly tempted to use Rust exclusively. But this is partly because databases rarely have a "business logic" layer that changes constantly, so there's less need for a high level scripting language. But with a different team or different constraints, C++ could also be the right choice.

hutzlibu 4y ago
"But this also comes with costs: everyone needs to be fairly good at two languages, and switch back and forth." Why does everyone needs to be good at both languages? You can seperate and have the core people writing efficient low level code - and you have higher level scripting/gluing code.
- pornel 4y ago
  You will have Conway's law in your codebase. Coordination between teams is hard, so teams will prefer to implement features entirely in their language, even where that is technically suboptimal. You will get hot loops in Python, because a Rust programmer wasn't around, and Rust programmers implementing whole complex business logic in Rust behind a single `do_it()` Python call.
  - hutzlibu 4y ago
    Communication and coordination is surely hard and things like that surely happen, but this is why project management exist. If it is doing things right, then the rust people don't do complex buisness logic, because it is not assigned to them and they would not even have the details. And if the python people were too eager and have core stuff implemented and it is affecting performance, than you can always reimplement it low level. It all depends on the project of course, of what would be the best mix.
lupire 4y ago
A brand new project doesn't need legacy C++ footguns. It can use modern C++. The part of Python (usually) is that you don't need to be "good at it" it you aren't trying to write super polymorphic core that runs super efficient computations like scipy. If you have a fast core engine for the innner loop, a slow Python management layer is plenty fast.
dkarl 4y ago
> Typed Python, however, scales nicely beyond this size. Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend. > It's tempting to split a project into a fast "core" language, and high-level "glue" language. I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C.
- heavyset_go 4y ago
  > Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend. Python with type annotations works really well with type checkers like Mypy, along with LSP servers, and both of those integrate with most development environments. Using a Python-oriented IDE like Pycharm with type annotated Python also allows for better refactoring options. It reduces the uncertainty and guesswork an IDE's static analyzer must engage in for even basic features you'd take for granted with IDE and statically typed languages. In practice, developers don't have to keep what can be a massively complex application running in their heads to modify code accurately. A nicely typed project makes it easy to see exactly what types of data get passed around and modified. Before gradual typing, you'd have to backtrack to all of a function's call sites to understand exactly what kind of data it takes and returns. With gradual typing, you can just look at types and rely on Mypy to ensure the right data is passed around. > I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C. I haven't used Boost Python, but there are some options for Rust and Python that work well and seem to suit this use case like PyO3.
exceptione 4y ago
Especially business logic should be taken in the firm grip of static compile time guarantees that the hand of a strong type system delivers. Even more so if it changes constantly! Refactoring without fear. Only software that does not have to run correctly (prototypes, personal hobby projects) can get away with a non-static type system. When I have to pick a tool and I see it is written in Python I will have a look for an alternative if possible. Because I know it will have many bugs: some known, lots hidden.

by ekidd 4y ago

hutzlibu 4y ago

"But this also comes with costs: everyone needs to be fairly good at two languages, and switch back and forth." Why does everyone needs to be good at both languages? You can seperate and have the core people writing efficient low level code - and you have higher level scripting/gluing code.

pornel 4y ago
You will have Conway's law in your codebase. Coordination between teams is hard, so teams will prefer to implement features entirely in their language, even where that is technically suboptimal. You will get hot loops in Python, because a Rust programmer wasn't around, and Rust programmers implementing whole complex business logic in Rust behind a single `do_it()` Python call.
- hutzlibu 4y ago
  Communication and coordination is surely hard and things like that surely happen, but this is why project management exist. If it is doing things right, then the rust people don't do complex buisness logic, because it is not assigned to them and they would not even have the details. And if the python people were too eager and have core stuff implemented and it is affecting performance, than you can always reimplement it low level. It all depends on the project of course, of what would be the best mix.

lupire 4y ago

A brand new project doesn't need legacy C++ footguns. It can use modern C++. The part of Python (usually) is that you don't need to be "good at it" it you aren't trying to write super polymorphic core that runs super efficient computations like scipy. If you have a fast core engine for the innner loop, a slow Python management layer is plenty fast.

dkarl 4y ago

> Typed Python, however, scales nicely beyond this size. Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend. > It's tempting to split a project into a fast "core" language, and high-level "glue" language. I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C.

heavyset_go 4y ago
> Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend. Python with type annotations works really well with type checkers like Mypy, along with LSP servers, and both of those integrate with most development environments. Using a Python-oriented IDE like Pycharm with type annotated Python also allows for better refactoring options. It reduces the uncertainty and guesswork an IDE's static analyzer must engage in for even basic features you'd take for granted with IDE and statically typed languages. In practice, developers don't have to keep what can be a massively complex application running in their heads to modify code accurately. A nicely typed project makes it easy to see exactly what types of data get passed around and modified. Before gradual typing, you'd have to backtrack to all of a function's call sites to understand exactly what kind of data it takes and returns. With gradual typing, you can just look at types and rely on Mypy to ensure the right data is passed around. > I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C. I haven't used Boost Python, but there are some options for Rust and Python that work well and seem to suit this use case like PyO3.

exceptione 4y ago

Especially business logic should be taken in the firm grip of static compile time guarantees that the hand of a strong type system delivers. Even more so if it changes constantly! Refactoring without fear. Only software that does not have to run correctly (prototypes, personal hobby projects) can get away with a non-static type system. When I have to pick a tool and I see it is written in Python I will have a look for an alternative if possible. Because I know it will have many bugs: some known, lots hidden.