3 ms·

Well, the second Steve case really depends on the overall outlook. If a big chunk needs to be in C++ then the question is, do you really want to have two langua

Well, the second Steve case really depends on the overall outlook. If a big chunk needs to be in C++ then the question is, do you really want to have two languages at the problem? Especially if the goal is a shipable binary later (which is possible but painful in python). Also it is a question about available stuff.

Stratoscope 5y ago
Yes, those are valid and important considerations. In this case it didn't need to be a shippable binary, it was just run internally. We had the infrastructure to seamlessly handle a mix of Python and C++, and the team members were familiar with both languages - other than the exceptions noted above. This was a set of geographic data importers for some messy data in a variety of formats from multiple countries. The data quality was so bad that north and south were sometimes reversed, or even worse, north and west would be reversed, among many other problems. This required a lot of ad hoc code to clean up the data. It was originally written in C++, with all the data importers linked into a monolithic binary with an eight-minute build time for a one-line change. (Or a half-hour for a full build.) So if you spotted one data error and wanted to clean it up, it was an eight-minute wait to try your code. When I changed it to use a Python front end that cleaned up the data and fed it to a simplified C++ back end for the compute intensive part, the C++ code hardly ever had to be rebuilt, and there was no build time at all for the Python part. So you could explore the data interactively and easily figure out how to clean it up. Technically, this was a huge win. Socially, not so much. OTOH, if you ever interview me and ask the "behavioral" questions: "Tell me about a time when a project went really well" and then "Tell me about a time when a project didn't go so well", I can use this episode for both questions!
- andi999 5y ago
  Thanks. I agree, this is like a textbook case when to use a hybrid approach. Maybe the manager wanted to make sure he will always have enough people assigned to the project (so make it more difficult to maintain)? (A very dark management pattern)
adsharma 5y ago
The idea that py2many is pursuing is to avoid having to deal with two languages and interfacing via python C-API. In other words you write your C++ or Rust with the python syntax, fixed width, signed/unsigned integer types and static typing and transpile to get the same effect as writing C++ by hand. You'd give up any dynamic features of cpython in the process.
- andi999 5y ago
  I do not think the idea of a transpiler will fly. (In the sense the market will not really accept it)

Stratoscope 5y ago

Yes, those are valid and important considerations. In this case it didn't need to be a shippable binary, it was just run internally. We had the infrastructure to seamlessly handle a mix of Python and C++, and the team members were familiar with both languages - other than the exceptions noted above. This was a set of geographic data importers for some messy data in a variety of formats from multiple countries. The data quality was so bad that north and south were sometimes reversed, or even worse, north and west would be reversed, among many other problems. This required a lot of ad hoc code to clean up the data. It was originally written in C++, with all the data importers linked into a monolithic binary with an eight-minute build time for a one-line change. (Or a half-hour for a full build.) So if you spotted one data error and wanted to clean it up, it was an eight-minute wait to try your code. When I changed it to use a Python front end that cleaned up the data and fed it to a simplified C++ back end for the compute intensive part, the C++ code hardly ever had to be rebuilt, and there was no build time at all for the Python part. So you could explore the data interactively and easily figure out how to clean it up. Technically, this was a huge win. Socially, not so much. OTOH, if you ever interview me and ask the "behavioral" questions: "Tell me about a time when a project went really well" and then "Tell me about a time when a project didn't go so well", I can use this episode for both questions!

andi999 5y ago
Thanks. I agree, this is like a textbook case when to use a hybrid approach. Maybe the manager wanted to make sure he will always have enough people assigned to the project (so make it more difficult to maintain)? (A very dark management pattern)

adsharma 5y ago

The idea that py2many is pursuing is to avoid having to deal with two languages and interfacing via python C-API. In other words you write your C++ or Rust with the python syntax, fixed width, signed/unsigned integer types and static typing and transpile to get the same effect as writing C++ by hand. You'd give up any dynamic features of cpython in the process.

andi999 5y ago
I do not think the idea of a transpiler will fly. (In the sense the market will not really accept it)