5 ms·

> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. People using c++ usually care abo

by josefx 6y ago

> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. People using c++ usually care about programs finishing before the heat death of the universe. I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant.

Barrin92 6y ago
it depends on what you're doing of course. It can even go the other way[1] Generally languages like C++ are obviously much faster, but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster, because doing things in low level compiled languages often comes with its own tricks and problems. [1]https://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python https://stackoverflow.com/questions/9371238/why-is-reading-l...
- barumi 6y ago
  > but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is, and Python's GIL simply eliminates that option except for multi-process applications which then require serializing stuff back and forth. Python is pretty much unparalleled as a glue language, and excels at putting together small exploratory scripts for data analysis and number crunching applications, but it makes no sense to present Python as a performant alternative to C++.
  - quietbritishjim 6y ago
    The overwhelming majority of C extensions release the Python GIL during calls to them. If you're in the situation described in the parent comment then you can parallelize just fine using threads.
    - deleted 6y ago
      [deleted]
    - barumi 6y ago
      > The overwhelming majority of C extensions release the Python GIL during calls to them. In that corner case you still have slow Python glue code calling fast C++ code. It makes no sense to claim that Python is performant based on the idea that it may be used to glue together calls to performant C++ code, while ignoring the fact that not only Python forces performance restrictions on it's code but also that C++ code is quite capable of calling performant C++ code itself.
      - quietbritishjim 6y ago
        The comment that started this discussion is about the specific case that most of your execution time is in libraries rather than your application code. In that Python is usually no slower to execute than C/C++. You disagreed with that by claiming that such programs cannot be multi threaded because of that GIL, and I just corrected that because it's untrue (edit: it's untrue in general, but especially untrue in this situation we're talking about). To be honest I'm not sure what your new comment is about. If you're saying that Python is not necessarily faster than C then I'm sure no one going to dispute that; in the situation we're discussing, the performance of your code is totally dependant on the efficiency of the libraries that are doing all the work, not your glue code. If you're saying that Python is slower than C when it's doing a non-trivial amount of the execution, then sure, but no one was claiming that either. Or maybe you just want to simplistically categorise languages as "performat" and "non-performant" without thinking about the specific contexts they can be used in, but that wouldn't make any sense. Did I misunderstand?
  - jnxx 6y ago
    > That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is I would not support that stance. There are areas where parallelization is helpful, like numerical weather models, graphics, and scientific number crunching, but for by far the most cases, parallelization is anything but trivial and requires a pretty deep knowledge about things like the C++ memory models, what read-write barriers, locks, and fences are, and so on. Less than 1% of C++ programmers can really handle that. And apart from that this is an area where other languages like Rust, Clojure, Scala have a strong point, because whether it is done correctly is completely implicit in a C++ program while in Rust, many errors will have the result that your program does not compile. Debugging the same mistakes in C++ will make you pull your hair. In fact, I'd urge anyone who wants to learn principles of safe concurrent computation in C++, to learn Rust, Clojure and Scala first, they are wonderful languages with strong, highly coherent concepts, and it will make you a much better C++ programmer even if you never use them again. Of course it is not only possible but common practice to implement parallel computation in games and such in C++, but this is not the standard application of C++.
    - pumpkinpal 6y ago
      Its funny that you mentioned games as an example of parallel computation because I'd argue they're some of the hardest programs to parallelize effectively since they don't generally involve that much bulk-processing of read-only data. Parellism in C++ is most often used for scientific applications and other forms of mass number crunching. It's really easy to just throw a "#pragma omp parallel for" on a loop and call it a day but of course that would also apply to C and Fortran and is somewhat limited. Parallelism libraries like Intel TBB which I'm most familiar with are very easy to use and performant. I think there's a large problem in the reluctance of educators to use libraries to teach parallelism and people always dive straight into locks, threads and atomics which are really not the way to approach parallel computing if you're looking to do parallel computing and not looking to implement parallel primitives yourself (i.e DIY tasking-system or lock-free queue) Focusing on TBB, it facilitates efficient parallelism by providing high-level canned algorithms such as parallel_invoke, parallel_reduce, parallel_for and parallel_do which anybody who claims to know C++ should be able to use easily. It also provides a task-graph which is great for more complex processing pipelines (things like join/split, fan-in/out and queueing). If you need more low level control you operate at the task level and TBB provides customization points for that. There's other libraries out there which provide similar functionality and even the STL in C++17 provides basic parallel algorithms such as transform (equivalent of map in other langs), reduce and many others.
  - pumpkinpal 6y ago
    Python and C++ are joined at the hip now primarily due to scientific computing and AI/ML adjacent fields. I feel the growth of Python has helped C++ grow too.
- formerly_proven 6y ago
  To be fair, C++'s standard I/O library is really bad. There is probably an almost universal consensus in the C++ world that the locale library is the worst standard library, immediately followed by iostreams. For small programs that's annoying, although using libc IO is usually fine for them. However, C++ is probably one of the top 3 languages for developing large applications (Java would also be there, not sure which one is the third in that pack) and for these having to roll your own IO is usually no biggie, because it probably would have been done anyway (e.g. even if iostreams wouldn't be so bad, it would likely have no place in the IO system of a content creation app or being used in HDF5). In that sense, C++ is not batteries included, which can be annoying for small and medium projects.
  - Retric 6y ago
    In terms of large projects C++’s role is very much shrinking. C++’s last major strength is being cross platform, but new platforms keep excluding it. You can’t program on iOS or client side websites with it. Microsoft actively discourages using C++ on Windows, and C# took over for large projects. Essentially C++ and the modern ideas about computer security are at war.
    - CodeGlitch 6y ago
      Just thought I'd post to say I think you are spot on with this comment.
    - Sindisil 6y ago
      > Microsoft actively discourages using C++ on Windows, Really? That's the opposite of my impression. It seems to me that they promote it as co-equal to C# as primary focus in their constellation of supported and encouraged application programming languages.
    - lenkite 6y ago
      Modern C++ is actually encouraged on Windows with Microsoft making standards compliant and supported frameworks like C++/WinRT for WinUI3. https://docs.microsoft.com/en-us/windows/apps/winui/winui3/xaml-templated-controls-cppwinrt-winui-3 https://docs.microsoft.com/en-us/windows/apps/winui/winui3/x...
dmortin 6y ago
> I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant. It depends on the task. If the parsing is done once a day then several seconds is nothing, so there is no point in using C++. If the parser is running very frequently then of course one uses a faster implementation.
- josefx 6y ago
  In my case it lead to a reverse on that reasoning. The script ended up being an explicit step to import the data when it changed instead of running implicitly whenever the data was needed. Of course that choice was based on the initial version which was even slower and didn't use multiprocessing to handle the files in parallel.
rndgermandude 6y ago
>I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant. Was it worth rewriting those scripts in C++? I'm not trying to be facetious; a few seconds (or minutes) of additional runtime for scripts you hardly ever run or even run only once doesn't really matter, but running such scripts often may then worth the effort to convert them to something faster. I regularly find myself porting regularly used python/js scripts to C#/dotnet (because it's fast enough, has good collections + LINQ for your somewhat-functional-programming needs and I can skip manual memory management, and async/await programming is not perfect but available and easy enough, + nuget has a lot of stuff and so does the standard lib). What made you pick C++ over e.g. go or rust or C#?
- josefx 6y ago
  > Was it worth rewriting those scripts in C++? Mostly for development, I ended up invoking the script quite often and had to wait for the results. In production the input it ran on only changed once a week, so there was no need to run it often. I most likely tried to run it with Pypy first, thought I am not 100% sure on that. If I ran into a similar issue today I would first try to compile it with Cython as I already try to use type annotations in every new script I write. > What made you pick C++ over e.g. go or rust or C#? Most of the project was already C++ and I only had to link in an Xml library for some of the output.
lokedhs 6y ago
You just compared C++ to the slowest popular language out there. It would be a much more fair comparison if you used pretty much any other high-level language. Since you can get within about 2x of C++ speed which still using a high-level language, that makes the argument that the times you really need C++ for performance reasons are quite limited. I'm not saying they don't exist, but they are rare.

by josefx 6y ago

Barrin92 6y ago

it depends on what you're doing of course. It can even go the other way[1] Generally languages like C++ are obviously much faster, but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster, because doing things in low level compiled languages often comes with its own tricks and problems. [1]https://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python https://stackoverflow.com/questions/9371238/why-is-reading-l...

barumi 6y ago
> but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is, and Python's GIL simply eliminates that option except for multi-process applications which then require serializing stuff back and forth. Python is pretty much unparalleled as a glue language, and excels at putting together small exploratory scripts for data analysis and number crunching applications, but it makes no sense to present Python as a performant alternative to C++.
- quietbritishjim 6y ago
  The overwhelming majority of C extensions release the Python GIL during calls to them. If you're in the situation described in the parent comment then you can parallelize just fine using threads.
  - deleted 6y ago
    [deleted]
  - barumi 6y ago
    > The overwhelming majority of C extensions release the Python GIL during calls to them. In that corner case you still have slow Python glue code calling fast C++ code. It makes no sense to claim that Python is performant based on the idea that it may be used to glue together calls to performant C++ code, while ignoring the fact that not only Python forces performance restrictions on it's code but also that C++ code is quite capable of calling performant C++ code itself.
    - quietbritishjim 6y ago
      The comment that started this discussion is about the specific case that most of your execution time is in libraries rather than your application code. In that Python is usually no slower to execute than C/C++. You disagreed with that by claiming that such programs cannot be multi threaded because of that GIL, and I just corrected that because it's untrue (edit: it's untrue in general, but especially untrue in this situation we're talking about). To be honest I'm not sure what your new comment is about. If you're saying that Python is not necessarily faster than C then I'm sure no one going to dispute that; in the situation we're discussing, the performance of your code is totally dependant on the efficiency of the libraries that are doing all the work, not your glue code. If you're saying that Python is slower than C when it's doing a non-trivial amount of the execution, then sure, but no one was claiming that either. Or maybe you just want to simplistically categorise languages as "performat" and "non-performant" without thinking about the specific contexts they can be used in, but that wouldn't make any sense. Did I misunderstand?
- jnxx 6y ago
  > That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is I would not support that stance. There are areas where parallelization is helpful, like numerical weather models, graphics, and scientific number crunching, but for by far the most cases, parallelization is anything but trivial and requires a pretty deep knowledge about things like the C++ memory models, what read-write barriers, locks, and fences are, and so on. Less than 1% of C++ programmers can really handle that. And apart from that this is an area where other languages like Rust, Clojure, Scala have a strong point, because whether it is done correctly is completely implicit in a C++ program while in Rust, many errors will have the result that your program does not compile. Debugging the same mistakes in C++ will make you pull your hair. In fact, I'd urge anyone who wants to learn principles of safe concurrent computation in C++, to learn Rust, Clojure and Scala first, they are wonderful languages with strong, highly coherent concepts, and it will make you a much better C++ programmer even if you never use them again. Of course it is not only possible but common practice to implement parallel computation in games and such in C++, but this is not the standard application of C++.
  - pumpkinpal 6y ago
    Its funny that you mentioned games as an example of parallel computation because I'd argue they're some of the hardest programs to parallelize effectively since they don't generally involve that much bulk-processing of read-only data. Parellism in C++ is most often used for scientific applications and other forms of mass number crunching. It's really easy to just throw a "#pragma omp parallel for" on a loop and call it a day but of course that would also apply to C and Fortran and is somewhat limited. Parallelism libraries like Intel TBB which I'm most familiar with are very easy to use and performant. I think there's a large problem in the reluctance of educators to use libraries to teach parallelism and people always dive straight into locks, threads and atomics which are really not the way to approach parallel computing if you're looking to do parallel computing and not looking to implement parallel primitives yourself (i.e DIY tasking-system or lock-free queue) Focusing on TBB, it facilitates efficient parallelism by providing high-level canned algorithms such as parallel_invoke, parallel_reduce, parallel_for and parallel_do which anybody who claims to know C++ should be able to use easily. It also provides a task-graph which is great for more complex processing pipelines (things like join/split, fan-in/out and queueing). If you need more low level control you operate at the task level and TBB provides customization points for that. There's other libraries out there which provide similar functionality and even the STL in C++17 provides basic parallel algorithms such as transform (equivalent of map in other langs), reduce and many others.
- pumpkinpal 6y ago
  Python and C++ are joined at the hip now primarily due to scientific computing and AI/ML adjacent fields. I feel the growth of Python has helped C++ grow too.
formerly_proven 6y ago
To be fair, C++'s standard I/O library is really bad. There is probably an almost universal consensus in the C++ world that the locale library is the worst standard library, immediately followed by iostreams. For small programs that's annoying, although using libc IO is usually fine for them. However, C++ is probably one of the top 3 languages for developing large applications (Java would also be there, not sure which one is the third in that pack) and for these having to roll your own IO is usually no biggie, because it probably would have been done anyway (e.g. even if iostreams wouldn't be so bad, it would likely have no place in the IO system of a content creation app or being used in HDF5). In that sense, C++ is not batteries included, which can be annoying for small and medium projects.
- Retric 6y ago
  In terms of large projects C++’s role is very much shrinking. C++’s last major strength is being cross platform, but new platforms keep excluding it. You can’t program on iOS or client side websites with it. Microsoft actively discourages using C++ on Windows, and C# took over for large projects. Essentially C++ and the modern ideas about computer security are at war.
  - CodeGlitch 6y ago
    Just thought I'd post to say I think you are spot on with this comment.
  - Sindisil 6y ago
    > Microsoft actively discourages using C++ on Windows, Really? That's the opposite of my impression. It seems to me that they promote it as co-equal to C# as primary focus in their constellation of supported and encouraged application programming languages.
  - lenkite 6y ago
    Modern C++ is actually encouraged on Windows with Microsoft making standards compliant and supported frameworks like C++/WinRT for WinUI3. https://docs.microsoft.com/en-us/windows/apps/winui/winui3/xaml-templated-controls-cppwinrt-winui-3 https://docs.microsoft.com/en-us/windows/apps/winui/winui3/x...

dmortin 6y ago

> I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant. It depends on the task. If the parsing is done once a day then several seconds is nothing, so there is no point in using C++. If the parser is running very frequently then of course one uses a faster implementation.

josefx 6y ago
In my case it lead to a reverse on that reasoning. The script ended up being an explicit step to import the data when it changed instead of running implicitly whenever the data was needed. Of course that choice was based on the initial version which was even slower and didn't use multiprocessing to handle the files in parallel.

rndgermandude 6y ago

>I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant. Was it worth rewriting those scripts in C++? I'm not trying to be facetious; a few seconds (or minutes) of additional runtime for scripts you hardly ever run or even run only once doesn't really matter, but running such scripts often may then worth the effort to convert them to something faster. I regularly find myself porting regularly used python/js scripts to C#/dotnet (because it's fast enough, has good collections + LINQ for your somewhat-functional-programming needs and I can skip manual memory management, and async/await programming is not perfect but available and easy enough, + nuget has a lot of stuff and so does the standard lib). What made you pick C++ over e.g. go or rust or C#?

josefx 6y ago
> Was it worth rewriting those scripts in C++? Mostly for development, I ended up invoking the script quite often and had to wait for the results. In production the input it ran on only changed once a week, so there was no need to run it often. I most likely tried to run it with Pypy first, thought I am not 100% sure on that. If I ran into a similar issue today I would first try to compile it with Cython as I already try to use type annotations in every new script I write. > What made you pick C++ over e.g. go or rust or C#? Most of the project was already C++ and I only had to link in an Xml library for some of the output.

lokedhs 6y ago

You just compared C++ to the slowest popular language out there. It would be a much more fair comparison if you used pretty much any other high-level language. Since you can get within about 2x of C++ speed which still using a high-level language, that makes the argument that the times you really need C++ for performance reasons are quite limited. I'm not saying they don't exist, but they are rare.