10 ms·

Like every programming language out there, C++ is a tool. And like every tool out there it has its uses. There isn't any point of using C++ to count words in a

Like every programming language out there, C++ is a tool. And like every tool out there it has its uses. There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. However, there's one thing you can do in C++ and not in Python or JavaScript or PHP: fully control the memory layout of your data. While you don't need it in most of the cases, it becomes a killer feature when directly dealing with large amounts of data: * Try implementing an on-disk hash table in Python and you're stuck manually packing ints and longs in an out of arrays. In C++ it could a simple template used with a memory-mapped file. * Try doing anything non-trivial on a microcontroller with 32KB of RAM. You could theoretically use a higher-level language, but you will end up using >10x amount of RAM. * Try designing an application-specific data structure in any other language. Let's say you have an ~8GB in-memory database that slowly adds records one-by-one and then invalidates them in chunks. A C++ implementation will rip anything else to shreds. You just won't get the same speed and memory efficiency. What it means in practice, is that unless you have an existing project that uses C++ anyway, you want to partition it: do the memory-critical part in C++, and communicate to it from a higher-level language via a high-level interface. You will get performance where you need it and a peace of mind everywhere else.

wirthjason 6y ago
> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. However, there's one thing you can do in C++ and not in Python or JavaScript or PHP: fully control the memory layout of your data. Any books or resources for digging deeper in these kinds of topics? I’m a Python developer and would like to learn C++ but the just the reasons you mentioned make it hard to leave Python.
- danudey 6y ago
  you can malloc() to get a memory range, or mmap() to get a disk-backed memory range with kernel-managed paging, and then you can do whatever you want with it. You get a pointer to the start of the range, and you know how large it is, so from there it's just pointer arithmetic and assignment. Easy peasy. Now you just have to make sure that you zero data when you're done with it, track how much of that memory is available or used (and which sections), and (optionally) implement your own defragmentation and garbage collection. Unless you desperately need to specifically manage the memory layout of your data (e.g. for alignment purposes for massive data processing), none of this is worth the time or trouble to do for most projects, not only because it's a huge amount of work for little gain, but because you'll almost definitely screw it up and potentially corrupt your entire data set. Note that, at least for the on-disk hashmap set, you absolutely can do this in python, so if you're interested in experimenting with this sort of data management, practice it there first. A good place to start: https://github.com/luispedro/diskhash https://github.com/luispedro/diskhash
  - barumi 6y ago
    > you can malloc() to get a memory range, or mmap() to get a disk-backed memory range with kernel-managed paging, and then you can do whatever you want with it. No need for that, as C++ provides placement new(). https://en.cppreference.com/w/cpp/language/new https://en.cppreference.com/w/cpp/language/new
    - saagarjha 6y ago
      This doesn’t really solve the same problem, even though the two often go hand-in-hand.
  - gpderetta 6y ago
    It is really not that much work and there are plenty of libs that can do it for you if you do not want to.
- john_moscow 6y ago
  To get a quick start, try declaring some class variables and then open the Memory window and go to "&variable_name". You can also evaluate "sizeof(variable_name)" to get its exact size. You'll see a bunch of bytes reflecting how exactly your variable is seen by the CPU. That's exactly how many RAM bytes it occupies. It can quickly copied to a new location, limited only by your RAM bandwidth. It can be dumped to a disk and loaded later, limited only by your disk bandwidth.
  - saagarjha 6y ago
    How does this help?
- nendroid 6y ago
  Python developer here just switched jobs to one that does C++. Let me warn you right now. It's not worth it. At all. Don't go down this path. Memory management is easy, but that's not the only crap you have to deal with. Templating and the whole C++ ecosystem with Cmake compiling to make is a giant headache. It takes about a week to integrate a new C++ library in source code and people are worried about installing or "trying" out new libraries because of memory leaks.... while in python I'll install an entire library in seconds just to try out a function. I would say if you made an app in python and the same exact app in C++, the C++ app would perform waay better. No question. But the developer experience and effort required to make that same app is much much more painful.
  - rightbyte 6y ago
    It just sounds like you are working on a big messy project.
    - nendroid 6y ago
      Nah it's a big unicorn startup down in Irvine, so the codebase is still relatively new. They use C++17 so pretty modern as well.
  - winrid 6y ago
    Why not just fuzz the library if that's a concern?
    - nendroid 6y ago
      The effort required for this makes it not worth it. If I want a library in python it takes about a second to install with one command, and these libraries tend to be safe so you're not going to get any resistance if you want to try out 10 libraries in the next hour or so, which is sort of impossible with c++. Also these libraries are easily uninstalled and isolated from other python apps you have. No need to use massively complex package managers like nix to handle isolation. Fuzzing a library just isn't something that's done in python. The developer just installs it and manually tests it 5 or 6 times in a shell and it's part of the ecosystem in 5 minutes. If you accidentally leave unused libraries in your code and it gets pushed to prod it's a non-issue. I switched to c++ because I was ok with the verbosity and also the complexity and also dealing with memory. But literally every single corner of the c++ ecosystem is a pain in the ass to work with. It's getting to the point where I'm just annoyed all the time. If you want high performance apps go with rust or golang. C++ is just not worth it. For a C++ dev moving to python is like night and day. Imagine a programming environment where you only have to deal with the intention of what your app should do and not all the yak shaving that comes with it on the side. I know some and they tell me they would never go back to c++.
- hchz 6y ago
  Kill two birds with one stone by learning more about the structure of the Python interpreter and builtins implementations by reading the source.
- billfruit 6y ago
  Not an exact reply.. but in python dealing with binary data is very inelegant and unintuitive.. Simple things like packing and forming and reading binary packets seems very convoluted in python..
oscargrouch 6y ago
From memory, a thing i've implemented in C++ that its basically impossible in almost any other languages (with few exceptions). Represent a data table in memory where the columns are from different types, where the data is backed by a arena allocator and represented contiguously in memory. Of course you can do a data table representation in almost any language, but once you benchmark the implementations, the languages that gives you full control of memory and are good at moving bytes are the ones that lead in performance. Lars Bak once said that C++ is a great "byte chunker" (if i recall the term correctly), explaining why its a good language to create a compiler or a VM in it.
- jnxx 6y ago
  > Of course you can do a data table representation in almost any language, but once you benchmark the implementations, the languages that gives you full control of memory and are good at moving bytes are the ones that lead in performance. The thing is, once you need even more performance, you might need parallel evaluation. And in C++, there are plenty of opportunities to shoot yourself with which even very experienced programmers have sometimes extreme difficulty doing it correctly: https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL... https://preshing.com/20130930/double-checked-locking-is-fixed-in-cpp11/ https://preshing.com/20130930/double-checked-locking-is-fixe... And this is the area which Rust, while being as performant as C++, is heavily improving on.
- eru 6y ago
  Sounds like Rust should handle this case just fine?
  - oscargrouch 6y ago
    Yes, Rust is one of those languages now. (C, C++ and the newcomers Rust and Zig)
stouset 6y ago
You list some good use-cases for C++, but the other side of the coin is that there are languages out there nowadays that are better than C++ for a large majority of the use-cases where it fits best (Rust being the obvious example). Yes, there are surely still a few cases where C++ beats Rust on raw merit. And of course C++ still has the advantage in the number of developers who can competently contribute to your project in short order. But both of these advantages are diminishing rapidly, and—to me at least—the writing is on the wall for C++. Its days are numbered.
josefx 6y ago
> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. People using c++ usually care about programs finishing before the heat death of the universe. I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant.
- Barrin92 6y ago
  it depends on what you're doing of course. It can even go the other way[1] Generally languages like C++ are obviously much faster, but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster, because doing things in low level compiled languages often comes with its own tricks and problems. [1]https://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python https://stackoverflow.com/questions/9371238/why-is-reading-l...
  - barumi 6y ago
    > but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is, and Python's GIL simply eliminates that option except for multi-process applications which then require serializing stuff back and forth. Python is pretty much unparalleled as a glue language, and excels at putting together small exploratory scripts for data analysis and number crunching applications, but it makes no sense to present Python as a performant alternative to C++.
    - quietbritishjim 6y ago
      The overwhelming majority of C extensions release the Python GIL during calls to them. If you're in the situation described in the parent comment then you can parallelize just fine using threads.
      - deleted 6y ago
golergka 6y ago
Wouldn't C be a better fit for these tasks?
- berkut 6y ago
  Given people (including in these comments about string splitting) often complain about C++ not having "batteries included" for everything, surely C would be even worse in that regard...
  - golergka 6y ago
    Personally, I think that the reason for these complaints is that C++ tries to look like a "batteries included" language that is suitable for tasks that high-level languages are typically used for, and falls short. C, on the other hand, is pretty obviously low-level. C is honest in what it is and what expectations you can have from it. It won't tempt you to rewrite your whole business logic with it — instead, you would write a small library or service, that would only take care of the memory and CPU intensive task that you need it for, and leave the rest of business logic written in language that really suits it.
  - andi999 6y ago
    Actually C is more batteries included (or accessible...). Since the C ABI is stable you easily can use external libs. For c++ you need to either compile those yourself or hope they precompiled for your exact compiler version.
nendroid 6y ago
>Like every programming language out there, C++ is a tool. And like every tool out there it has its uses. And like every tool some tools are crap and other tools are good. Not saying C++ is crap but I'm definitely saying that it is unwise to use this analogy to believe that there are no crap tools out there. There's this thing that happens among programming languages where everyone likes to think all programming languages are just apples and oranges and everything is equal. The truth about the world is equality is rare... some things are good and others pure liquid shit. Programming languages are not immune from this property of the universe. Now about C++... What you say about it is true. I completely agree. However, much of the good of C++ comes with a pile of crap that you have to deal with that's often not worth it for the majority of projects. In my opinion C++ is only good because there's no alternative unless you count Rust or D.
noctune 6y ago
> * Try implementing an on-disk hash table in Python and you're stuck manually packing ints and longs in an out of arrays. In C++ it could a simple template used with a memory-mapped file. This is only simpler if you don't need to handle read failures. If you want to handle read failures, memory mapping can quickly become way more complicated. You have to install signal handlers and do a complicated dance to correctly handle not having data in some range.
- eps 6y ago
  Indeed. Memory mapped files are the bubble sort 2.0. They look simple and approachable on the surface, lead to an elegant code and seem to solve the problem in basic test cases, but in practice they have nasty scalability problems. In particular, trying to work with very large files leads to cache thrashing that affects the entire system, which makes mmaped files particularly unfit for implementing off-memory data structures. Moreover, the code has zero control over this behavior, because we ultimately over-allocate memory and outsource swapping details to the OS.
jnxx 6y ago
> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. Well, I think that Python will probably do that in the shortest code but it is also more or less the slowest language which is today widely in use, at frequently about 1/50 the speed of C. You of course could write the example in the link in Rust but for a quick script with decent performance, at around 1/6th of the speed of C, I'd probably use Racket. Racket is really a pretty sweet spot between performance and expressiveness. > C++ is a tool. And like every tool out there it has its uses. [ ... ] While you don't need it in most of the cases ... The thing is that the space in which C++ is really useful and the best solution is more and more shrinking. If one wants to control the hardware registers and write a kernel, I think C is still best. If one aims for safety and an performance equal to C, Rust is by now a good choice. For example in the task given in the original article, it would correctly process Unicode without any extra effort. It is of course complex, but much less so than modern C++. If one wants to get things done quickly, Python has its place, but I think Racket and similar languages (babashka) have many advantages. If things get more abstract, Ocaml might be an interesting choice. And I think for parallelism on the server, Clojure and Scala are pretty good. What happened is that, once upon a time, programs in C++ or C were far far faster than anything else, and Java programs were as slow as a slug. But, compilers got better, and this has changed. And also, modern software development is source-code centric, not based on compiled Windows DLLs, much of the used code is in libraries on Github and elsewhere on the net, and it turns out that for this kind of code reuse, functional languages are often a better fit. Now, C++ is still competing with this and adopting functional features, but the result is getting more and more complex. At the same time, languages like the mentioned ones are getting better and better compilers, and they are conceptually far far simpler. And the speed advantage of C++ is shrinking steadily. In fact, by some comparisons it already does not exist any more: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/racket-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/ocaml-gpp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-gpp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > fully control the memory layout of your data. This is, looked at closely, not more than a means toward an objective, and that objective is performance. Control over the memory layout is normally not a value in itself. > * Try implementing an on-disk hash table in Python and you're stuck manually packing ints and longs in an out of arrays. In C++ it could a simple template used with a memory-mapped file. This can easily be done if you use LMDB. In fact, using it might be faster and LMDB allows for safely accessing the data from multiple processes. LMDB is written in C and of course there exists a Python library for that. > * Try doing anything non-trivial on a microcontroller with 32KB of RAM. You can use uLisp on the Arduino: http://www.ulisp.com/show?1LG8 http://www.ulisp.com/show?1LG8 See also https://news.ycombinator.com/item?id=11777662 https://news.ycombinator.com/item?id=11777662 > * Try designing an application-specific data structure in any other language. Let's say you have an ~8GB in-memory database that slowly adds records one-by-one and then invalidates them in chunks. Sounds perhaps like a use case for InfluxDB, which is implemented on top of LMDB, by the way. > What it means in practice, is that unless you have an existing project that uses C++ anyway, you want to partition it: do the memory-critical part in C++, and communicate to it from a higher-level language via a high-level interface. That's of course possible and is arguable often done that way. But there is no necessity to use C++. You can do the low-level, performance-critical part which needs some control over the memory layout in Rust, and do the rest in Racket, for example. The functional nature of Racket will make parallelism in the Rust part easier. I've tried that and it is really really neat. Just to say, I've been using C and C++ professionally since, well, now 25 years. And this along with, for example, Python and IEC 61131-3 Structured Text, which is used to program PLCs. Of course, C++ is entrenched and will continue to be used for a long time. There is also heavy investment and interest in industry. But, it is not as essential as C for the hardware access part and there are today a number of interesting other options. I think C will be used for a loooong time from now, but I would not bet my house that C++ is being used as widely 20 years from now. (The issue with Python is that it is easy to get started with it, and write small personal projects in it, but for the case of programming reliably in-the-large, and using a lot of dependencies, it is showing its deficiencies).
casept 6y ago
Just because there's a tool in a niche doesn't mean it's the best one. I could do all those things you listed in Rust with much less headache.
- cute_boi 6y ago
  plus rust type system etc are just wow. And mostly if it compiles it works :)
nottorp 6y ago
Aren't you describing compiled languages with manual memory management vs interpreted languages with automatic memory management? Not merely C++ vs python or js
- eru 6y ago
  Of course, it's not inherent to a language whether it's compiled or interpreted. That's just an implementation detail. You can total write a C++ interpreter. And there are Python and JavaScript compilers. C++'s memory management isn't even necessarily particularly manual. Nowadays people use reference counting in C++ occasionally. (They call them 'smart pointers' I think.)
eru 6y ago
> [...] fully control the memory layout of your data. Is that actually guaranteed by the standard? > * Try doing anything non-trivial on a microcontroller with 32KB of RAM. You could theoretically use a higher-level language, but you will end up using >10x amount of RAM. It's a shame Forth never took off. Sometimes I wonder what it would have been like to live in an alternate universe where Forth had been the lingua franca instead of C. (Ie imagine that universe's equivalent of Unix being implemented in Forth as its killer application and going from there.)
- andi999 6y ago
  What is so good about forth?
  - eru 6y ago
    It's rather simple, can easily run on bare metal (ie no OS). You can go very low level, but it has better higher level abstraction capabilities than C. It's also really weird, which fascinates me. Eg most Forth code doesn't use variable names. I've toyed around with Forth a bit. But never used it in anger. Wikipedia says that Forth is used today in some bootloaders and outer space. I suspect the latter is a historic accident, because the inventor of Forth worked on a big telescope. See https://en.wikipedia.org/wiki/Forth_(programming_language) https://en.wikipedia.org/wiki/Forth_(programming_language)
    - andi999 6y ago
      Ahh. I remember. I always get fascinated first, but when I see 2 3 4 + * meaning (2+3)*4 my eyes glaze over.
      - eru 6y ago
        Thanks to a good amount of time spent with Lisp, I don't mind neither post-fix nor prefix notation that much.
- im3w1l 6y ago
d0mine 6y ago
fyi, you can read mmap files in Python too: import mmap # Unix, Windows with open('c_structs.bin', 'rb') as file, mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_COPY)) as buf: result = (YourStruct * 3).from_buffer(buf) # without copying https://stackoverflow.com/questions/17244488/reading-struct-in-python-from-created-struct-in-c https://stackoverflow.com/questions/17244488/reading-struct-...

wirthjason 6y ago

> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. However, there's one thing you can do in C++ and not in Python or JavaScript or PHP: fully control the memory layout of your data. Any books or resources for digging deeper in these kinds of topics? I’m a Python developer and would like to learn C++ but the just the reasons you mentioned make it hard to leave Python.

danudey 6y ago
you can malloc() to get a memory range, or mmap() to get a disk-backed memory range with kernel-managed paging, and then you can do whatever you want with it. You get a pointer to the start of the range, and you know how large it is, so from there it's just pointer arithmetic and assignment. Easy peasy. Now you just have to make sure that you zero data when you're done with it, track how much of that memory is available or used (and which sections), and (optionally) implement your own defragmentation and garbage collection. Unless you desperately need to specifically manage the memory layout of your data (e.g. for alignment purposes for massive data processing), none of this is worth the time or trouble to do for most projects, not only because it's a huge amount of work for little gain, but because you'll almost definitely screw it up and potentially corrupt your entire data set. Note that, at least for the on-disk hashmap set, you absolutely can do this in python, so if you're interested in experimenting with this sort of data management, practice it there first. A good place to start: https://github.com/luispedro/diskhash https://github.com/luispedro/diskhash
- barumi 6y ago
  > you can malloc() to get a memory range, or mmap() to get a disk-backed memory range with kernel-managed paging, and then you can do whatever you want with it. No need for that, as C++ provides placement new(). https://en.cppreference.com/w/cpp/language/new https://en.cppreference.com/w/cpp/language/new
  - saagarjha 6y ago
    This doesn’t really solve the same problem, even though the two often go hand-in-hand.
- gpderetta 6y ago
  It is really not that much work and there are plenty of libs that can do it for you if you do not want to.
john_moscow 6y ago
To get a quick start, try declaring some class variables and then open the Memory window and go to "&variable_name". You can also evaluate "sizeof(variable_name)" to get its exact size. You'll see a bunch of bytes reflecting how exactly your variable is seen by the CPU. That's exactly how many RAM bytes it occupies. It can quickly copied to a new location, limited only by your RAM bandwidth. It can be dumped to a disk and loaded later, limited only by your disk bandwidth.
- saagarjha 6y ago
  How does this help?
nendroid 6y ago
Python developer here just switched jobs to one that does C++. Let me warn you right now. It's not worth it. At all. Don't go down this path. Memory management is easy, but that's not the only crap you have to deal with. Templating and the whole C++ ecosystem with Cmake compiling to make is a giant headache. It takes about a week to integrate a new C++ library in source code and people are worried about installing or "trying" out new libraries because of memory leaks.... while in python I'll install an entire library in seconds just to try out a function. I would say if you made an app in python and the same exact app in C++, the C++ app would perform waay better. No question. But the developer experience and effort required to make that same app is much much more painful.
- rightbyte 6y ago
  It just sounds like you are working on a big messy project.
  - nendroid 6y ago
    Nah it's a big unicorn startup down in Irvine, so the codebase is still relatively new. They use C++17 so pretty modern as well.
- winrid 6y ago
  Why not just fuzz the library if that's a concern?
  - nendroid 6y ago
    The effort required for this makes it not worth it. If I want a library in python it takes about a second to install with one command, and these libraries tend to be safe so you're not going to get any resistance if you want to try out 10 libraries in the next hour or so, which is sort of impossible with c++. Also these libraries are easily uninstalled and isolated from other python apps you have. No need to use massively complex package managers like nix to handle isolation. Fuzzing a library just isn't something that's done in python. The developer just installs it and manually tests it 5 or 6 times in a shell and it's part of the ecosystem in 5 minutes. If you accidentally leave unused libraries in your code and it gets pushed to prod it's a non-issue. I switched to c++ because I was ok with the verbosity and also the complexity and also dealing with memory. But literally every single corner of the c++ ecosystem is a pain in the ass to work with. It's getting to the point where I'm just annoyed all the time. If you want high performance apps go with rust or golang. C++ is just not worth it. For a C++ dev moving to python is like night and day. Imagine a programming environment where you only have to deal with the intention of what your app should do and not all the yak shaving that comes with it on the side. I know some and they tell me they would never go back to c++.
hchz 6y ago
Kill two birds with one stone by learning more about the structure of the Python interpreter and builtins implementations by reading the source.
billfruit 6y ago
Not an exact reply.. but in python dealing with binary data is very inelegant and unintuitive.. Simple things like packing and forming and reading binary packets seems very convoluted in python..

oscargrouch 6y ago

From memory, a thing i've implemented in C++ that its basically impossible in almost any other languages (with few exceptions). Represent a data table in memory where the columns are from different types, where the data is backed by a arena allocator and represented contiguously in memory. Of course you can do a data table representation in almost any language, but once you benchmark the implementations, the languages that gives you full control of memory and are good at moving bytes are the ones that lead in performance. Lars Bak once said that C++ is a great "byte chunker" (if i recall the term correctly), explaining why its a good language to create a compiler or a VM in it.

jnxx 6y ago
> Of course you can do a data table representation in almost any language, but once you benchmark the implementations, the languages that gives you full control of memory and are good at moving bytes are the ones that lead in performance. The thing is, once you need even more performance, you might need parallel evaluation. And in C++, there are plenty of opportunities to shoot yourself with which even very experienced programmers have sometimes extreme difficulty doing it correctly: https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL... https://preshing.com/20130930/double-checked-locking-is-fixed-in-cpp11/ https://preshing.com/20130930/double-checked-locking-is-fixe... And this is the area which Rust, while being as performant as C++, is heavily improving on.
eru 6y ago
Sounds like Rust should handle this case just fine?
- oscargrouch 6y ago
  Yes, Rust is one of those languages now. (C, C++ and the newcomers Rust and Zig)

stouset 6y ago

You list some good use-cases for C++, but the other side of the coin is that there are languages out there nowadays that are better than C++ for a large majority of the use-cases where it fits best (Rust being the obvious example). Yes, there are surely still a few cases where C++ beats Rust on raw merit. And of course C++ still has the advantage in the number of developers who can competently contribute to your project in short order. But both of these advantages are diminishing rapidly, and—to me at least—the writing is on the wall for C++. Its days are numbered.

josefx 6y ago

> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. People using c++ usually care about programs finishing before the heat death of the universe. I had a few Python scripts that spend several seconds parsing larger files, after a rewrite to c++ that changed to almost instant.

Barrin92 6y ago
it depends on what you're doing of course. It can even go the other way[1] Generally languages like C++ are obviously much faster, but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster, because doing things in low level compiled languages often comes with its own tricks and problems. [1]https://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python https://stackoverflow.com/questions/9371238/why-is-reading-l...
- barumi 6y ago
  > but a lot of primitive stuff in languages like python is also just C calls and may be optimised to the point of being faster That assertion makes zero sense if we take into account that parallelization is one of the most basic performance techniques there is, and Python's GIL simply eliminates that option except for multi-process applications which then require serializing stuff back and forth. Python is pretty much unparalleled as a glue language, and excels at putting together small exploratory scripts for data analysis and number crunching applications, but it makes no sense to present Python as a performant alternative to C++.
  - quietbritishjim 6y ago
    The overwhelming majority of C extensions release the Python GIL during calls to them. If you're in the situation described in the parent comment then you can parallelize just fine using threads.
    - deleted 6y ago

golergka 6y ago

Wouldn't C be a better fit for these tasks?

berkut 6y ago
Given people (including in these comments about string splitting) often complain about C++ not having "batteries included" for everything, surely C would be even worse in that regard...
- golergka 6y ago
  Personally, I think that the reason for these complaints is that C++ tries to look like a "batteries included" language that is suitable for tasks that high-level languages are typically used for, and falls short. C, on the other hand, is pretty obviously low-level. C is honest in what it is and what expectations you can have from it. It won't tempt you to rewrite your whole business logic with it — instead, you would write a small library or service, that would only take care of the memory and CPU intensive task that you need it for, and leave the rest of business logic written in language that really suits it.
- andi999 6y ago
  Actually C is more batteries included (or accessible...). Since the C ABI is stable you easily can use external libs. For c++ you need to either compile those yourself or hope they precompiled for your exact compiler version.

nendroid 6y ago

>Like every programming language out there, C++ is a tool. And like every tool out there it has its uses. And like every tool some tools are crap and other tools are good. Not saying C++ is crap but I'm definitely saying that it is unwise to use this analogy to believe that there are no crap tools out there. There's this thing that happens among programming languages where everyone likes to think all programming languages are just apples and oranges and everything is equal. The truth about the world is equality is rare... some things are good and others pure liquid shit. Programming languages are not immune from this property of the universe. Now about C++... What you say about it is true. I completely agree. However, much of the good of C++ comes with a pile of crap that you have to deal with that's often not worth it for the majority of projects. In my opinion C++ is only good because there's no alternative unless you count Rust or D.

noctune 6y ago

> * Try implementing an on-disk hash table in Python and you're stuck manually packing ints and longs in an out of arrays. In C++ it could a simple template used with a memory-mapped file. This is only simpler if you don't need to handle read failures. If you want to handle read failures, memory mapping can quickly become way more complicated. You have to install signal handlers and do a complicated dance to correctly handle not having data in some range.

eps 6y ago
Indeed. Memory mapped files are the bubble sort 2.0. They look simple and approachable on the surface, lead to an elegant code and seem to solve the problem in basic test cases, but in practice they have nasty scalability problems. In particular, trying to work with very large files leads to cache thrashing that affects the entire system, which makes mmaped files particularly unfit for implementing off-memory data structures. Moreover, the code has zero control over this behavior, because we ultimately over-allocate memory and outsource swapping details to the OS.

jnxx 6y ago

> There isn't any point of using C++ to count words in a text file. Any high-level language like Python will beat you to it. Well, I think that Python will probably do that in the shortest code but it is also more or less the slowest language which is today widely in use, at frequently about 1/50 the speed of C. You of course could write the example in the link in Rust but for a quick script with decent performance, at around 1/6th of the speed of C, I'd probably use Racket. Racket is really a pretty sweet spot between performance and expressiveness. > C++ is a tool. And like every tool out there it has its uses. [ ... ] While you don't need it in most of the cases ... The thing is that the space in which C++ is really useful and the best solution is more and more shrinking. If one wants to control the hardware registers and write a kernel, I think C is still best. If one aims for safety and an performance equal to C, Rust is by now a good choice. For example in the task given in the original article, it would correctly process Unicode without any extra effort. It is of course complex, but much less so than modern C++. If one wants to get things done quickly, Python has its place, but I think Racket and similar languages (babashka) have many advantages. If things get more abstract, Ocaml might be an interesting choice. And I think for parallelism on the server, Clojure and Scala are pretty good. What happened is that, once upon a time, programs in C++ or C were far far faster than anything else, and Java programs were as slow as a slug. But, compilers got better, and this has changed. And also, modern software development is source-code centric, not based on compiled Windows DLLs, much of the used code is in libraries on Github and elsewhere on the net, and it turns out that for this kind of code reuse, functional languages are often a better fit. Now, C++ is still competing with this and adopting functional features, but the result is getting more and more complex. At the same time, languages like the mentioned ones are getting better and better compilers, and they are conceptually far far simpler. And the speed advantage of C++ is shrinking steadily. In fact, by some comparisons it already does not exist any more: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/racket-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/ocaml-gpp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-gpp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > fully control the memory layout of your data. This is, looked at closely, not more than a means toward an objective, and that objective is performance. Control over the memory layout is normally not a value in itself. > * Try implementing an on-disk hash table in Python and you're stuck manually packing ints and longs in an out of arrays. In C++ it could a simple template used with a memory-mapped file. This can easily be done if you use LMDB. In fact, using it might be faster and LMDB allows for safely accessing the data from multiple processes. LMDB is written in C and of course there exists a Python library for that. > * Try doing anything non-trivial on a microcontroller with 32KB of RAM. You can use uLisp on the Arduino: http://www.ulisp.com/show?1LG8 http://www.ulisp.com/show?1LG8 See also https://news.ycombinator.com/item?id=11777662 https://news.ycombinator.com/item?id=11777662 > * Try designing an application-specific data structure in any other language. Let's say you have an ~8GB in-memory database that slowly adds records one-by-one and then invalidates them in chunks. Sounds perhaps like a use case for InfluxDB, which is implemented on top of LMDB, by the way. > What it means in practice, is that unless you have an existing project that uses C++ anyway, you want to partition it: do the memory-critical part in C++, and communicate to it from a higher-level language via a high-level interface. That's of course possible and is arguable often done that way. But there is no necessity to use C++. You can do the low-level, performance-critical part which needs some control over the memory layout in Rust, and do the rest in Racket, for example. The functional nature of Racket will make parallelism in the Rust part easier. I've tried that and it is really really neat. Just to say, I've been using C and C++ professionally since, well, now 25 years. And this along with, for example, Python and IEC 61131-3 Structured Text, which is used to program PLCs. Of course, C++ is entrenched and will continue to be used for a long time. There is also heavy investment and interest in industry. But, it is not as essential as C for the hardware access part and there are today a number of interesting other options. I think C will be used for a loooong time from now, but I would not bet my house that C++ is being used as widely 20 years from now. (The issue with Python is that it is easy to get started with it, and write small personal projects in it, but for the case of programming reliably in-the-large, and using a lot of dependencies, it is showing its deficiencies).

casept 6y ago

Just because there's a tool in a niche doesn't mean it's the best one. I could do all those things you listed in Rust with much less headache.

cute_boi 6y ago
plus rust type system etc are just wow. And mostly if it compiles it works :)

nottorp 6y ago

Aren't you describing compiled languages with manual memory management vs interpreted languages with automatic memory management? Not merely C++ vs python or js

eru 6y ago
Of course, it's not inherent to a language whether it's compiled or interpreted. That's just an implementation detail. You can total write a C++ interpreter. And there are Python and JavaScript compilers. C++'s memory management isn't even necessarily particularly manual. Nowadays people use reference counting in C++ occasionally. (They call them 'smart pointers' I think.)

eru 6y ago

> [...] fully control the memory layout of your data. Is that actually guaranteed by the standard? > * Try doing anything non-trivial on a microcontroller with 32KB of RAM. You could theoretically use a higher-level language, but you will end up using >10x amount of RAM. It's a shame Forth never took off. Sometimes I wonder what it would have been like to live in an alternate universe where Forth had been the lingua franca instead of C. (Ie imagine that universe's equivalent of Unix being implemented in Forth as its killer application and going from there.)

andi999 6y ago
What is so good about forth?
- eru 6y ago
  It's rather simple, can easily run on bare metal (ie no OS). You can go very low level, but it has better higher level abstraction capabilities than C. It's also really weird, which fascinates me. Eg most Forth code doesn't use variable names. I've toyed around with Forth a bit. But never used it in anger. Wikipedia says that Forth is used today in some bootloaders and outer space. I suspect the latter is a historic accident, because the inventor of Forth worked on a big telescope. See https://en.wikipedia.org/wiki/Forth_(programming_language) https://en.wikipedia.org/wiki/Forth_(programming_language)
  - andi999 6y ago
    Ahh. I remember. I always get fascinated first, but when I see 2 3 4 + * meaning (2+3)*4 my eyes glaze over.
    - eru 6y ago
      Thanks to a good amount of time spent with Lisp, I don't mind neither post-fix nor prefix notation that much.
im3w1l 6y ago

d0mine 6y ago

fyi, you can read mmap files in Python too: import mmap # Unix, Windows with open('c_structs.bin', 'rb') as file, mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_COPY)) as buf: result = (YourStruct * 3).from_buffer(buf) # without copying https://stackoverflow.com/questions/17244488/reading-struct-in-python-from-created-struct-in-c https://stackoverflow.com/questions/17244488/reading-struct-...