5 ms·

As someone who doesn't write C++, why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Both C++ and C# take a "bags in

As someone who doesn't write C++, why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Both C++ and C# take a "bags included" approach to the standard library, but no one is writing their own `List<T>` for C#.[a] Yet, everyone seems to have their own opinion about `std::vector<T>`. [a]: Obviously, people still write their own collections/containers in C#, but they tend to only do so for very specific/performance-sensitive circumstances.

deleted 3y ago
[deleted]
blovescoffee 3y ago
Because people tend to write c++ for performance reasons and the perf profiles of std::vector are not always sufficient
- jeffbee 3y ago
  The only thing you can really quibble about with std::vector is whether your library has made an optimal choice of growth strategy, which you can often hack around by reserving. Aside from that, access via `operator[]` and growth via `emplace_back` will compile down to optimal code that is going to be close to impossible to beat. After the compiler gets done with it, it looks the same as if you had hand-coded it with arrays in a C-style, but without the plethora of bugs that often results from that approach.
  - jsheard 3y ago
    There's also the issue that std::vector<bool> is required by the standard to be specialized as a bitset, which is a footgun in generic code since you can normally take the address of a vector element but not if it's a vector of bool. Having a bitset in the standard library is fine but it should have been a seperate type. Admittedly that's not a performance issue, but it's annoying.
    - jeffbee 3y ago
      That's true but I think everyone knows about vector<bool> being quirky. By the way, the standard does not require vector<bool> to be implemented as a bitset. Instead, it relaxes some of the details of vector, in a way that allows the implementation to do it that way. But these choices are implementation details. Vector<bool> is a little weird if you are just starting with C++, but it does have major performance benefits in its niche, and it came from the 1990s so we can be generous in overlooking its rough edges.
      - HelloNurse 3y ago
        It came from the 1990s, but it overstayed its welcome. I remember seeing proposals to remove the vector<bool> special case, what is the situation?
  - dataflow 3y ago
    > The only thing you can really quibble about with std::vector is [...] That's actually not true, though I certainly don't fault you for believing it :-) but there are definitely more things to quibble about around vector if you're serious about performance. As an example, try writing a can_fit(n) function, which tells you whether the vector can fit n elements without reallocating. Observe the performance difference between (a) a smart manual version, (b) a naive manual version, and (c) the only STL version: https://godbolt.org/z/88sfM1sxW https://godbolt.org/z/88sfM1sxW #include <vector> template<class T> struct Vec { T *b, *e, *f; }; template<class T> bool can_fit_fast(Vec<T> const &v, size_t n) { return reinterpret_cast<char*>(v.f) - reinterpret_cast<char*>(v.e) >= n * sizeof(T); } template<class T> bool can_fit(Vec<T> const &v, size_t n) { return v.f - v.e >= n; } template<class T> bool can_fit(std::vector<T> const &v, size_t n) { return v.capacity() - v.size() >= n; } struct S { size_t a[3]; }; template bool can_fit_fast(Vec<S> const &, size_t); template bool can_fit(Vec<S> const &, size_t); template bool can_fit(std::vector<S> const &, size_t);
    - CyberDildonics 3y ago
      Why would that be faster? Those function calls are going to be inlined.
      - jeffbee 3y ago
        It's not the function calls they are alluding to, it's the way the compiler generates a bunch of shifts and multiplies except in the can_fit_2 case.
        dataflow 3y ago
        Turns out GCC is more clever than Clang here, and MSVC is just horrendous. (See update, I posted a link.)
        jeffbee 3y ago
        I somewhat agree with your point (esp. that MSVC is hideous) but I also stand by mine. I don't feel that checking the capacity is something that would be in my tight loop, because checking for capacity to hold N is something you do before adding N items, which amortizes the cost, meaning the capacity check is just off the real hot path. So it doesn't feel like a realistic use case. Speaking of realism, putting these in quickbench seems to confirm that the differences between them are not material, and that the STL version is in fact the quickest, but they are all essentially free. There's not a way to make a realistic microbenchmark for this, for the same reason that it doesn't feel like a real-world performance issue. By the way clang does a much better job here: https://quick-bench.com/q/XcKK782d-7A6YHbiBRTlOnIRnPY https://quick-bench.com/q/XcKK782d-7A6YHbiBRTlOnIRnPY
- CyberDildonics 3y ago
  What specifically is wrong with vector? There have been a lot of hash maps done with flat memory to minimize allocations and pointer hopping over the STL but vector doesn't have those problems.
sfpotter 3y ago
There are so many reasons. I'm sure others will provide their own favorites. One that I find particularly annoying lately: if you work on a project that makes heavy use of the STL (or, really, any heavily templated library written in a similar style, with a focus on "ergonomics" :-( ), you'll quickly find that backtraces for debug builds can easily be 50-100 levels deep with most stack frames just consisting of incomprehensible layers of abstraction which get optimized away. So, debug builds are totally useless, and build times are typically long. Contrast that with something written in "C style" or simply making far fewer use of C++ features, written in a more straightline style, with fewer levels of abstraction: builds will be fast, stack traces will be small and can easily map directly onto the concepts relevant to the program or library, and debug builds are useful once more. Night and day difference.
- jeffbee 3y ago
  In decades of professional C++ experience I have never noted depth of backtrace as a usability issue. Give us an example of how a 100-deep backtrace arises as a consequence of the STL.
  - sfpotter 3y ago
    "Functional" C++ with deeply nested loops implemented using stuff from <algorithm> and <functional> and lambda.
  - o11c 3y ago
    Before concepts it was certainly possible, but when it happened you just had to add some extra type assertions to make it clear where the error is.
  - summerlight 3y ago
    Agreed, at least for STL containers I didn't really have to dig into the implementation for 99.9% of the cases. One or two specific old cases were due to memory corruption, but nowadays address sanitizer should do the job. Of course, a lengthy symbol for heavily templatized functions/classes can sometimes be a problem, but it's a separate issue from stack trace depth.
jsheard 3y ago
Take your pick: undercooked functionality, lack of behavior/performance consistency between different toolchains, bloated implementations causing long compile times, inscrutable template vomit error messages and poor performance in debug builds (a dealbreaker for realtime apps like games), and design mistakes which can't easily be fixed, such as std::vector<bool> being a footgun or the way that the standard map types are defined making them impossible to implement efficiently.
neonsunset 3y ago
Because unlike CoreLib, the standard library in C++ tends to have a lot of insanity backed in. My knowledge on it is somewhat limited but looking into its APIs for working with strings (in large inherited from C) was so traumatizing that I only ever wonder why there are still self-respecting projects that don't reimplement parts of STL. Example: want to measure the length of a UTF-8 code point in a string and did not synchronize the call? Well, too bad, now you might have corrupted the global mutable state it relies on! (the fact that you can make such a trivial piece of code have two!!! points of thread-safety failure one is C locale and another is transcoder still refuses to leave my mind)
- neonsunset 3y ago
  Can't believe someone actually went out of their way to downvote this haha
  - Chaosvex 3y ago
    Sounds like you weren't working with C++'s string API. std::string knows nothing about the encoding of the data it encapsulates and can be used to store anything. There are the codecvt functions but it doesn't sound like you're talking about those, either.
lenkite 3y ago
Because C#, at several points in its history, sensibly took the decision to explicitly break backward compatibility to satisfy legitimate requirements, adapting and evolving to the tenets of modern software practices. (Generics and new collections) Whereas due to C++'s "never break compatibility" decision, the standard library has progressively decayed over time. It has become a bloated, rotting dinosaur where even the slowest of interpreted languages can comfortably beat several of its aspects. (Ex: std::regex is pathetic and pitiful, vector<bool> triggers laughter, substandard maps, etc). Considering that C++ thumps its chest and loudly proclaims its superb performance, this has now become a sad joke. In the natural world, a species that cannot adapt to new circumstances and never discards undesirable characteristics simply perishes. The C++ Standard Committee has firmly and unequivocally decided that the C++ language should mirror the same approach and limp down the road, carrying the full-weight of its sins for all its journey, until it falls into oblivion.
- pjmlp 3y ago
  Note that as documented in the history of F# HOPL paper, .NET already had a generics prototype before 1.0 release, Microsoft decided to go ahead without generics to avoid delaying it further until they were stable. Also breaking everything a couple of times is why most corporations are nowadays stuck in a Python 2 / 3 parallel world in .NET Framework / .NET Core , or in Xamarin.Forms / MAUI, UWP / WinUI,...
jandrewrogers 3y ago
The containers in the STL have issues due to the limitations of C++ when they were designed, catering to the lowest common denominator user, and simplifying things for implementors. It is not uncommon to find cases where the STL is a poor fit or annoying to use due to its implementation and design details. In the case of modern C++, you also have the issue that the STL containers aren't designed to be used in an advanced metaprogramming context, because they pre-date that being an intentional part of C++. Some of this is unfixable because backward compatibility. Also, C++ is commonly used in contexts where performance is critical, so the optimality of the STL for purpose matters more. Some of the common issues: static allocation or lack thereof, requiring default constructible classes, initializing memory you are going to overwrite anyway, inability to be used in some metaprogramming contexts, suboptimal allocation behavior, etc. The STL is opinionated but unfortunately that opinion dates to a time when C++ was primarily used for ordinary app development, not high-performance code. Like many, I maintain my own C++ "standard library" that is much better designed for the kinds of software I tend to work on (database kernels and data infrastructure, mostly).
MichaelMoser123 3y ago
Doesn't it contradicts the 'bloat free' requirement, if every library needs to have its own version of std::list std::vector, std::unordered_map etc ? I mean: i remember the days, when each and every project had its own string class.
- MathMonkeyMan 3y ago
  The implementations of vector, list, and unordered_map in every standard library I've used have been sufficient for every C++ library I've worked on. There are special cases, and there are engineering politics, but it's all basically fine.
- pagghiu 3y ago
  The bloat free is more referring to executable size, build complexity, compile time and in general to hidden complexity. Every library having its own version of common data structure is unfortunately something that C++ programmers can't really seem to agree on :)
summerlight 3y ago
> why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Not really. Unless you have a very specific set of performance requirements, STL containers are usually more than enough. And we have third party libraries such as Abseil/Boost to cover the major gaps in the rest. I do see some legitimate cases to write own container libraries. But for many cases people don't really measure their primary workload before writing such libraries, instead they just write it because they can write (and it's fun).

deleted 3y ago

[deleted]

blovescoffee 3y ago

Because people tend to write c++ for performance reasons and the perf profiles of std::vector are not always sufficient

jeffbee 3y ago
The only thing you can really quibble about with std::vector is whether your library has made an optimal choice of growth strategy, which you can often hack around by reserving. Aside from that, access via `operator[]` and growth via `emplace_back` will compile down to optimal code that is going to be close to impossible to beat. After the compiler gets done with it, it looks the same as if you had hand-coded it with arrays in a C-style, but without the plethora of bugs that often results from that approach.
- jsheard 3y ago
  There's also the issue that std::vector<bool> is required by the standard to be specialized as a bitset, which is a footgun in generic code since you can normally take the address of a vector element but not if it's a vector of bool. Having a bitset in the standard library is fine but it should have been a seperate type. Admittedly that's not a performance issue, but it's annoying.
  - jeffbee 3y ago
    That's true but I think everyone knows about vector<bool> being quirky. By the way, the standard does not require vector<bool> to be implemented as a bitset. Instead, it relaxes some of the details of vector, in a way that allows the implementation to do it that way. But these choices are implementation details. Vector<bool> is a little weird if you are just starting with C++, but it does have major performance benefits in its niche, and it came from the 1990s so we can be generous in overlooking its rough edges.
    - HelloNurse 3y ago
      It came from the 1990s, but it overstayed its welcome. I remember seeing proposals to remove the vector<bool> special case, what is the situation?
- dataflow 3y ago
  > The only thing you can really quibble about with std::vector is [...] That's actually not true, though I certainly don't fault you for believing it :-) but there are definitely more things to quibble about around vector if you're serious about performance. As an example, try writing a can_fit(n) function, which tells you whether the vector can fit n elements without reallocating. Observe the performance difference between (a) a smart manual version, (b) a naive manual version, and (c) the only STL version: https://godbolt.org/z/88sfM1sxW https://godbolt.org/z/88sfM1sxW #include <vector> template<class T> struct Vec { T *b, *e, *f; }; template<class T> bool can_fit_fast(Vec<T> const &v, size_t n) { return reinterpret_cast<char*>(v.f) - reinterpret_cast<char*>(v.e) >= n * sizeof(T); } template<class T> bool can_fit(Vec<T> const &v, size_t n) { return v.f - v.e >= n; } template<class T> bool can_fit(std::vector<T> const &v, size_t n) { return v.capacity() - v.size() >= n; } struct S { size_t a[3]; }; template bool can_fit_fast(Vec<S> const &, size_t); template bool can_fit(Vec<S> const &, size_t); template bool can_fit(std::vector<S> const &, size_t);
  - CyberDildonics 3y ago
    Why would that be faster? Those function calls are going to be inlined.
    - jeffbee 3y ago
      It's not the function calls they are alluding to, it's the way the compiler generates a bunch of shifts and multiplies except in the can_fit_2 case.
      - dataflow 3y ago
        Turns out GCC is more clever than Clang here, and MSVC is just horrendous. (See update, I posted a link.)
        jeffbee 3y ago
        I somewhat agree with your point (esp. that MSVC is hideous) but I also stand by mine. I don't feel that checking the capacity is something that would be in my tight loop, because checking for capacity to hold N is something you do before adding N items, which amortizes the cost, meaning the capacity check is just off the real hot path. So it doesn't feel like a realistic use case. Speaking of realism, putting these in quickbench seems to confirm that the differences between them are not material, and that the STL version is in fact the quickest, but they are all essentially free. There's not a way to make a realistic microbenchmark for this, for the same reason that it doesn't feel like a real-world performance issue. By the way clang does a much better job here: https://quick-bench.com/q/XcKK782d-7A6YHbiBRTlOnIRnPY https://quick-bench.com/q/XcKK782d-7A6YHbiBRTlOnIRnPY
CyberDildonics 3y ago
What specifically is wrong with vector? There have been a lot of hash maps done with flat memory to minimize allocations and pointer hopping over the STL but vector doesn't have those problems.

sfpotter 3y ago

There are so many reasons. I'm sure others will provide their own favorites. One that I find particularly annoying lately: if you work on a project that makes heavy use of the STL (or, really, any heavily templated library written in a similar style, with a focus on "ergonomics" :-( ), you'll quickly find that backtraces for debug builds can easily be 50-100 levels deep with most stack frames just consisting of incomprehensible layers of abstraction which get optimized away. So, debug builds are totally useless, and build times are typically long. Contrast that with something written in "C style" or simply making far fewer use of C++ features, written in a more straightline style, with fewer levels of abstraction: builds will be fast, stack traces will be small and can easily map directly onto the concepts relevant to the program or library, and debug builds are useful once more. Night and day difference.

jeffbee 3y ago
In decades of professional C++ experience I have never noted depth of backtrace as a usability issue. Give us an example of how a 100-deep backtrace arises as a consequence of the STL.
- sfpotter 3y ago
  "Functional" C++ with deeply nested loops implemented using stuff from <algorithm> and <functional> and lambda.
- o11c 3y ago
  Before concepts it was certainly possible, but when it happened you just had to add some extra type assertions to make it clear where the error is.
- summerlight 3y ago
  Agreed, at least for STL containers I didn't really have to dig into the implementation for 99.9% of the cases. One or two specific old cases were due to memory corruption, but nowadays address sanitizer should do the job. Of course, a lengthy symbol for heavily templatized functions/classes can sometimes be a problem, but it's a separate issue from stack trace depth.

jsheard 3y ago

Take your pick: undercooked functionality, lack of behavior/performance consistency between different toolchains, bloated implementations causing long compile times, inscrutable template vomit error messages and poor performance in debug builds (a dealbreaker for realtime apps like games), and design mistakes which can't easily be fixed, such as std::vector<bool> being a footgun or the way that the standard map types are defined making them impossible to implement efficiently.

neonsunset 3y ago

Because unlike CoreLib, the standard library in C++ tends to have a lot of insanity backed in. My knowledge on it is somewhat limited but looking into its APIs for working with strings (in large inherited from C) was so traumatizing that I only ever wonder why there are still self-respecting projects that don't reimplement parts of STL. Example: want to measure the length of a UTF-8 code point in a string and did not synchronize the call? Well, too bad, now you might have corrupted the global mutable state it relies on! (the fact that you can make such a trivial piece of code have two!!! points of thread-safety failure one is C locale and another is transcoder still refuses to leave my mind)

neonsunset 3y ago
Can't believe someone actually went out of their way to downvote this haha
- Chaosvex 3y ago
  Sounds like you weren't working with C++'s string API. std::string knows nothing about the encoding of the data it encapsulates and can be used to store anything. There are the codecvt functions but it doesn't sound like you're talking about those, either.

lenkite 3y ago

Because C#, at several points in its history, sensibly took the decision to explicitly break backward compatibility to satisfy legitimate requirements, adapting and evolving to the tenets of modern software practices. (Generics and new collections) Whereas due to C++'s "never break compatibility" decision, the standard library has progressively decayed over time. It has become a bloated, rotting dinosaur where even the slowest of interpreted languages can comfortably beat several of its aspects. (Ex: std::regex is pathetic and pitiful, vector<bool> triggers laughter, substandard maps, etc). Considering that C++ thumps its chest and loudly proclaims its superb performance, this has now become a sad joke. In the natural world, a species that cannot adapt to new circumstances and never discards undesirable characteristics simply perishes. The C++ Standard Committee has firmly and unequivocally decided that the C++ language should mirror the same approach and limp down the road, carrying the full-weight of its sins for all its journey, until it falls into oblivion.

pjmlp 3y ago
Note that as documented in the history of F# HOPL paper, .NET already had a generics prototype before 1.0 release, Microsoft decided to go ahead without generics to avoid delaying it further until they were stable. Also breaking everything a couple of times is why most corporations are nowadays stuck in a Python 2 / 3 parallel world in .NET Framework / .NET Core , or in Xamarin.Forms / MAUI, UWP / WinUI,...

jandrewrogers 3y ago

The containers in the STL have issues due to the limitations of C++ when they were designed, catering to the lowest common denominator user, and simplifying things for implementors. It is not uncommon to find cases where the STL is a poor fit or annoying to use due to its implementation and design details. In the case of modern C++, you also have the issue that the STL containers aren't designed to be used in an advanced metaprogramming context, because they pre-date that being an intentional part of C++. Some of this is unfixable because backward compatibility. Also, C++ is commonly used in contexts where performance is critical, so the optimality of the STL for purpose matters more. Some of the common issues: static allocation or lack thereof, requiring default constructible classes, initializing memory you are going to overwrite anyway, inability to be used in some metaprogramming contexts, suboptimal allocation behavior, etc. The STL is opinionated but unfortunately that opinion dates to a time when C++ was primarily used for ordinary app development, not high-performance code. Like many, I maintain my own C++ "standard library" that is much better designed for the kinds of software I tend to work on (database kernels and data infrastructure, mostly).

MichaelMoser123 3y ago

Doesn't it contradicts the 'bloat free' requirement, if every library needs to have its own version of std::list std::vector, std::unordered_map etc ? I mean: i remember the days, when each and every project had its own string class.

MathMonkeyMan 3y ago
The implementations of vector, list, and unordered_map in every standard library I've used have been sufficient for every C++ library I've worked on. There are special cases, and there are engineering politics, but it's all basically fine.
pagghiu 3y ago
The bloat free is more referring to executable size, build complexity, compile time and in general to hidden complexity. Every library having its own version of common data structure is unfortunately something that C++ programmers can't really seem to agree on :)

summerlight 3y ago

> why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Not really. Unless you have a very specific set of performance requirements, STL containers are usually more than enough. And we have third party libraries such as Abseil/Boost to cover the major gaps in the rest. I do see some legitimate cases to write own container libraries. But for many cases people don't really measure their primary workload before writing such libraries, instead they just write it because they can write (and it's fun).