4 ms·

Of course. What I meant is that if I am writing bigtask myself, then preallocating resources for any non-table-like data structure (like map) means I cannot use

by 112233 9y ago

Of course. What I meant is that if I am writing bigtask myself, then preallocating resources for any non-table-like data structure (like map) means I cannot use STL as it is now, even with custom allocators. I have to write custom data structures, or deal with any allocation potentially erroring out.

jcelerier 9y ago
> I cannot use STL as it is now, even with custom allocators. if you are ready to take the runtime hit you could use the polymorphic allocator support and pass your allocator object down the chain. See http://en.cppreference.com/w/cpp/memory/polymorphic_allocator http://en.cppreference.com/w/cpp/memory/polymorphic_allocato... ; http://en.cppreference.com/w/cpp/container/vector http://en.cppreference.com/w/cpp/container/vector ; http://en.cppreference.com/w/cpp/container/map http://en.cppreference.com/w/cpp/container/map . I don't understand why you are making a distinction between table-like and non-table-like containers: both can take allocators. But in general, I don't really understand why you would want to do this: how do you expect your code to differentiate between the containers that you want to be affected by the memory changes and the ones that you don't want to ? If you don't want to make this difference and have a blanket coverage of, say, every std::vector called by your function it means that you cannot use any external library in your function, since other libraries may have other allocation requirements that your code would break. This looks like this would completely break encapsulation. The cleanest thing to do is to do like RapidJSON for instance and just pass allocator objects in your functions : http://rapidjson.org/md_doc_tutorial.html#MoveSemantics http://rapidjson.org/md_doc_tutorial.html#MoveSemantics
- 112233 9y ago
  > I don't understand why you are making a distinction between table-like and non-table-like containers: both can take allocators. Because for map<> in general I cannot know how much allocations and of what size will it need to store 1000 elements. I might glean it for particular implementation of STL, but not via any api. Doing anything bearssl-like, "No dynamic allocation whatsoever", just does not work with STL. > you cannot use any external library in your function, since other libraries may have other allocation requirements that your code would break. Exactly. Most C++ libraries are not made with the idea of allowing you to police resource allocation. It's malloc all the way.
  - jcelerier 9y ago
    > Exactly. Most C++ libraries are not made with the idea of allowing you to police resource allocation I was saying this from the point of view of the library author, not the code. e.g. for instance I spent some weeks optimizing allocations in a library that I'm working on, and am then making assumptions in the rest of the code according to the optimizations I made. If I did let users of the library change the allocation policy, I would have to introduce costly runtime checks everywhere to ensure that the invariants I set do still hold, that I really have enough memory to do what I must, and abort / throw in case such an invariant is broken. All of these situations are less desirable than enforcing my allocation policy in my code.
  - pjmlp 9y ago
    You can have an idea for sure, that is what profilers are for. Have datasets of typical execution runs, profile test executions and gather statistics for ideal average sizes. Gut feeling for sizes never ends well.

by 112233 9y ago

jcelerier 9y ago

> I cannot use STL as it is now, even with custom allocators. if you are ready to take the runtime hit you could use the polymorphic allocator support and pass your allocator object down the chain. See http://en.cppreference.com/w/cpp/memory/polymorphic_allocator http://en.cppreference.com/w/cpp/memory/polymorphic_allocato... ; http://en.cppreference.com/w/cpp/container/vector http://en.cppreference.com/w/cpp/container/vector ; http://en.cppreference.com/w/cpp/container/map http://en.cppreference.com/w/cpp/container/map . I don't understand why you are making a distinction between table-like and non-table-like containers: both can take allocators. But in general, I don't really understand why you would want to do this: how do you expect your code to differentiate between the containers that you want to be affected by the memory changes and the ones that you don't want to ? If you don't want to make this difference and have a blanket coverage of, say, every std::vector called by your function it means that you cannot use any external library in your function, since other libraries may have other allocation requirements that your code would break. This looks like this would completely break encapsulation. The cleanest thing to do is to do like RapidJSON for instance and just pass allocator objects in your functions : http://rapidjson.org/md_doc_tutorial.html#MoveSemantics http://rapidjson.org/md_doc_tutorial.html#MoveSemantics

112233 9y ago
> I don't understand why you are making a distinction between table-like and non-table-like containers: both can take allocators. Because for map<> in general I cannot know how much allocations and of what size will it need to store 1000 elements. I might glean it for particular implementation of STL, but not via any api. Doing anything bearssl-like, "No dynamic allocation whatsoever", just does not work with STL. > you cannot use any external library in your function, since other libraries may have other allocation requirements that your code would break. Exactly. Most C++ libraries are not made with the idea of allowing you to police resource allocation. It's malloc all the way.
- jcelerier 9y ago
  > Exactly. Most C++ libraries are not made with the idea of allowing you to police resource allocation I was saying this from the point of view of the library author, not the code. e.g. for instance I spent some weeks optimizing allocations in a library that I'm working on, and am then making assumptions in the rest of the code according to the optimizations I made. If I did let users of the library change the allocation policy, I would have to introduce costly runtime checks everywhere to ensure that the invariants I set do still hold, that I really have enough memory to do what I must, and abort / throw in case such an invariant is broken. All of these situations are less desirable than enforcing my allocation policy in my code.
- pjmlp 9y ago
  You can have an idea for sure, that is what profilers are for. Have datasets of typical execution runs, profile test executions and gather statistics for ideal average sizes. Gut feeling for sizes never ends well.