3 ms·

Wait a sec. You can't bust out numbers like that without being clear about a few key points: 1. What was std::vector's removal algorithm like? Are you sure

Wait a sec. You can't bust out numbers like that without being clear about a few key points: 1. What was std::vector's removal algorithm like? Are you sure that .remove() wasn't shifting all elements in the vector? The "correct" removal algorithm in that case looks like this: Swap the target element to the end, and then decrease the vector's size by 1. But if I remember correctly, std::vector's remove method doesn't work like that, so of course it will have terrible worst-case performance if it was shifting all elements of the array when you remove the first one. 2. What's causing the worst-case slowdown? Are you sure you weren't simply seeing the effects of the L2 CPU cache being invalidated? A linked list will thrash the CPU cache constantly, whereas vectors thrash the cache much less since the elements are stored in contiguous memory. For a modern game running on a modern CPU, vectors are almost always the better choice for that reason. EDIT: I'm not sure your numbers are trustworthy. For example, there's no way it rendered 35859663 frames in 100 seconds. That would be 358,596 FPS. Assuming your FPS was actually something like 358, not 358,596, that's still an extremely high FPS. When your FPS is so high, it becomes difficult to accurately measure the reasons for worst-case performance. A sudden spike within a single frame could be for any of a dozen reasons, such as CPU cache eviction, a GPU pipeline stall, or some other program on your computer taking CPU resources.

falcolas 12y ago
> You can't bust out numbers like that Nobody is preventing you from writing your own benchmark and "busting out [your own] numbers". :) That said, yes, my deletion algorithm is naive, and I welcome a pull request to fix it. https://github.com/garrickp/intrusive_benchmark https://github.com/garrickp/intrusive_benchmark > Are you sure you weren't simply seeing the effects of the L2 CPU cache being invalidated Perhaps, but I doubt it's the cause of the worst case scenario, particularly since I'm doing identical loops over the physics and renderable objects, and as you say the linked list should have the worse performance. EDIT: One more small point - if the programmer is relying on the order of the items at all, the "tail tuck" method of deletion would not work. EDIT[2]: Addressing your edit - the numbers are only for the action of applying physics and adding/deleting objects - the code should reflect that. There is no actual rendering time or AI or anything else which would prevent us from seeing upwards of 100,000 "frames" per second. The times are also fairly stable (the standard deviation of max times is around 20%) between runs.
- sillysaurus3 12y ago
  I apologize that my comment probably came off as a bit too upfront. I need to work on that. I was only saying that it isn't really a good idea to present numbers like that without having a clear idea of what's causing the worst-case performance. Glancing over the code, and assuming the first bullet fired is always the first one to impact the ground, it looks like the code is always going to run std::vector::erase on the first element of the vector, which will always shift the 499 remaining elements. It's possible that's the reason for the worst-case performance, rather than vector being inherently a worse choice than list. It was very counterintuitive and weird to realize that vector is almost always a better choice than list due to how powerful the CPU caches are. CPU caching is so good that taking full advantage of caching will usually outweigh algorithmic tweaks, like whether you're using a list or a vector. So the power of vector is that its elements are in contiguous memory, whereas the downfall of list is that they usually aren't. It's difficult to prove that's true except in real-world scenarios. Unless a test is designed to introduce memory fragmentation, the cache performance penalty of using list won't really show up in the results. All of the list allocations will be pretty much contiguous in a test environment, so the tests will show performance on par with vector. But in the real engine, you'll be left scratching your head why everything's running so slowly, when the answer is because the elements are sprayed all across your memory space and it's causing the CPU cache to stall when they're accessed... EDIT: There is no actual rendering time or AI or anything else which would prevent us from seeing upwards of 100,000 "frames" per second. When your FPS is higher than, say, 1,000, it becomes very difficult to pin down the reasons for performance fluctuations. An engine under load behaves differently than an artificial test, so you'll need to measure the performance impact in a real engine, especially when the framerates in the test are so high. Your monitor is capped at 70 fps, or some high-end monitors can do ~130. Something like 100,000 fps isn't an accurate reflection of the behavior of the real system. Most of the discrepancy is due to memory fragmentation causing CPU thrashing, but your specific test might be running into some other discrepancies, like: - clock() might not be a high resolution timer, so the worst-case timing might be due to discrepancies in the clock mechanism itself, especially at 100,000+ FPS. See http://stackoverflow.com/questions/6749621/high-resolution-timer-in-linux http://stackoverflow.com/questions/6749621/high-resolution-t... - since your test is running for 100+ seconds and your timing measurements are taking into account an entire frame, some frames are going to run interrupts. If an interrupt takes a long time to process, that load will show up in your timing measurements. The result is that vector (or list) will be unfairly pinned as the reason for the performance issue, when interrupts were the cause. I'm not confident about the second one, but the point is that there are all kinds of conflating variables.
ANTSANTS 12y ago
>Assuming your FPS was actually something like 358, not 358,596, that's still an extremely high FPS. When your FPS is so high, it becomes difficult to accurately measure the reasons for worst-case performance. A sudden spike within a single frame could be for any of a dozen reasons, such as CPU cache eviction, a GPU pipeline stall, or some other program on your computer taking CPU resources. As an aside, programmers need to stop using FPS and other inverse units of measurement when performance tuning. It's much easier to reason about 2.79~ milliseconds than it is to reason about 358 FPS. For instance, assuming a single threaded application, let's say I know my game simulation alone can be updated at 125FPS, while my rendering code can be updated at 30FPS. How many FPS does my game run at when I run both together? You'd be hard pressed to figure that out without converting the two to 8 and 33⅓ milliseconds per tick, respectively, making the combination of the two obviously 41⅓ milliseconds, and thus about 24.2FPS. Yes, your comment about performance measurements under lights loads not accurately reflecting performance under greater loads is true, but 2.79 milliseconds really aren't so far from the standard target of 16.66 milliseconds per frame than the 358 vs. 60FPS comparison would make one guess.

falcolas 12y ago

> You can't bust out numbers like that Nobody is preventing you from writing your own benchmark and "busting out [your own] numbers". :) That said, yes, my deletion algorithm is naive, and I welcome a pull request to fix it. https://github.com/garrickp/intrusive_benchmark https://github.com/garrickp/intrusive_benchmark > Are you sure you weren't simply seeing the effects of the L2 CPU cache being invalidated Perhaps, but I doubt it's the cause of the worst case scenario, particularly since I'm doing identical loops over the physics and renderable objects, and as you say the linked list should have the worse performance. EDIT: One more small point - if the programmer is relying on the order of the items at all, the "tail tuck" method of deletion would not work. EDIT[2]: Addressing your edit - the numbers are only for the action of applying physics and adding/deleting objects - the code should reflect that. There is no actual rendering time or AI or anything else which would prevent us from seeing upwards of 100,000 "frames" per second. The times are also fairly stable (the standard deviation of max times is around 20%) between runs.

sillysaurus3 12y ago
I apologize that my comment probably came off as a bit too upfront. I need to work on that. I was only saying that it isn't really a good idea to present numbers like that without having a clear idea of what's causing the worst-case performance. Glancing over the code, and assuming the first bullet fired is always the first one to impact the ground, it looks like the code is always going to run std::vector::erase on the first element of the vector, which will always shift the 499 remaining elements. It's possible that's the reason for the worst-case performance, rather than vector being inherently a worse choice than list. It was very counterintuitive and weird to realize that vector is almost always a better choice than list due to how powerful the CPU caches are. CPU caching is so good that taking full advantage of caching will usually outweigh algorithmic tweaks, like whether you're using a list or a vector. So the power of vector is that its elements are in contiguous memory, whereas the downfall of list is that they usually aren't. It's difficult to prove that's true except in real-world scenarios. Unless a test is designed to introduce memory fragmentation, the cache performance penalty of using list won't really show up in the results. All of the list allocations will be pretty much contiguous in a test environment, so the tests will show performance on par with vector. But in the real engine, you'll be left scratching your head why everything's running so slowly, when the answer is because the elements are sprayed all across your memory space and it's causing the CPU cache to stall when they're accessed... EDIT: There is no actual rendering time or AI or anything else which would prevent us from seeing upwards of 100,000 "frames" per second. When your FPS is higher than, say, 1,000, it becomes very difficult to pin down the reasons for performance fluctuations. An engine under load behaves differently than an artificial test, so you'll need to measure the performance impact in a real engine, especially when the framerates in the test are so high. Your monitor is capped at 70 fps, or some high-end monitors can do ~130. Something like 100,000 fps isn't an accurate reflection of the behavior of the real system. Most of the discrepancy is due to memory fragmentation causing CPU thrashing, but your specific test might be running into some other discrepancies, like: - clock() might not be a high resolution timer, so the worst-case timing might be due to discrepancies in the clock mechanism itself, especially at 100,000+ FPS. See http://stackoverflow.com/questions/6749621/high-resolution-timer-in-linux http://stackoverflow.com/questions/6749621/high-resolution-t... - since your test is running for 100+ seconds and your timing measurements are taking into account an entire frame, some frames are going to run interrupts. If an interrupt takes a long time to process, that load will show up in your timing measurements. The result is that vector (or list) will be unfairly pinned as the reason for the performance issue, when interrupts were the cause. I'm not confident about the second one, but the point is that there are all kinds of conflating variables.

ANTSANTS 12y ago

>Assuming your FPS was actually something like 358, not 358,596, that's still an extremely high FPS. When your FPS is so high, it becomes difficult to accurately measure the reasons for worst-case performance. A sudden spike within a single frame could be for any of a dozen reasons, such as CPU cache eviction, a GPU pipeline stall, or some other program on your computer taking CPU resources. As an aside, programmers need to stop using FPS and other inverse units of measurement when performance tuning. It's much easier to reason about 2.79~ milliseconds than it is to reason about 358 FPS. For instance, assuming a single threaded application, let's say I know my game simulation alone can be updated at 125FPS, while my rendering code can be updated at 30FPS. How many FPS does my game run at when I run both together? You'd be hard pressed to figure that out without converting the two to 8 and 33⅓ milliseconds per tick, respectively, making the combination of the two obviously 41⅓ milliseconds, and thus about 24.2FPS. Yes, your comment about performance measurements under lights loads not accurately reflecting performance under greater loads is true, but 2.79 milliseconds really aren't so far from the standard target of 16.66 milliseconds per frame than the 358 vs. 60FPS comparison would make one guess.