5 ms·

Lots of things inside Windows emit ETW events, which is Windows equivalent of DTrace (basically the entire OS, and .NET), it's super useful for debugging perfor

Lots of things inside Windows emit ETW events, which is Windows equivalent of DTrace (basically the entire OS, and .NET), it's super useful for debugging performance related events. It's not "Telemetry" like Google Analytics, it's for _you_ to debug your own programs. The easiest way to view the output of them is via WPA, you can watch some videos about it at https://msdn.microsoft.com/en-us/library/windows/hardware/hh448170.aspx https://msdn.microsoft.com/en-us/library/windows/hardware/hh...

userbinator 10y ago
It's not "Telemetry" like Google Analytics, it's for _you_ to debug your own programs. People have been debugging just fine for the past few decades without extra bloating code in their runtime libraries. A good old-fashioned debugger with tracepoints will accomplish the same thing, and doesn't add overhead when it's not in use.
- xpaulbettsx 10y ago
  Tracepoints are very expensive and will definitely make any performance traces incorrect
  - userbinator 10y ago
    I don't know what you mean exactly by "expensive", but the time spent in profiling code should obviously not be counted, and it is quite simple to do so. The real point I'm trying to make is that tracepoints do not require any changes to the original code. They are entirely opt-in and if you do not choose to use them, your application won't have this extra useless code compiled into it. That said, the few times I've had to use Visual Studio's tracepoints, they were horribly slow. Apparently this is "by design": https://connect.microsoft.com/VisualStudio/feedback/details/934631/tracepoints-feature-is-very-slow https://connect.microsoft.com/VisualStudio/feedback/details/... Being an RE, I've implemented and used simple tracers that were more than 100x faster than VS' on hardware not quite as fast as that today (around a decade ago). Watching variables in memory and CPU registers changing in near-realtime is something I very much miss when I have to work in higher-level languages...
    - MaulingMonkey 10y ago
      > I don't know what you mean exactly by "expensive", but the time spent in profiling code should obviously not be counted, and it is quite simple to do so. Only for the exceedingly trivial case where you exclude profiling CPU time from overall CPU time. Profiling to see when exactly you're CPU bound, and when you're disk bound? Eh. Profiling to see if false sharing is eating your lunch? Eh. Trying to see when you'll fail to make vsync and the framerate will hitch? Eh. I've had to #ifdef out tracing info that boiled down to "if (happensToBeFalse) {...}" in tight loops because it was throwing off my profiling results too much. Compiler and debugger injected tracing info isn't going to be any cheaper than that. > The real point I'm trying to make is that tracepoints do not require any changes to the original code. They are entirely opt-in and if you do not choose to use them, your application won't have this extra useless code compiled into it. Debugger injected tracing is useful, but hardly a panacea. You know this or you wouldn't have written your own tracers, most likely. When you have the source code, writing a separate tracer doesn't make sense - you just embed your stuff directly into your source code and #ifdef it out when necessary (same result: no "extra useless code".) Easier to add data to your events ("what file am I hanging on?") without performance killing deep copies, or reverse engineering structure offsets. No need to subvert platform security on e.g. Xbox One to figure out how to inject your own code at runtime. No need to deal with another process's address space. Sometimes tracing info is useful to leave in your release builds. Which is cheaper: A few rare if() checks and some dormant code? Or spending hours rebuilding, retesting, redeploying to the production machine that is experiencing some issue you need tracing info to diagnose? As long as you're not shoving it into your inner loops it's fine. An event on entry/exit of main is about as far away from an inner loop as you can manage. It's fine.
hirsin 10y ago
Even easier, you can also use Device Portal [1] in new Windows 10 builds to get a live feed of ETW providers from your device in a browser. Think logcat but with more structure to the events and providers (but without the buffers, sadly) 1. https://msdn.microsoft.com/en-us/windows/uwp/debug-test-perf/device-portal#event-tracing-for-windows-etw- https://msdn.microsoft.com/en-us/windows/uwp/debug-test-perf... Disclaimer - I work on Device Portal.
retox 10y ago
A very similar post appeared on reddit. Almost identical.
DominikD 10y ago
Man, you're fighting uphill battle. This is a typical case of people being scared of something they don't understand coupled with not willing to learn. Yesterday[1] I read that RFID tags emit radiation because RF means radio frequency, so it's harmful. Today I read that ETW is harmful telemetry and MS injecting code into binaries. I thought I got over the fact that a lot of people programming on Windows don't understand its internals. But I didn't. Understanding ETW is essential on Windows, just like DTrace is a must on Linux or ktrace on OpenBSD. It's also well documented on MSDN[2]. ETW also typically boils down to a single check for any listener interested in messages. This info is public and has been for ages. I guess I'd understand this attitude if there were no real reasons to bash MS. But I guess what pisses me the most is the context of that reddit thread. People complain about mutex performance and show code where they mutex vector accesses. This kind of code does not belong in performance critical game code. "Hey, I've got a salad to make, what tools do I need? I've got a hammer but these veggies are not a nail. But I'm sure if I take two hammers to the task my problem suddenly changes to nailing things together!" For Pete's sake, if a slight performance drop in mutex performance of user mode code hogs your application, you're mutexing too much. Your code is a problem, not a mutex. And if you're using single threaded STL primitives for communication between threads you have no right to complain about CRT performance. None. This lack of intellectual curiosity in people sinks my heart. Rant out. :S [1] not on HN though, thankfully [2] well by MSDN standards at least - ETW documentation is hard to grok without referencing sample code in the Windows SDK
- pjmlp 10y ago
  Another good example was when Microsost introduced C++/CX, their language extensions for creating WinRT components in C++. Lots of devs were up in arms that C++ was going to be managed in the new WinRT world, just because C++/CX syntax is based on C++/CLI. Microsoft had to do lots of sessions to clarify that, and I bet even today there are developers that don't get the difference between C++/CX and C++/CLI.
frozenport 10y ago
But it could be telemetry like Google analytics. Given that MS didn't tell us they were doing it, what else are they sneaking in? At the end of the day this sets a really dangerous precedent. This also might have performance side-effects in highly parallel codes.

userbinator 10y ago

It's not "Telemetry" like Google Analytics, it's for _you_ to debug your own programs. People have been debugging just fine for the past few decades without extra bloating code in their runtime libraries. A good old-fashioned debugger with tracepoints will accomplish the same thing, and doesn't add overhead when it's not in use.

xpaulbettsx 10y ago
Tracepoints are very expensive and will definitely make any performance traces incorrect
- userbinator 10y ago
  I don't know what you mean exactly by "expensive", but the time spent in profiling code should obviously not be counted, and it is quite simple to do so. The real point I'm trying to make is that tracepoints do not require any changes to the original code. They are entirely opt-in and if you do not choose to use them, your application won't have this extra useless code compiled into it. That said, the few times I've had to use Visual Studio's tracepoints, they were horribly slow. Apparently this is "by design": https://connect.microsoft.com/VisualStudio/feedback/details/934631/tracepoints-feature-is-very-slow https://connect.microsoft.com/VisualStudio/feedback/details/... Being an RE, I've implemented and used simple tracers that were more than 100x faster than VS' on hardware not quite as fast as that today (around a decade ago). Watching variables in memory and CPU registers changing in near-realtime is something I very much miss when I have to work in higher-level languages...
  - MaulingMonkey 10y ago
    > I don't know what you mean exactly by "expensive", but the time spent in profiling code should obviously not be counted, and it is quite simple to do so. Only for the exceedingly trivial case where you exclude profiling CPU time from overall CPU time. Profiling to see when exactly you're CPU bound, and when you're disk bound? Eh. Profiling to see if false sharing is eating your lunch? Eh. Trying to see when you'll fail to make vsync and the framerate will hitch? Eh. I've had to #ifdef out tracing info that boiled down to "if (happensToBeFalse) {...}" in tight loops because it was throwing off my profiling results too much. Compiler and debugger injected tracing info isn't going to be any cheaper than that. > The real point I'm trying to make is that tracepoints do not require any changes to the original code. They are entirely opt-in and if you do not choose to use them, your application won't have this extra useless code compiled into it. Debugger injected tracing is useful, but hardly a panacea. You know this or you wouldn't have written your own tracers, most likely. When you have the source code, writing a separate tracer doesn't make sense - you just embed your stuff directly into your source code and #ifdef it out when necessary (same result: no "extra useless code".) Easier to add data to your events ("what file am I hanging on?") without performance killing deep copies, or reverse engineering structure offsets. No need to subvert platform security on e.g. Xbox One to figure out how to inject your own code at runtime. No need to deal with another process's address space. Sometimes tracing info is useful to leave in your release builds. Which is cheaper: A few rare if() checks and some dormant code? Or spending hours rebuilding, retesting, redeploying to the production machine that is experiencing some issue you need tracing info to diagnose? As long as you're not shoving it into your inner loops it's fine. An event on entry/exit of main is about as far away from an inner loop as you can manage. It's fine.

hirsin 10y ago

Even easier, you can also use Device Portal [1] in new Windows 10 builds to get a live feed of ETW providers from your device in a browser. Think logcat but with more structure to the events and providers (but without the buffers, sadly) 1. https://msdn.microsoft.com/en-us/windows/uwp/debug-test-perf/device-portal#event-tracing-for-windows-etw- https://msdn.microsoft.com/en-us/windows/uwp/debug-test-perf... Disclaimer - I work on Device Portal.

retox 10y ago

A very similar post appeared on reddit. Almost identical.

DominikD 10y ago

Man, you're fighting uphill battle. This is a typical case of people being scared of something they don't understand coupled with not willing to learn. Yesterday[1] I read that RFID tags emit radiation because RF means radio frequency, so it's harmful. Today I read that ETW is harmful telemetry and MS injecting code into binaries. I thought I got over the fact that a lot of people programming on Windows don't understand its internals. But I didn't. Understanding ETW is essential on Windows, just like DTrace is a must on Linux or ktrace on OpenBSD. It's also well documented on MSDN[2]. ETW also typically boils down to a single check for any listener interested in messages. This info is public and has been for ages. I guess I'd understand this attitude if there were no real reasons to bash MS. But I guess what pisses me the most is the context of that reddit thread. People complain about mutex performance and show code where they mutex vector accesses. This kind of code does not belong in performance critical game code. "Hey, I've got a salad to make, what tools do I need? I've got a hammer but these veggies are not a nail. But I'm sure if I take two hammers to the task my problem suddenly changes to nailing things together!" For Pete's sake, if a slight performance drop in mutex performance of user mode code hogs your application, you're mutexing too much. Your code is a problem, not a mutex. And if you're using single threaded STL primitives for communication between threads you have no right to complain about CRT performance. None. This lack of intellectual curiosity in people sinks my heart. Rant out. :S [1] not on HN though, thankfully [2] well by MSDN standards at least - ETW documentation is hard to grok without referencing sample code in the Windows SDK

pjmlp 10y ago
Another good example was when Microsost introduced C++/CX, their language extensions for creating WinRT components in C++. Lots of devs were up in arms that C++ was going to be managed in the new WinRT world, just because C++/CX syntax is based on C++/CLI. Microsoft had to do lots of sessions to clarify that, and I bet even today there are developers that don't get the difference between C++/CX and C++/CLI.

frozenport 10y ago

But it could be telemetry like Google analytics. Given that MS didn't tell us they were doing it, what else are they sneaking in? At the end of the day this sets a really dangerous precedent. This also might have performance side-effects in highly parallel codes.