5 ms·

Debug your programs like they're closed source

126 points by zimbu668 12y ago

Terr_ 12y ago
For those on Windows, the excellent System Internals suite contains "Process Monitor", which does pretty much what you'd expect, allowing you to see all sorts of file (and registry) activity.
- lagg 12y ago
  Agreed. I get by without having things I'm used to on my linux system like strace, lsof and so forth with it. It has a pane that displays pretty much everything with a handle from non-IP IPC sockets, normal sockets, files, registry, threads, DLLs and even obscure stuff like event handles and security context tokens.
- shitlord 12y ago
  Process Hacker is also nice, and the last time I used it, it displayed even more information. Plus it lets you inject DLLs, scan memory, and stuff!
- SonOfLilit 12y ago
  SysInternals procmon.exe and procexp.exe are the only things I really miss when on Linux (either developing or just troubleshooting Firefox being slow again.) A few procmon features you probably weren't aware of: - It can log your boot process (by loading a driver very early in the boot process) - It stores a lot of details about every line of output, including a stack trace (!), accessible by doubleclicking
  - Gurkenmaster 12y ago
    I miss Search Everything on linux. Nothing beats real-time indexing of the filesystem without overhead.
    - keeperofdakeys 12y ago
      You could setup something almost like it. mlocate is the defacto indexed filename search for linux, and the update script skips directories that haven't been updated - so an update call is quite cheap. You could run it every 15m, and have 'almost' uptodate indexed search. Most systems should have it setup to run daily out of the box.
      - comex 12y ago
        There are also more sophisticated desktop search tools like Recoll, of course.
- agumonkey 12y ago
  Got to System Internals for procexp and tcpview but I'll never forget the first time I ran procmon. ps: Windows really isn't sleepy, a lot happen all the time it's a bit scary. I don't know how other OSes are behaving when idleing though.
basicallydan 12y ago
Great article, very interesting! I never knew about this :) For those on OSX, `dtruss` seems to work in more-or-less the same way. sudo dtruss -t open ssh whatever.com More info on stack overflow: http://stackoverflow.com/questions/1925978/equivalent-of-strace-feopen-command-on-mac-os-x http://stackoverflow.com/questions/1925978/equivalent-of-str...
- Someone 12y ago
  On Mac OS X you really want to learn about DTrace. dtruss is a DTrace script. See http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-mac-os-x/ http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scr... for more useful DTrace scripts.
  - alexisnorman 12y ago
    This just changed my life, thanks so much
- vitd 12y ago
  Also on OS X, `fs_usage` will tell you which processes are accessing which files. Very useful!
  - Joky 12y ago
    From what I understand it report all interactions with the filesystem after its launch, it does tell you what process is currently accessing which file. lsof does a pretty good job for that.
jordigh 12y ago
Ah, an article on tracing. I am very impressed by lttng myself: http://lttng.org/ http://lttng.org/
cottonseed 12y ago
I used to work for a (no defunct) company, ARDI, that made a Mac emulator. We dynamically translated 68K to x86 and did a clean room reimplementation of Mac OS 9. (Yes, crazy.) Much of the job involved debugging the OS when running 3rd party software (binaries). The first step was to collect a syscall trace. Then lots of thought experiments. It was a great way to develop extreme debugging skills.
- meric 12y ago
  > clean room reimplementation of Mac OS 9 Wow. Was the project successful? Where can we read more about it? Was it a perfect implementation - all the images in the right places pixel by pixel?
  - Someone 12y ago
    http://en.m.wikipedia.org/wiki/Executor_(software) http://en.m.wikipedia.org/wiki/Executor_(software) Source is at https://github.com/ctm/executor https://github.com/ctm/executor I think the OP is of by two OS versions, but I didn't work at ARDI, so I may be wrong.
    - cottonseed 12y ago
      No, it looks like you're right, I misremembered. I haven't thought about this in a long time. Whoa, Cliff open-sourced Executor. I didn't know that.
  - cottonseed 12y ago
    > Was the project successful? As a business? No. As a hack? It was compatible with quite a bit of software and with syn68k on a Pentium whatever of the time, it was faster than any Mac.
- frozenport 12y ago
  Your my hero, I used this to play Escape Velocity before the Windows port. Anyways, Mac OS 9 doesn't run on 68k and exucator didn't run PPC applications?
akkartik 12y ago
Everytime I read about strace I feel guilty about not having tried it out. Another awesome story: http://lwn.net/1999/0121/a/mec.html http://lwn.net/1999/0121/a/mec.html
Nutomic 12y ago
I'm curious, is there anything like this for Android and calls to any Java system functions? (it should be possible with XPrivacy)
userbinator 12y ago
Syscall tracing works well for debugging applications or pieces of one that interact externally, or for ensuring that the OS is doing what you think it should, but I've found that more often than not the buggy part is "inside"; e.g. if an output file is incorrect, you would be able to see the incorrect data being written to it via the OS (which would've been clear from inspecting the file itself), but not what lead to that. Coincidentally, I prefer to use an Asm-level debugger (WinDbg, OllyDbg) whenever I need to use one - which for debugging something I wrote, is not very often. Maybe it has to do with the fact that I started with Asm and disassembling other programs, so it's relatively straightforward to map between source code and the instructions the compiler generated. This also helped with figuring out a very subtle compiler bug - I know that they're quite rare, but when they do occur, they tend to be extremely difficult to figure out otherwise.
- Gurkenmaster 12y ago
  OllyDbg is pretty good but it doesn't work with 64bit applications. Can anyone point me to one that supports 64bit?
  - j_s 12y ago
    Here is a discussion from 6 months ago: https://news.ycombinator.com/item?id=8092273 https://news.ycombinator.com/item?id=8092273
- Terr_ 12y ago
  > WinDbg, OllyDbg As someone who seldom leaves higher-abstraction languages, I've had some fun using Cheat Engine against old single-player games. I remember being pretty proud of the first time I managed to replace a few instructions with a larger set sitting in a code-cave. (Nothing impressive for those who learned assembly in a structured way, of course.)
  - girvo 12y ago
    Ah, Cheat Engine. I used to use it for multiplayer flash or shockwave games, as well as a couple other games along the years. Poor Nitto 1320, we killed that game. Got me into disassembly and hacking at that level, which was a lot of fun too.
antoaravinth 12y ago
Even though I havent followed the above idea ( I will try to do in sometime ), I have followed this way, which have helped me a lot. I used to make guess when I debug my day to day problem. Say for example, when I run selenium, and the browser doesn't open up: 1. Does the jar is valid? 2. Safari has selenium plugin? 3. Have custom installation rather than default installation location? Then the smart you make the guess, the smart you can solve the problem. Its about removing the un-wanted space ( by making smart guess ) and debugging the right space.

126 points by zimbu668 12y ago

Terr_ 12y ago

For those on Windows, the excellent System Internals suite contains "Process Monitor", which does pretty much what you'd expect, allowing you to see all sorts of file (and registry) activity.

lagg 12y ago
Agreed. I get by without having things I'm used to on my linux system like strace, lsof and so forth with it. It has a pane that displays pretty much everything with a handle from non-IP IPC sockets, normal sockets, files, registry, threads, DLLs and even obscure stuff like event handles and security context tokens.
shitlord 12y ago
Process Hacker is also nice, and the last time I used it, it displayed even more information. Plus it lets you inject DLLs, scan memory, and stuff!
SonOfLilit 12y ago
SysInternals procmon.exe and procexp.exe are the only things I really miss when on Linux (either developing or just troubleshooting Firefox being slow again.) A few procmon features you probably weren't aware of: - It can log your boot process (by loading a driver very early in the boot process) - It stores a lot of details about every line of output, including a stack trace (!), accessible by doubleclicking
- Gurkenmaster 12y ago
  I miss Search Everything on linux. Nothing beats real-time indexing of the filesystem without overhead.
  - keeperofdakeys 12y ago
    You could setup something almost like it. mlocate is the defacto indexed filename search for linux, and the update script skips directories that haven't been updated - so an update call is quite cheap. You could run it every 15m, and have 'almost' uptodate indexed search. Most systems should have it setup to run daily out of the box.
    - comex 12y ago
      There are also more sophisticated desktop search tools like Recoll, of course.
agumonkey 12y ago
Got to System Internals for procexp and tcpview but I'll never forget the first time I ran procmon. ps: Windows really isn't sleepy, a lot happen all the time it's a bit scary. I don't know how other OSes are behaving when idleing though.

basicallydan 12y ago

Great article, very interesting! I never knew about this :) For those on OSX, `dtruss` seems to work in more-or-less the same way. sudo dtruss -t open ssh whatever.com More info on stack overflow: http://stackoverflow.com/questions/1925978/equivalent-of-strace-feopen-command-on-mac-os-x http://stackoverflow.com/questions/1925978/equivalent-of-str...

Someone 12y ago
On Mac OS X you really want to learn about DTrace. dtruss is a DTrace script. See http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-mac-os-x/ http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scr... for more useful DTrace scripts.
- alexisnorman 12y ago
  This just changed my life, thanks so much
vitd 12y ago
Also on OS X, `fs_usage` will tell you which processes are accessing which files. Very useful!
- Joky 12y ago
  From what I understand it report all interactions with the filesystem after its launch, it does tell you what process is currently accessing which file. lsof does a pretty good job for that.

jordigh 12y ago

Ah, an article on tracing. I am very impressed by lttng myself: http://lttng.org/ http://lttng.org/

cottonseed 12y ago

I used to work for a (no defunct) company, ARDI, that made a Mac emulator. We dynamically translated 68K to x86 and did a clean room reimplementation of Mac OS 9. (Yes, crazy.) Much of the job involved debugging the OS when running 3rd party software (binaries). The first step was to collect a syscall trace. Then lots of thought experiments. It was a great way to develop extreme debugging skills.

meric 12y ago
> clean room reimplementation of Mac OS 9 Wow. Was the project successful? Where can we read more about it? Was it a perfect implementation - all the images in the right places pixel by pixel?
- Someone 12y ago
  http://en.m.wikipedia.org/wiki/Executor_(software) http://en.m.wikipedia.org/wiki/Executor_(software) Source is at https://github.com/ctm/executor https://github.com/ctm/executor I think the OP is of by two OS versions, but I didn't work at ARDI, so I may be wrong.
  - cottonseed 12y ago
    No, it looks like you're right, I misremembered. I haven't thought about this in a long time. Whoa, Cliff open-sourced Executor. I didn't know that.
- cottonseed 12y ago
  > Was the project successful? As a business? No. As a hack? It was compatible with quite a bit of software and with syn68k on a Pentium whatever of the time, it was faster than any Mac.
frozenport 12y ago
Your my hero, I used this to play Escape Velocity before the Windows port. Anyways, Mac OS 9 doesn't run on 68k and exucator didn't run PPC applications?

akkartik 12y ago

Everytime I read about strace I feel guilty about not having tried it out. Another awesome story: http://lwn.net/1999/0121/a/mec.html http://lwn.net/1999/0121/a/mec.html

Nutomic 12y ago

I'm curious, is there anything like this for Android and calls to any Java system functions? (it should be possible with XPrivacy)

userbinator 12y ago

Syscall tracing works well for debugging applications or pieces of one that interact externally, or for ensuring that the OS is doing what you think it should, but I've found that more often than not the buggy part is "inside"; e.g. if an output file is incorrect, you would be able to see the incorrect data being written to it via the OS (which would've been clear from inspecting the file itself), but not what lead to that. Coincidentally, I prefer to use an Asm-level debugger (WinDbg, OllyDbg) whenever I need to use one - which for debugging something I wrote, is not very often. Maybe it has to do with the fact that I started with Asm and disassembling other programs, so it's relatively straightforward to map between source code and the instructions the compiler generated. This also helped with figuring out a very subtle compiler bug - I know that they're quite rare, but when they do occur, they tend to be extremely difficult to figure out otherwise.

Gurkenmaster 12y ago
OllyDbg is pretty good but it doesn't work with 64bit applications. Can anyone point me to one that supports 64bit?
- j_s 12y ago
  Here is a discussion from 6 months ago: https://news.ycombinator.com/item?id=8092273 https://news.ycombinator.com/item?id=8092273
Terr_ 12y ago
> WinDbg, OllyDbg As someone who seldom leaves higher-abstraction languages, I've had some fun using Cheat Engine against old single-player games. I remember being pretty proud of the first time I managed to replace a few instructions with a larger set sitting in a code-cave. (Nothing impressive for those who learned assembly in a structured way, of course.)
- girvo 12y ago
  Ah, Cheat Engine. I used to use it for multiplayer flash or shockwave games, as well as a couple other games along the years. Poor Nitto 1320, we killed that game. Got me into disassembly and hacking at that level, which was a lot of fun too.

antoaravinth 12y ago

Even though I havent followed the above idea ( I will try to do in sometime ), I have followed this way, which have helped me a lot. I used to make guess when I debug my day to day problem. Say for example, when I run selenium, and the browser doesn't open up: 1. Does the jar is valid? 2. Safari has selenium plugin? 3. Have custom installation rather than default installation location? Then the smart you make the guess, the smart you can solve the problem. Its about removing the un-wanted space ( by making smart guess ) and debugging the right space.