In this part, I’ll show how to use the debugging tools to analyze crash dumps from windows when it crashes and gives the ‘blue screen of death’

Precusor: Setup Windows to generate a crash dump file

Windows doesn’t necessarily need to create a crash dump file, but it’s a good idea to do so.

In the system properties control panel, On the advanced tab, click ‘Startup and Recovery->Settings’ (This is right above where you had to set the environment variable from part1)

Be sure there’s a memory dump type selected. Note that “Kernel memory Dump” = “full memory dump” and you’ll need a swap file on your C drive thats at least as big as your RAM.

Memory dump locations:

• Kernel/Full: c:\windows\MEMORY.DMP (Only one copy kept unless moved/renamed)
• Small/Minidump: c:\windows\minidumps

One more interesting tidbit – you can set windows up to allow you to crash it at will – That’s a topic for another article.

———–

Checking out your first crash dump file:
Start WinDbg
(the first time you do this it takes a while as it downloads the symbols)
File->Open Crash Dump…
Find your crash dump file.
watch the output…

If you’re lucky, You’ll see a line such as:
Probably caused by: filename.ext

Many times, this first step of opening the crash dump tells you what you need to know.

Sometimes it doesn’t and you want to look a little deeper.

The next thing to try is often
!analyze -v

think of the debugger as a CMD window – you won’t find !analyze -v in the menus, but you can type it in at the > prompt.
(You can also click it if you see it in the display)

 If this doesn’t work the options get a little tricker…

In my next post, I’ll show a few hand picked commands that let you see what was running.

   This article will show you what you need to read into a windows crash dump file…

   What you’ll need:

• Debugging tools for windows
• Symbol files

  Download Debugging tools for windows -
  Rather than including a link, I recommend searching for ‘debugging tools for windows’
  You should find a bunch of links on a Microsoft site to download the latest version.

    What to download?

• if your OS is 32bit, download the x86 version
• if your OS is 64 bit, download Both the x86 and x64 versions. (the reason for this is that you’ll use the 32 bit debugger to debug 32bit crash dumps from other machines, and also 32bit apps running on your 64 bit machine. – also note, there are 2 64 bit verisons on Microsoft’s website – 99% of us want x64, not Itanium)

    I assume you can downl0ad the 1 or 2 debuggers you’ll need, Go ahead and run the setup for each and install them to thier default locations.
We’re not quite ready to do anything with the debuggers yet, first we need to discuss Symbol Files…

Symbol files:
      You’ll want symbol files when you look at a crash dump or debug an app.

     Symbol files are tied to each piece of software – Ideally you’ll have them for everything on your system.

     Unforunately, that’s rarely possible. The good news is you can get them for Windows, and that’s often enough.

     In the old days, you’d download the symbols you think you’d need from Microsoft and install them on your machine.

     That’s no longer necessary. The current Debugging tools for windows supports auto downloading symbol files from microsoft as needed.
Unfortunately, the debugging tools don’t work that way out of the box, so that’ll be the first thing we get setup…

  Setting up windows, so your debugger knows what to do about symbol files
  It’s a shame this isn’t default behavior, but at least it’s not that hard…  I’ll show you how to configure an environment variable in windows, which the debugger will use automatically each time it runs. You don’t have to do this, you can still run the debugger without doing this, or you can run the debugger and then tell it manually about the symbolfiles, but you’re better off doing the environment variable thing now – get it out of the way so you don’t have to worry about it later…

  Setup the Environment Variable _NT_SYMBOL_PATH 
  Set it to SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols

  To do this, in windows, right click on ‘my computer’ or ‘computer’ and click ‘properties’.  Earlier versions of windows show a nice tab dialog, later versions of windows (Vista, 2008, 7…) show a fancy screen also known as the ‘system’ control panel – if this iswhat you see, then click on ‘Advanced System Settings’. Now you should see what your XP friends saw 2 sentences ago- the “System properties” dialog – click on the ‘advanced’ tab, then on the ‘environment variables’ button.

 Add a new system variable and name it _NT_SYMBOL_PATH (note the text begins with the underscore) 
  Set it to SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols

 Notice in the line above there’s some references to the directory c:\mysymbols – We need to create that directory now. (If you want it somewhere else, that’s fine, just be sure to change it in _NT_SYMBOL_PATH.

The other bit of interest is that link to microsoft’s symbol website..

Basically what we’ve told the debugger is:

1. Look for symbol files in c:\mysymbols
2. If you can’t find them, look for them at the website http://msdl.microsoft.com/download/symbols

If you didn’t create the c:\mysymbols directory yet, do so now…

If you have 2 debuggers installed (x86 and x64) you only need to do the above once.

Congratulations! You’re all setup.

Closing Comments:

The easiest thing to forget here is likely the environment variable _NT_SYMBOL_PATH and what to point it to.  Fortunately, this is actually pretty easy information to find in the help file.
After installing the debugging tools for windows, Open the help file, click the ‘index’ tab, then enter the word ‘env’ (you can type out environment variables if you want, but it finds them after env)

Environment variables brings up 2 sections, General and Kernel-Mode – you want General. The page that appears shows all the _NT_YADA_YADA_YADA variables. NT_SYMBOL_PATH is the 4th one down.
You have to click one more time to find out what to set it to, in the description there’s a link to ‘Symbol Path’ – click that and get a page talking about he symbol path – down towards the bottom in red, is the symbol path you need.

In my next part, I’ll show how to open a crash dump file and a few easy commands you can use before you go searching google for that long hex code you wrote down by hand when your system blue screened.

below is a summary of some techniques – sorry that its not real descriptive. I’ll add to it one day – I promise..

In vista, you need to attach to the dying process.
Launch windebug and attach to the dying process.

Configure vista or 2008 to always generate a dump file (for application crashes)
Create key named:
Hklm\software\microsoft\windows\windows error reporting\localdumps

Dumps go to %localappdata%\crashdumps
Override with a Dumpfolder (string) value
Limit dump history with a dumpcount (dword) value

!analyze –v

If this doesn’t work –
Look at the thread stack for functions with ‘fault’ ‘exception’ or ‘error’
Open callstack window – the look at the threads looking for stacks

~0 kb (inspect call stack of CPU0)
~1 kb  (inspect call stack of CPU1)
Lm kvmvpn* (list driver version?)