Opening Memory Snapshots generated on other machines locally - mitikov/KeepSitecoreSimple GitHub Wiki

Intro

MSDN has a good article describing technical aspects, but it is so boring to read =\

Here are the steps needed to open memory snapshot and start exploring internals..

Pre-requirement

WinDbg must be installed on a local machine. It is a part of Debugging Tools for Windows

As we want Debugging Tools for Windows only, we can install the Windows SDK, and, during the installation, select the Debugging Tools for Windows box and clear all the other boxes.

Routine to open snapshot

Open WinDBG

Launch correct version WinDBG (x64 VS x86) depending on application pool bitness:

openwindbg

Not much rangers who keep on using x86 apps left nowadays, thus it is safe to launch WinDBG x64 in 99% cases.

Then click File - Open Crash Dump. You should see similar output:

dump_load_view

Loading .NET Framework files

SOS debugging extension (sos.dll) and mscordacwks.dll from the machine where memory snapshot was collected must be loaded.

Files can be found in %windows%\Microsoft.NET\Framework64\v4.0.x\folder of the target machine, and must be ported to a machine on which analysis will take place. As files can be a part of the MS updates, in 99 % cases a local machine and a target one will have different revisions, thereby cannot be loaded.

SSPG can collect .NET framework files — required for working with process memory dumps for you :-)

.load command should be executed in debugger window:

loadnetframeworkfiles

It can be followed by .chain command to see already loaded extensions ( WinDbg will automatically load some ).

Setting up symbols folder

WinDbg will try to load debug symbols ( PDBs ) from Microsoft Symbol Server when needed. We will create local cache folder ( *c:\symbols* in the example), and will use it further on:

.sympath "c:\symbols\"; .symfix+;.reload;

WARNING First load will take some time, as it needs to download lotsa data.

.sympath command can be used to check if everything works:

setting_sympath

Using SOSEX extension

SOSEX extends possibilities of SOS and has more user-friendly commands.

You need to download latest version of SOSEX and load it via .load [path_to_sosex]

Summarize

All aforementioned steps can be embedded into one line:

.sympath "c:\symbols\"; .symfix+;.reload;.load c:\path_to_mscordacwks;.load c:\path_to_sos;.load c:\sosex

Further steps

Commands provided by debugging extensions are documented here:

Tips

Use RAM Drive

The WinDbg performance much depends on random I/O reads, thereby reading random chunks from HDD will make an analysis slow.

Loading memory dump into RAM Drive will do the trick and save your day.

Memory dumps can be corrupted

This is especially true when a large snapshot of w3wp.exe is to be generated, and Application Pool settings have ping enabled.

Process can be killed before all the state copied to the snapshot.

An attempt to open snapshots without framework files

.cordll command will make an attempt to load matching version from local machine (.cordll -u -l -ve)

You'll see the exact runtime version that was failed to be loaded in case failure.

As .NET Framework files would be a part of Windows Updates, you can try to find a matching windows update, and unpack its content a few times ( you will get GDRGDR.cab and QFEGDR.cab files -> unzip them too ).

The resulting folders would have needed assemblies named like: clr_dll_amd64 -> should rename to clr.dll

You will need to find needed ones taking CPU architecture and bitness into account.

Launch DebugDiag Analysis first

DebugDiag tool ships with DebugDiag Analysis tool that can bring a brief summary of the snapshot, thereby saving tons of your time.

You can write C# code to perform some operations automatically

ClrMD is a set of advanced APIs for programmatically inspecting a crash dump of a .NET program much in the same way that the SOS Debugging Extensions (SOS) do. This allows you to write automated crash analysis for your applications as well as automate many common debugger tasks.

This is huge area that will not fit this post.