Windows Heap - microsoft/MSO-Scripts GitHub Wiki

Several tools measure amount of memory currently in use within Windows and its applications, mainly:
Task Manager, Process Explorer, and VMMap

Exposing the source of that memory usage requires runtime tracing via ETW (Event Tracing for Windows).

Here we provide three different ways to trace Virtual Memory allocations via the Windows Heap Manager as well as the Windows base memory allocator, VirtualAlloc.

1. Windows Heap Tracing by process (highly detailed, high overhead)
2. Windows Heap Snapshots by process
3. Heap 'Sampling': Trace VirtualAlloc across the system

Windows Heap Tracing

• Either download and unzip a recent Release, or clone the Repository: <> Code ↓
  

• MSO-Scripts\TraceHeap Start <options>
  options:
  ◦ Trace a currently running process: -ProcessId <PID>
  ◦ Trace a process at launch: -EXE <Name>.exe
  ◦ Trace multiple processes at launch: -EXE <Name1>.exe,<Name2>.exe,...
  

  Exercise the code.

• MSO-Scripts\TraceHeap Stop

• MSO-Scripts\TraceHeap View

  To trace and analyze via Heap Snapshots, add the -Snap switch to each of the commands above.

List all options:

• MSO-Scripts\TraceHeap -?

Most applications use the Windows Heap Manager to allocate and manage memory. The Heap Manager acquires blocks of Virtual Memory via the base Windows allocator, VirtualAlloc, to subdivide and distribute to the app/process.

Standard C / C++ allocators (new, malloc, etc.) use the Windows Heap. In fact, ALL memory allocations within Windows applications will go through either the Windows Heap or ultimately through VirtualAlloc.

There are two different ETW providers for tracing the Windows Heap Manager: Heap Tracing and Heap Snapshots
There is also an ETW provider for tracing the base allocator VirtualAlloc.

The most detailed view of Windows Heap activity is to record every single allocation and free.

• Chart the rise and fall of Heap memory usage.
• Separately track outstanding allocations vs. freed allocations.
• Expose the code responsible for each allocation via stackwalk.

Trace the Windows Heap activity of a running process:

• MSO-Scripts\TraceHeap Start -ProcessId <PID>   Get the Process ID (PID) from Task Manager or Process Explorer.
  Exercise the code.
• MSO-Scripts\TraceHeap Stop
• MSO-Scripts\TraceHeap View   Launches WPA.

Trace the Windows Heap activity of a process as it launches:

• MSO-Scripts\TraceHeap Start -EXE <Name>.exe
  Launch the app.
• MSO-Scripts\TraceHeap Stop -WPA   Also launches WPA.

Screenshot: Heap Tracing

The Heap Chart/Table answers the question: What code is allocating memory via Windows Heap!?

Legend
Type: AIFI = "Allocated In, Freed Inside the trace",
         AIFO = "Allocated In, Freed Outside the trace" (any leaks would be included here), etc.
Lifetime: Time elapsed from HeapAlloc to HeapFree (or end of trace).
Count _Sum : Total count of all Windows Heap allocations logged in that part of the trace.
Size _Sum : Sum-total size of all Windows Heap allocations (the heap churn) in that part of the trace.
Count _{Peak Outstanding} : Total count of unfreed Windows Heap allocations at the highest point, with time stamp.
Size _{Peak Outstanding} : Total size of unfreed Windows Heap allocations at the highest point, with time stamp.

See Also

• Exploring Process Heaps using WPA
• Understanding heap data using Windows Performance Analyzer (Video) (Microsoft Build 2011)

Imagine the Windows Heap keeping a database of all outstanding (unfreed) allocations:
address, size, allocation stack walk, etc.
Each HeapAlloc goes into the database. Each HeapFree gets removed from the database.

Imagine also that you can snapshot that Heap database: You get a view of all outstanding Heap allocations at specific points in time. Some of those allocations (in the final snapshot) may be leaks, since they're not yet freed...

And finally, imagine that you can take a Diff (difference) of two Heap Snapshots, revealing which allocations were added or removed during the interval between them.

Capture Heap Snapshots of a running process:

• MSO-Scripts\TraceHeap Start -Snap -ProcessId <PID>   Get the Process ID (PID) from Task Manager or Process Explorer.
  Exercise the code.
• MSO-Scripts\TraceHeap Stop -Snap
• MSO-Scripts\TraceHeap View -Snap   Launches WPA.

Capture Heap Snapshots of a process as it launches:

• MSO-Scripts\TraceHeap Start -Snap -EXE <Name>.exe
  Launch the app.
• MSO-Scripts\TraceHeap Stop -Snap -WPA   Also launches WPA.

How to Diff two Heap Snapshots in WPA:

• Select two rows from different snapshots (see the 'Instance' column).
• Right-click a selected row and choose: Diff View
• Scroll down to find a Difference table of the two Heap Snapshots.

Screenshot: Heap Snapshot Diff

Diff (difference) of 2 Heap Snapshots in WPA: Which allocations were added and which were removed between the snapshots?

See also: Record a Heap Snapshot

Imagine a low overhead way to "sample" the growth of the Windows Heap within a process: Every heap allocation has a chance of being sampled, proportional to the size of the allocation.

As the Windows Heap grows, it requests blocks of Virtual Memory via VirtualAlloc which it can then subdivide for allocation. (When the requested size of a Heap allocation is greater than 64 KB, it goes directly to VirtualAlloc.) Thus tracing VirtualAlloc effectively samples the growth of the Windows Heap.

Capture VirtualAlloc activity of all running processes:

• MSO-Scripts\TraceHeap Start -Lean
  Exercise the code.
• MSO-Scripts\TraceHeap Stop
• MSO-Scripts\TraceHeap View -Lean
  In the 'Commit Stack Tag' column find "Windows Heap". (This requires using the -Lean switch with the View command.)

Screenshot: Tracing VirtualAlloc

The VirtualAlloc Chart/Table: What code is allocating memory via VirtualAlloc by way of the Windows Heap, etc.?

Legend
Type: AIFI = "Allocated In, Freed Inside the trace",
         AIFO = "Allocated In, Freed Outside the trace" (any leaks would be included here), etc.
Commit Stack Tag: With View -Lean: it's the allocation origin (Windows Heap, VirtualAlloc directly, GDI, ...),
else it's the thread type.
Commit Stack: Execution stack which used VirtualAlloc to allocated committed virtual memory.
Count _Sum : Total count of all virtual memory allocations logged in that part of the trace.
Size _Sum : Sum-total size of all virtual memory allocations (the memory churn) in that part of the trace.
Count _{Peak Outstanding} : Total count of unfreed virtual memory allocations at the highest point, with time stamp.
Size _{Peak Outstanding} : Total size of unfreed virtual memory allocations at the highest point, with time stamp.
Impacting Size _Sum : Difference between the (outstanding) allocation size at end minus at the start of the trace (or of the current zoom window).

• Expose Heap Memory Activity on Windows with TraceHeap (MSO-Scripts) Video
• Recording for Heap Analysis (Heap Tracing)
• Record a Heap Snapshot
• Track Process Memory Allocations (VirtualAlloc & HeapAlloc)
• Improve app performance by reducing the use of memory and disk space