Characterization Tests: Getting Legacy Code Under Test
Legacy code is not old code. It is code you are afraid to change, and the fear is rational: without tests, every edit is a guess about behavior that only production can confirm.
Characterization tests break that deadlock. They do not describe what the code should do - they record what it does right now, so that a refactoring which changes the answer fails loudly on your machine instead of quietly on a customer's.
This is one of three deep dives under Proven Techniques to Tackle Tech Debt. The companion pages are the framework refactoring catalog and dependency untangling.
The Legacy Code Deadlock
Michael Feathers named the trap in Working Effectively with Legacy Code: to change code safely you need tests, and to write tests you usually need to change the code. A five-hundred-line method that constructs its own database connection, reads the system clock, and writes to a static cache cannot be instantiated in a test harness at all. So the tests never get written, the method keeps growing, and every release carries the same risk it did last year.
The way out has three moves, and they always happen in this order:
Find a seam
A place where you can change behavior without editing the code in place - a constructor parameter, an overridable method, a module resolution hook.
Pin the behavior
Write tests that assert what the code currently returns, including the parts that look like bugs. You are recording, not judging.
Then refactor
Now the recipes in the refactoring catalog are safe to apply, because a structural change that alters behavior turns a test red.
A characterization test that fails on day one is not a broken test. It means your understanding of the code was wrong, which is the single most valuable thing this exercise produces. Record the real answer, add a comment saying it looks wrong, and file the bug separately. Fixing it in the same change destroys your ability to tell a refactoring mistake from an intentional fix.
Writing a Characterization Test
The mechanics are deliberately dumb. Call the code with an input, assert something you know is wrong, run the test, and let the failure message tell you the real answer. Paste that answer into the assertion. Repeat until the interesting branches are covered. There is no design work here at all, which is exactly why it can be done on a codebase you do not yet understand.
# THE CODE: nobody knows what it does at the boundaries
# pricing.py - untouched since 2014, no tests, business critical
def calculate_discount(customer_type, order_total, coupon_code, years_active):
discount = 0
if customer_type == "premium":
discount = 0.15
if years_active > 5:
discount += 0.05
elif customer_type == "business":
discount = 0.20 if order_total > 1000 else 0.10
if coupon_code and coupon_code.startswith("SAVE"):
discount += int(coupon_code[4:]) / 100
if discount > 0.35:
discount = 0.35
return round(order_total * (1 - discount), 2)# THE CHARACTERIZATION SUITE: record, do not judge
# test_pricing_characterization.py
import pytest
from pricing import calculate_discount
# Step 1: write the assertion wrong on purpose.
# assert calculate_discount("premium", 100.0, None, 2) == 0
# FAILED: assert 85.0 == 0
# Step 2: paste the real answer back in. That is the whole technique.
@pytest.mark.parametrize("customer_type,total,coupon,years,expected", [
("premium", 100.0, None, 2, 85.00), # base premium 15%
("premium", 100.0, None, 9, 80.00), # loyalty adds 5%
("business", 100.0, None, 1, 90.00), # small business order
("business", 2000.0, None, 1, 1600.00), # large business order
("standard", 100.0, None, 1, 100.00), # unknown type = no discount
("premium", 100.0, "SAVE10", 9, 70.00), # coupon stacks on loyalty
("premium", 100.0, "SAVE99", 9, 65.00), # 35% cap holds
])
def test_current_behavior(customer_type, total, coupon, years, expected):
"""Records behavior as of the refactoring start date. NOT a spec."""
assert calculate_discount(customer_type, total, coupon, years) == expected
# Behavior that looks like a defect. Recorded anyway, bug filed separately.
# Do NOT fix this in the same change as the refactoring - if you do, you can
# no longer tell a refactoring mistake from an intentional behavior change.
def test_coupon_without_digits_raises():
"""BUG-4471: 'SAVE' with no number raises ValueError instead of ignoring."""
with pytest.raises(ValueError):
calculate_discount("premium", 100.0, "SAVE", 1)
def test_negative_total_is_not_rejected():
"""BUG-4472: refunds pass through the discount ladder unchecked."""
assert calculate_discount("premium", -100.0, None, 9) == -80.00Choosing inputs when the input space is large
- Follow the branches, not the arguments. Coverage tooling in "branch" mode tells you which paths your inputs have never executed. Aim for every branch once before you aim for anything else.
- Take the boundaries. Zero, one, negative, empty string, null, the exact threshold value and one either side. Legacy bugs cluster on comparison operators.
- Mine production. Real inputs from logs or a sanitized database export beat invented ones, because they contain the combinations that actually occur - including the ones nobody designed for.
- Check the recording with mutation testing. Tools like mutmut, PIT, or Stryker deliberately break the code and see whether your suite notices. A characterization suite that survives mutation is a suite that would not have caught your refactoring mistake either.
Seams: Testing Code That Refuses to Be Tested
A seam is a place where you can change what the program does without editing the code at that place. Every hard-to-test class is hard to test for the same reason: it decides for itself where its collaborators come from. Find the seam and that decision moves to the caller, which in a test means it moves to you.
Object seam
The collaborator arrives through a parameter, constructor, or property. The most useful seam and the one to create if none exists.
Link seam
The build or runtime decides which implementation is loaded - module resolution, assembly binding, classpath order. Useful when you cannot edit the file at all.
Preprocessing seam
Something rewrites the code before it runs - macros, bundler aliases, bytecode instrumentation. Powerful, invisible, and the last resort.
// BEFORE: no seam - the class builds its own world
public class InvoiceProcessor
{
public InvoiceResult Process(int invoiceId)
{
// Hard-wired database. Cannot run without a live server.
var db = new SqlConnection(ConfigurationManager
.ConnectionStrings["Prod"].ConnectionString);
// Hard-wired clock. Test results change at midnight.
var today = DateTime.Now;
// Hard-wired email. A test run sends real mail to real customers.
var mailer = new SmtpClient("smtp.corp.local");
var invoice = LoadInvoice(db, invoiceId);
if (invoice.DueDate < today)
{
invoice.Status = "OVERDUE";
mailer.Send(BuildOverdueNotice(invoice));
}
return Save(db, invoice);
}
}// AFTER: three object seams, zero behavior change
public interface IClock { DateTime Now { get; } }
public interface IInvoiceStore { Invoice Load(int id); InvoiceResult Save(Invoice i); }
public interface INotifier { void Send(Notice notice); }
public class InvoiceProcessor
{
private readonly IInvoiceStore _store;
private readonly IClock _clock;
private readonly INotifier _notifier;
public InvoiceProcessor(IInvoiceStore store, IClock clock, INotifier notifier)
{
_store = store; _clock = clock; _notifier = notifier;
}
// Preserved so existing callers compile unchanged. This is the whole
// trick: the seam is added WITHOUT touching a single call site.
public InvoiceProcessor()
: this(new SqlInvoiceStore(), new SystemClock(), new SmtpNotifier()) { }
public InvoiceResult Process(int invoiceId)
{
var invoice = _store.Load(invoiceId);
if (invoice.DueDate < _clock.Now)
{
invoice.Status = "OVERDUE";
_notifier.Send(BuildOverdueNotice(invoice));
}
return _store.Save(invoice);
}
}
// The test that was impossible five minutes ago:
[Fact]
public void OverdueInvoice_IsMarkedAndNotified()
{
var store = new FakeInvoiceStore(new Invoice { DueDate = new DateTime(2026, 1, 1) });
var clock = new FixedClock(new DateTime(2026, 7, 25));
var notifier = new RecordingNotifier();
new InvoiceProcessor(store, clock, notifier).Process(42);
Assert.Equal("OVERDUE", store.Saved.Status);
Assert.Single(notifier.Sent);
}Notice what the "after" version does not do: it does not rename anything, restructure the logic, or change a single call site. Adding a seam should be the most boring commit in the repository. Save the interesting changes for after the tests exist, and see the dependency untangling guide for what to do once every collaborator arrives through the front door.
Sprout, Wrap, and Extract-and-Override
Sometimes you have to add behavior to a method you cannot get under test today, and the deadline is real. These three techniques let the new code be fully tested even while the old code stays untested, so at least the debt stops growing.
Sprout method
Write the new logic as a new, fully tested method, then add a single call to it from the untested monster. The monster grows by one line, which is reviewable, and every bit of new behavior has coverage from birth. Use this by default for new requirements landing in old code.
Sprout class
Same idea, one size up. When the new behavior needs its own collaborators or state, put it in a new class and have the legacy method instantiate it. You now own a tested island inside untested territory, and later extractions have somewhere to move to.
Wrap method
Rename the original method, then create a new method with the original name that calls both the renamed original and your new code. Callers are untouched, the new behavior is testable in isolation, and the wrapper documents the sequence explicitly instead of burying it in the middle of a long method.
Extract and override
Move the untestable part - the network call, the clock read, the static lookup - into its own protected method, then subclass the type in your test and override that one method. It creates a seam in a class you are not otherwise allowed to restructure, without adding constructor parameters that ripple through every call site.
Golden master and approval testing are the blunt instruments for code with no clean unit boundary at all - a report generator, a batch job, a rendering pipeline. Capture the entire output for a set of inputs, store it as an approved file in version control, and have the test diff the new output against it. It gives you a wide safety net in an afternoon. The cost is that a failure tells you something changed without telling you what, so keep the inputs small and named, and never approve a diff you have not read.
Step Zero: Decide What to Pin First
You cannot characterize an entire legacy system, and you should not try. Target the code where risk and change frequency overlap: the functions with the most branches, sitting in the files that appear in the most commits. The scripts below find both. The complexity scanner ranks your most dangerous functions, which is exactly the order to write characterization tests in, and the marker hunter surfaces the places where a previous developer already left a note saying this part is fragile.
PowerShell Debt Hunter Toolkit
Automated scripts to find and quantify technical debt in your codebase
You don't need to guess where the debt is - tools can measure it. Here are two practical PowerShell scripts you can run right now on any of your repositories to quantify your debt and create actionable backlog items.
Script 1: The TODO Hunter
We all write TODO, FIXME, HACK, or BUG comments with good intentions. This script finds them all so you can move them into your backlog as real tickets.
<#
.SYNOPSIS
Hunts for technical debt markers (TODO, FIXME, HACK, BUG, OPTIMIZE) across your codebase
.DESCRIPTION
Scans all code files recursively and reports lines containing debt markers.
Outputs results to console with line numbers and context for easy backlog creation.
.PARAMETER Path
Root directory to scan (default: current directory)
.PARAMETER OutputCsv
Optional: Export results to CSV file for import into Jira/Linear/etc.
.EXAMPLE
Find-TechDebt -Path "C:\Projects\MyLegacyApp"
.EXAMPLE
Find-TechDebt -Path "." -OutputCsv "debt-report.csv"
#>
function Find-TechDebt {
param(
[string]$Path = ".",
[string]$OutputCsv = ""
)
Write-Host "[SCAN] Scanning for technical debt markers..." -ForegroundColor Cyan
Write-Host "[DIR] Path: $Path`n" -ForegroundColor Gray
# Debt keywords to search for
$debtKeywords = @("TODO", "FIXME", "HACK", "BUG", "OPTIMIZE", "XXX", "REFACTOR")
$pattern = $debtKeywords -join "|"
# File extensions to scan (customize for your stack)
$extensions = @("*.cs", "*.js", "*.ts", "*.tsx", "*.ps1", "*.sql", "*.py", "*.java", "*.rb", "*.go", "*.php")
try {
$results = Get-ChildItem -Path $Path -Recurse -Include $extensions -ErrorAction SilentlyContinue |
Select-String -Pattern $pattern -CaseSensitive:$false |
Select-Object @{
Name='File'; Expression={$_.Filename}
}, @{
Name='Line'; Expression={$_.LineNumber}
}, @{
Name='Type'; Expression={
if ($_.Line -match "TODO") { "TODO" }
elseif ($_.Line -match "FIXME") { "FIXME" }
elseif ($_.Line -match "HACK") { "HACK" }
elseif ($_.Line -match "BUG") { "BUG" }
elseif ($_.Line -match "OPTIMIZE") { "OPTIMIZE" }
elseif ($_.Line -match "XXX") { "XXX" }
elseif ($_.Line -match "REFACTOR") { "REFACTOR" }
}
}, @{
Name='Content'; Expression={$_.Line.Trim()}
}, @{
Name='Path'; Expression={$_.Path}
}
# Group by type for summary
$summary = $results | Group-Object Type |
Select-Object @{Name='Type'; Expression={$_.Name}}, @{Name='Count'; Expression={$_.Count}} |
Sort-Object Count -Descending
# Display results
Write-Host "[STATS] SUMMARY" -ForegroundColor Yellow
Write-Host "=======================================" -ForegroundColor Gray
$summary | Format-Table -AutoSize
Write-Host "`n[LOG] DETAILED FINDINGS" -ForegroundColor Yellow
Write-Host "=======================================" -ForegroundColor Gray
$results | Format-Table File, Line, Type, Content -AutoSize
# Export to CSV if requested
if ($OutputCsv) {
$results | Export-Csv -Path $OutputCsv -NoTypeInformation
Write-Host "`n[OK] Results exported to: $OutputCsv" -ForegroundColor Green
}
# Final summary
$totalDebt = ($results | Measure-Object).Count
Write-Host "`n[TARGET] TOTAL DEBT MARKERS FOUND: $totalDebt" -ForegroundColor Cyan
if ($totalDebt -gt 100) {
Write-Host "[WARN] HIGH DEBT ALERT: You have over 100 debt markers. Consider scheduling a tech debt sprint." -ForegroundColor Red
} elseif ($totalDebt -gt 50) {
Write-Host "[WARN] MODERATE DEBT: Start chipping away with the Boy Scout Rule." -ForegroundColor Yellow
} else {
Write-Host "[OK] MANAGEABLE DEBT: Keep up the good work!" -ForegroundColor Green
}
} catch {
Write-Error "Error scanning for debt: $_"
}
}
# Example usage:
# Find-TechDebt -Path "C:\Projects\MyApp"
# Find-TechDebt -Path "." -OutputCsv "tech-debt-report.csv"How to Use This Script:
- Save script as
Find-TechDebt.ps1 - Open PowerShell in your project root
- Run:
.\Find-TechDebt.ps1 - Review results and create Jira tickets from the highest-priority TODOs
- Optional: Export to CSV and import directly into your issue tracker
Script 2: The Complexity Scanner
Cyclomatic complexity measures how many paths (if/else/switch/loops) are in your code. High complexity = brittle, bug-prone code. This script uses PSScriptAnalyzer to find your most dangerous functions.
Prerequisites: This script requires the PSScriptAnalyzer module. Install it once with:
Install-Module -Name PSScriptAnalyzer -Scope CurrentUser -Force<#
.SYNOPSIS
Scans PowerShell scripts for high cyclomatic complexity (code that's too complex)
.DESCRIPTION
Uses PSScriptAnalyzer to identify functions with complexity > threshold.
High complexity (15+) indicates code that's hard to test, understand, and maintain.
.PARAMETER Path
File or directory to scan
.PARAMETER ComplexityThreshold
Minimum complexity to report (default: 15, industry standard for "too complex")
.EXAMPLE
Find-ComplexCode -Path "C:\Scripts\MyScript.ps1"
.EXAMPLE
Find-ComplexCode -Path "C:\Projects\Automation" -ComplexityThreshold 10
.NOTES
Cyclomatic Complexity Scale:
1-10: Simple, easy to test
11-20: Moderate complexity
21-50: High complexity - refactor recommended
51+: Very high - urgent refactor needed
#>
function Find-ComplexCode {
param(
[Parameter(Mandatory=$true)]
[string]$Path,
[int]$ComplexityThreshold = 15
)
Write-Host "[ANALYZE] Analyzing code complexity..." -ForegroundColor Cyan
Write-Host "[DIR] Path: $Path" -ForegroundColor Gray
Write-Host "[TARGET] Threshold: $ComplexityThreshold`n" -ForegroundColor Gray
try {
# Check if PSScriptAnalyzer is installed
if (-not (Get-Module -ListAvailable -Name PSScriptAnalyzer)) {
Write-Host "[FAIL] PSScriptAnalyzer not found. Installing..." -ForegroundColor Red
Install-Module -Name PSScriptAnalyzer -Scope CurrentUser -Force
Write-Host "[OK] PSScriptAnalyzer installed`n" -ForegroundColor Green
}
Import-Module PSScriptAnalyzer
# Get all PowerShell files
$files = if (Test-Path -Path $Path -PathType Leaf) {
@(Get-Item $Path)
} else {
Get-ChildItem -Path $Path -Recurse -Include *.ps1, *.psm1
}
$allResults = @()
foreach ($file in $files) {
Write-Host " Scanning: $($file.Name)" -ForegroundColor Gray
# Run PSScriptAnalyzer
$results = Invoke-ScriptAnalyzer -Path $file.FullName -IncludeRule PSAvoidUsingCmdletAliases, PSAvoidUsingWMICmdlet, PSAvoidUsingPositionalParameters
# Parse for complexity (note: PSScriptAnalyzer doesn't directly measure complexity,
# but we can analyze function length and structure as a proxy)
$content = Get-Content $file.FullName -Raw
$functions = [regex]::Matches($content, 'function\s+[\w-]+\s*{[^}]*}', [System.Text.RegularExpressions.RegexOptions]::Singleline)
foreach ($func in $functions) {
$funcText = $func.Value
$funcName = if ($funcText -match 'function\s+([\w-]+)') { $matches[1] } else { "Unknown" }
# Count complexity indicators (if, elseif, switch, while, for, foreach, catch)
$complexity = ([regex]::Matches($funcText, '\b(if|elseif|switch|while|for|foreach|catch)\b')).Count + 1
if ($complexity -ge $ComplexityThreshold) {
$allResults += [PSCustomObject]@{
File = $file.Name
Function = $funcName
Complexity = $complexity
Lines = ($funcText -split "`n").Count
Severity = if ($complexity -gt 50) { "CRITICAL" }
elseif ($complexity -gt 20) { "HIGH" }
else { "MODERATE" }
Path = $file.FullName
}
}
}
}
if ($allResults.Count -eq 0) {
Write-Host "`n[OK] No high-complexity functions found! Your code is clean." -ForegroundColor Green
return
}
# Display results sorted by complexity
Write-Host "`n[STATS] HIGH COMPLEXITY FUNCTIONS" -ForegroundColor Yellow
Write-Host "===========================================================" -ForegroundColor Gray
$allResults | Sort-Object Complexity -Descending | Format-Table File, Function, Complexity, Lines, Severity -AutoSize
# Summary by severity
$summary = $allResults | Group-Object Severity |
Select-Object @{Name='Severity'; Expression={$_.Name}}, @{Name='Count'; Expression={$_.Count}}
Write-Host "`n[TARGET] SUMMARY BY SEVERITY" -ForegroundColor Yellow
Write-Host "=======================================" -ForegroundColor Gray
$summary | Format-Table -AutoSize
# Recommendations
Write-Host "`n[TIP] RECOMMENDATIONS" -ForegroundColor Cyan
Write-Host "=======================================" -ForegroundColor Gray
$critical = ($allResults | Where-Object Severity -eq "CRITICAL").Count
if ($critical -gt 0) {
Write-Host "[!] CRITICAL: $critical functions with complexity > 50" -ForegroundColor Red
Write-Host " Action: Refactor immediately. These are bug factories." -ForegroundColor Red
}
$high = ($allResults | Where-Object Severity -eq "HIGH").Count
if ($high -gt 0) {
Write-Host "[WARN] HIGH: $high functions with complexity 21-50" -ForegroundColor Yellow
Write-Host " Action: Add to tech debt backlog. Break into smaller functions." -ForegroundColor Yellow
}
$moderate = ($allResults | Where-Object Severity -eq "MODERATE").Count
if ($moderate -gt 0) {
Write-Host "[INFO] MODERATE: $moderate functions with complexity 15-20" -ForegroundColor Cyan
Write-Host " Action: Apply Boy Scout Rule when touching these files." -ForegroundColor Cyan
}
} catch {
Write-Error "Error analyzing complexity: $_"
}
}
# Example usage:
# Find-ComplexCode -Path "C:\Projects\MyApp"
# Find-ComplexCode -Path ".\MyScript.ps1" -ComplexityThreshold 10Understanding Complexity Scores:
- 1-10: Simple code, easy to test and understand
- 11-20: Moderate complexity, manageable
- 21-50: High complexity - refactoring recommended
- 51+: Very high - these are your biggest tech debt items
Pro Tip: Functions over 50 complexity are nearly impossible to test thoroughly. Break them into smaller, single-responsibility functions.
Next Steps After Running These Scripts
- Export results to CSV for easy import into Jira/Linear/Azure DevOps
- Create tickets for high-severity items (complexity > 50, or TODOs over 6 months old)
- Add "Fix complexity in UserService.ProcessOrder" to your tech debt backlog
- Use findings to prioritize your 20% time work
- Re-run monthly to track improvements over time
Related Resources
Techniques Hub
The decision guide: which technique fits which situation, and how much permission each one needs.
Refactoring Catalog
Once the behavior is pinned, these are the framework-specific recipes that are now safe to apply.
Testing Strategies
The broader testing picture: pyramid shape, coverage targets, flaky test debt, and CI quality gates.
Frequently Asked Questions
A characterization test records what a piece of code currently does, rather than what anyone thinks it should do. You call the code, observe the real output, and assert that exact value - bugs included. Its only job is to detect change. That makes it the right first move on legacy code, because you can write one without understanding the requirements, without a specification, and without asking anyone what the code was supposed to do. Once it exists, any refactoring that alters behavior fails immediately instead of six weeks later in production.
A seam is a place where you can change the behavior of a program without editing the code at that place. There are three kinds. An object seam supplies a collaborator through a parameter, constructor, or property. A link seam lets the build or runtime choose which implementation is loaded, through module resolution, assembly binding, or classpath order. A preprocessing seam rewrites the code before it runs, using macros, bundler aliases, or bytecode instrumentation. Object seams are the ones to prefer and the ones to create when none exists.
Not in the same change. Record the buggy behavior exactly as it is, add a comment and a ticket number explaining that it looks wrong, and fix it in a separate commit after the refactoring lands. The reason is diagnostic, not bureaucratic: if a test changes at the same time as the code structure, a red build no longer tells you whether you broke something or intentionally changed it. There is also a real chance that something downstream already depends on the bug, and you want that discovery isolated to one small change.
Enough to cover every branch you are about to move, which is usually far less than the whole file. A global coverage percentage is the wrong target here; the useful question is whether every path through the specific code you are restructuring is exercised by at least one test. Branch coverage tooling answers that directly. Then confirm the tests would actually notice a mistake by running a mutation testing tool against them - a suite that never fails when the code is deliberately broken will not save you either.
It is characterization testing for code with no clean unit boundary - report generators, batch jobs, rendering pipelines. You run the system over a fixed set of inputs, capture the entire output, review it once, and commit it as the approved result. Later runs diff against that file. The advantage is enormous coverage for very little work. The disadvantage is that a failure tells you something changed without telling you what, so keep each input small and clearly named, and never approve a new output file without reading the diff.
Convert them rather than delete them. Once you understand what the code should do, rewrite each recorded expectation as an intentional assertion with a name that states the rule, and drop the ones that only pinned incidental output. Keeping a large suite of unexplained recorded values around long term is its own debt, because nobody dares change any of them. The characterization suite is scaffolding: valuable while the building is going up, and a hazard if you leave it attached afterwards.
Tests First, Then the Refactor
Once the behavior is pinned, pick the recipe that matches your stack and start moving code with confidence.
Open the Refactoring Catalog