Risk Management
In a perfect world we'd have infinite time and budget to test everything. In reality, deadlines loom and teams must make hard choices about what to test and what to skip. This topic covers how to make those choices safely.
~12 min read
The core ideas, explained
The Definition of Risk
A risk is an uncertain event in the future that could have a negative impact. In software testing, the Risk Level is calculated as Likelihood times Impact. Likelihood is how probable it is that the problem will actually happen (usually high for complex code or inexperienced developers). Impact is how catastrophic the damage would be if it does happen (financial loss, ruined reputation, physical harm).
A spelling error on the "About Us" page has a high likelihood, but a very low impact - Low Risk overall. A miscalculation in a heart pacemaker's timing has a low likelihood, but a fatal impact - High Risk overall.
Product Risks (Quality Risks)
Product risks relate directly to the software being built. If the software goes live with this risk realized, it negatively affects the end-user or the business. Common types include the software failing to perform its intended function, security vulnerabilities like data breaches, poor performance under load, and poor usability.
A major Product Risk is that a user can transfer more money than they actually have in their account, resulting in massive financial loss for the bank.
Project Risks
Project risks relate to the management, environment, and execution of the project itself. If realized, they impact the project's schedule, budget, or quality - but they aren't flaws in the software itself. Organizational risks include key testers quitting, budget cuts, or management demanding a faster release. Technical risks include a test environment crashing or automation tools failing. Supplier risks include third-party APIs delivered late or contractors failing to meet standards.
You're scheduled to begin System Testing on Monday, but the database administrator is out sick and hasn't set up the test data. This is a Project Risk because it halts your ability to work, even though the software's code hasn't changed.
Risk-Based Testing (RBT)
Risk-Based Testing uses the identified Product Risks to guide the entire testing process, answering "where should we focus our effort?" For prioritization, high-risk items are tested first, so if the project runs out of time, only low-risk items are left untested. For depth of testing, high-risk items get intensive, thorough testing using multiple techniques like BVA, Decision Tables, and automated regression, while low-risk items get light, informal testing like a quick exploratory pass. For reporting, test reports focus on risk mitigated, not just raw numbers run.
An e-commerce site has a "Checkout Payment" module (High Risk) and a "Change Profile Avatar" module (Low Risk). RBT dictates spending 80% of your time testing the payment module using rigorous techniques, and 20% of your time clicking through the avatar uploader.
The Risk Management Process
Managing risk is a continuous, four-step lifecycle. Risk Identification brainstorms what could go wrong using checklists, expert interviews, and retrospectives. Risk Assessment categorizes the risks and calculates their Risk Level (Likelihood times Impact). Risk Mitigation takes action to reduce the likelihood or the impact - testing itself is a mitigation activity, since finding and fixing bugs before release reduces the likelihood of them happening in production. Risk Monitoring continuously watches the risks throughout the project, since a low risk can suddenly become a high risk if the code changes.
A team brainstorms that "the payment gateway might time out" (Identification), rates it Medium likelihood and High impact (Assessment), writes intensive timeout tests to catch the bug before release (Mitigation), and keeps re-checking that rating every sprint as the payment code changes (Monitoring).
Key points to remember
- Product Risk = the software fails (affects the user).
- Project Risk = the team/process fails (affects the schedule/budget).
- Risk Level = Likelihood x Impact - you must measure both to know how dangerous a risk is.
- Testing is risk mitigation - we test to reduce the likelihood of product risks causing damage in the real world.
- RBT drives prioritization - always execute the tests covering the highest risk features first.
Terminology
A few terms from this topic worth knowing precisely.
A risk's overall danger, calculated as Likelihood (how probable the problem is) multiplied by Impact (how bad the damage would be if it happened).
A risk directly related to the software being built - such as a functional failure, security vulnerability, or poor performance - that harms the end-user or business if realized.
A risk related to the management, environment, or execution of the project itself - organizational, technical, or supplier-related - that threatens schedule, budget, or quality without being a flaw in the software.
An approach that uses identified product risks to guide test prioritization, depth of testing, and reporting - the highest-risk items get tested first and most thoroughly.
The first step of risk management - brainstorming what could go wrong using checklists, expert interviews, and retrospectives.
Categorizing identified risks and calculating their Risk Level (Likelihood times Impact).
Taking action to reduce a risk's likelihood or impact - testing itself is a mitigation activity, since finding and fixing bugs before release reduces the likelihood they reach production.
Continuously watching identified risks throughout the project, since a risk's level can change - a module once deemed low-risk might become high-risk after a rewrite.
Summary
Risk management ensures testing is focused and strategic. By differentiating between Project Risks (threats to the schedule and team) and Product Risks (threats to the software's quality), teams can identify what threatens success. By calculating the Risk Level (Likelihood x Impact), they can implement Risk-Based Testing to prioritize test execution, ensuring the most dangerous parts of the software are validated first.
| Concept | One-line memory hook |
|---|---|
| Risk Level Formula | Risk = Likelihood × Impact |
| Product Risk | The software is broken (e.g. bad calculation, security flaw) |
| Project Risk | The work is blocked (e.g. test server down, tester is sick) |
| Risk-Based Testing (RBT) | Test the most dangerous stuff first and deepest |
| Risk Mitigation | Taking action to reduce danger - testing is mitigation |
| Risk Management Steps | Identify → Assess → Mitigate → Monitor |
Check your understanding
20 questions, easy to hard - click an option to see if you got it right.