InteractiveMap of Operations Research
Back to Foundations of OR
MethodFoundations01.06
Foundations of OR

Model Validation & Sensitivity

Check whether a model is useful, credible, and stable.

Overview

Model Validation & Sensitivity focuses on check whether a model is useful, credible, and stable. In the map of OR, it connects Calibration, Backtesting, Scenario analysis to decisions that must be modeled, solved, explained, and revised as evidence changes.

Validation compares model behavior with reality, tests assumptions, and studies how recommendations change under parameter, data, or scenario shifts. The practical use case is clearest in Policy models, Digital twins, Revenue models, Risk analysis, where the method helps turn constraints and tradeoffs into a decision artifact someone can inspect.

Core ideas

Calibration

Calibration is a core checkpoint for Model Validation & Sensitivity: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

Backtesting

Backtesting is a core checkpoint for Model Validation & Sensitivity: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

Scenario analysis

Scenario analysis is a core checkpoint for Model Validation & Sensitivity: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

Sensitivity

Sensitivity is a core checkpoint for Model Validation & Sensitivity: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

Stress testing

Stress testing is a core checkpoint for Model Validation & Sensitivity: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

How to use it

  1. 1Start with Policy models: write the decision, time horizon, actors, and objective in operational language.
  2. 2Translate the problem into Calibration, Backtesting, and Scenario analysis; define units and data sources for each one.
  3. 3Build a small instance of Model Validation & Sensitivity that can be solved or simulated by hand inspection before using full production data.
  4. 4Compare the recommendation against a baseline policy, not just against mathematical optimality.
  5. 5Document assumptions, sensitivity results, and the conditions under which the recommendation should be revisited.

Applications

Policy modelsDigital twinsRevenue modelsRisk analysis
  • Policy models: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Digital twins: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Revenue models: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Risk analysis: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.

Common pitfalls

  • Applying Model Validation & Sensitivity because the label sounds appropriate while leaving the actual decision boundary vague.
  • Treating Calibration as a technical detail instead of a modeling choice that affects the recommendation.
  • Reporting one answer without showing sensitivity to demand, capacity, costs, or behavioral assumptions.
  • Ignoring implementation details such as data quality, explainability, ownership, and how users will override bad recommendations.

Resources