InteractiveMap of Operations Research
Back to Optimization Core
MethodAdvanced02.06
Optimization Core

Dynamic Programming

Sequential decisions decomposed into stages and states.

Overview

Dynamic Programming focuses on sequential decisions decomposed into stages and states. In the map of OR, it connects State, Bellman equation, Finite & infinite horizon to decisions that must be modeled, solved, explained, and revised as evidence changes.

DP underlies inventory control, MDPs, reinforcement learning, and optimal control. The Bellman equation expresses optimal value recursively. The practical use case is clearest in Shortest paths, Inventory, Revenue management, Reinforcement learning, where the method helps turn constraints and tradeoffs into a decision artifact someone can inspect.

Core ideas

State

The state must contain enough information for the next decision; missing state creates invalid recursions.

Bellman equation

The recursive value equation is the organizing principle for sequential decisions.

Finite & infinite horizon

Finite & infinite horizon is a core checkpoint for Dynamic Programming: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

Approximate DP

Approximate DP is a core checkpoint for Dynamic Programming: define it concretely, attach units or rules where possible, and test whether stakeholders interpret it the same way.

How to use it

  1. 1Start with Shortest paths: write the decision, time horizon, actors, and objective in operational language.
  2. 2Translate the problem into State, Bellman equation, and Finite & infinite horizon; define units and data sources for each one.
  3. 3Build a small instance of Dynamic Programming that can be solved or simulated by hand inspection before using full production data.
  4. 4Compare the recommendation against a baseline policy, not just against mathematical optimality.
  5. 5Document assumptions, sensitivity results, and the conditions under which the recommendation should be revisited.

Applications

Shortest pathsInventoryRevenue managementReinforcement learning
  • Shortest paths: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Inventory: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Revenue management: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.
  • Reinforcement learning: compare feasible policies, quantify the operating tradeoffs, and make the assumptions behind the recommendation visible.

Common pitfalls

  • Applying Dynamic Programming because the label sounds appropriate while leaving the actual decision boundary vague.
  • Treating State as a technical detail instead of a modeling choice that affects the recommendation.
  • Reporting one answer without showing sensitivity to demand, capacity, costs, or behavioral assumptions.
  • Ignoring implementation details such as data quality, explainability, ownership, and how users will override bad recommendations.

Resources