09 / Systems and reinforcement learning · 2026
MDP dynamic programming with explicit C++ ownership
RAII, sanitizers and multi-OS CI turn four crashing tests into a defensible implementation.
The initial test executables all failed with memory errors. Ownership was modernized and the unsupported 270% performance claim removed.
Evidence register
4/4
tests
Debug and Release locally
2
operating systems
Linux and Windows CI
ASan
memory checks
Sanitizer job passes
01 / Problem
Correct equations are not enough when unsafe ownership makes every automated test crash.
02 / Approach
Replace ambiguous raw ownership with RAII containers, modernize CMake and exercise the algorithms under debug, release and sanitizers.
03 / Outcome
All four tests pass across configurations and the README now limits claims to measured evidence.
System trace
How the evidence is produced.
Repaired memory management, build portability, automated validation and technical documentation.
- 01Finite MDP → owned structures
- 02Bellman operators → iterations
- 03Policies and values → assertions
- 04CMake matrix → CI
Validation scope
Debug 4/4, Release 4/4, Linux/Windows workflows and sanitizer checks pass.
Known limitation
The test suite covers small finite MDPs, not large-scale performance or approximate reinforcement learning.
What is inspectable
- Ownership is visible in types.
- Undefined behavior is checked automatically.
- Unsupported speedup claim removed.
Next proof to add
- 01Add property-based Bellman tests.
- 02Benchmark larger sparse MDPs.
- 03Document convergence tolerances.
Main stack