Research · Offline RL · 2026
ESS / OPE Diagnostics
A first-author paper (ICML 2026 DEMO workshop) making the case that effective sample size measures how concentrated the importance weights are, not how uncertain an off-policy estimate is, backed by controlled experiments, exact ground truth, and coverage checks.
Problem
People lean on effective sample size as a reliability check for off-policy estimates, even when the estimator, the reward variance, or the bias makes that reading misleading.
Approach
I set up controlled bandit and tabular-MDP problems with known answers, lined effective sample size up against other estimators, and measured how well the confidence intervals actually cover the truth.
Outcome
The work became a DEMO 2026 poster at the ICML workshop, and the repository turns the argument into diagnostics you can run yourself.