Interactive companion

One config dict, six estimators, and the defaults that decide the answer

mlsynth puts the whole ladder of single-treated-unit estimators behind one interface. That makes it easy to fit six of them on the Brexit referendum, and it makes something else easy to miss: several of the library's default settings move the answer further than the choice of estimator does. Everything below is computed in Python by analysis.py and read from a file. Nothing is estimated in the browser.

Part I — the ladder

1. The counterfactual

Pick a stage. Each one is a single mlsynth class fitted with the same five config fields; the dashed line is the synthetic United Kingdom it produces and the shaded band after the referendum is the estimated shortfall.

2. Who is in the blend?

The unit weights, read straight off result.donor_weights. Four of the six recipes produce nearly the same blend. Only the augmented synthetic control is allowed to go negative, because it is the one stage that drops the simplex constraint.

3. Which quarters count?

Synthetic difference-in-differences is the one stage that also weights the pre-treatment quarters. Difference-in-differences implicitly gives all eighty-six of them the same weight. SDID fits them instead, and on an outcome that behaves like a random walk it puts almost everything on one.

4. The ladder, side by side

Every stage's answer against the 2.4% previously published for this dataset. Hover a bar for the call that produced it and, where they differ, the R edition's number.

Part II — the package

5. Anatomy of a fit

Every estimator in the library takes a dictionary and returns an object with the same accessors. Learn these two columns once and the other eighty-six estimators cost nothing. These are the real values from the synthetic-control fit in panel 1, not an illustration.

The config goes in

VanillaSC({
}).fit()

    The result comes out

      Where estimators deviate

      6. A name that means two things

      This is the trap most likely to cost you an afternoon, and it raises no error.

      mlsynth.DSC

      Distributional synthetic control (Gunsilius 2023). Matches Wasserstein quantiles across micro-level data and returns quantile treatment effects. It wants a different dataset and answers a different question.

      Not the estimator in this post.

      TSSC(method="MSCa")

      Demeaned synthetic control (Doudchenko and Imbens 2016; Ferman and Pinto 2021). Weights on the simplex plus a constant offset, fitted on aggregate panel data.

      This is the post's DSC — stage 2.

      Import the wrong one and nothing warns you. The neighbours are just as close: SCD is a differencing estimator, DRSC is distribution-regression, DSCAR is auto-regressive and MEDSC does mediation. Names in this library are mnemonics, not definitions — check the class docstring before you trust the acronym.

      7. The defaults dial

      The shaded band is how much the answer moves as you climb the entire ladder, from the lowest stage to the highest. Each capsule is how much it moves when you leave one option at its default instead of setting it. A capsule wider than the band is a setting that matters more than the choice of estimator.

      Look at one of them

      8. Three meanings of "control for"

      The literature contains three different answers to "add a covariate to synthetic difference-in-differences", so mlsynth takes them as a dict keyed by method. They are not variations on one estimator. They are three estimators, and on this panel they disagree by more than the whole ladder does.

      And now the honest column

      Pre-treatment fit error for the same specifications. The route that lands closest to the published 2.4% is also the one that tracks the UK worst before the referendum.

      Part III — the evidence

      9. The fire drill

      Move the treatment date back to a quarter when nothing happened, rebuild the counterfactual on data up to that point only, and compare with what actually occurred. The true effect is zero, so every estimate is pure error and lower is better. Switch the grading horizon to see why the published ranking of the three SDID variants does not survive.

      10. Placebo in space

      The other falsification test: hand the treatment to each donor in turn and refit. If the UK's post-referendum gap is nothing special, it will sit inside the crowd.

      11. The event study

      SDID reports a whole path alongside the headline number, in result.event_study. The pre-treatment half is the falsification test a single ATT cannot give you: it should sit on zero, and it does.

      12. The inference menu

      One estimator, one fit, six values of inference=. They disagree about the width of the interval and one of them does not run at all — which is worth knowing before you build a result around it.

      13. The solver's fingerprint

      Two cells in panel 4 disagree with the R edition. Neither is a bug. The synthetic-control objective on this panel has a condition number around 7.5 × 105 — a long, nearly flat valley of near-optimal weights — and the two libraries stop walking down it in different places.