------------------------------------------------------------------------------- name: log: /Users/carlosmendez/Documents/GitHub/starter-academic-v501/content > /post/stata_sc/analysis.log log type: text opened on: 5 Oct 2026, 12:01:16 . . di _newline(2) . di "============================================" ============================================ . di " Synthetic Control Method (SCM) Tutorial" Synthetic Control Method (SCM) Tutorial . di " Abadie, Diamond, and Hainmueller (2010)" Abadie, Diamond, and Hainmueller (2010) . di " $S_DATE $S_TIME" 5 Oct 2026 12:01:16 . di "============================================" ============================================ . . * Apple Silicon compatibility check . di _newline . di "Machine type:" Machine type: . display c(machine_type) Macintosh (Intel 64-bit) . di _newline . di "Note: On Macs with Apple Silicon, the synth plugin needs Rosetta 2." Note: On Macs with Apple Silicon, the synth plugin needs Rosetta 2. . di "Right-click Stata, choose Get Info, and check Open using Rosetta." Right-click Stata, choose Get Info, and check Open using Rosetta. . di "The machine type above shows the architecture that Stata uses." The machine type above shows the architecture that Stata uses. . . . *=================================================== . * PART 1: DATA AND BASELINE SCM . * Dataset: smoking_sc.dta (39 states x 31 years) . *=================================================== . . . *--------------------------------------------------- . * Section 1: Load and explore the dataset . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 1: DATA LOADING AND EXPLORATION" SECTION 1: DATA LOADING AND EXPLORATION . di "========================================" ======================================== . . use "https://github.com/quarcs-lab/data-open/raw/master/isds/smoking_sc.dta", > clear (Tobacco Sales in 39 US States) . . * Inspect variable labels and storage types . describe Contains data from https://github.com/quarcs-lab/data-open/raw/master/isds/smok > ing_sc.dta Observations: 1,209 Tobacco Sales in 39 US States Variables: 7 23 Mar 2007 21:59 ------------------------------------------------------------------------------- Variable Storage Display Value name type format label Variable label ------------------------------------------------------------------------------- state long %14.0g state state no year float %9.0g year cigsale float %9.0g cigarette sale per capita (in packs) lnincome float %9.0g log state per capita gdp beer float %9.0g beer consumption per capita age15to24 float %9.0g percent of state population aged 15-24 years retprice float %9.0g retail price of cigarettes ------------------------------------------------------------------------------- Sorted by: state year . . * Check means, standard deviations, minimums, and maximums . summarize Variable | Obs Mean Std. dev. Min Max -------------+--------------------------------------------------------- state | 1,209 20 11.25929 1 39 year | 1,209 1985 8.947973 1970 2000 cigsale | 1,209 118.8932 32.7674 40.7 296.2 lnincome | 1,014 9.861634 .1706769 9.397449 10.48662 beer | 546 23.4304 4.22319 2.5 40.4 -------------+--------------------------------------------------------- age15to24 | 819 .175472 .0151589 .1294482 .2036753 retprice | 1,209 108.3419 64.38199 27.3 351.2 . . * Show the first six observations . list in 1/6 +------------------------------------------------------------------+ | state year cigsale lnincome beer age15~24 retprice | |------------------------------------------------------------------| 1. | Alabama 1970 89.8 . . .1788618 39.6 | 2. | Alabama 1971 95.4 . . .1799278 42.7 | 3. | Alabama 1972 101.1 9.498476 . .1809939 42.3 | 4. | Alabama 1973 102.9 9.550107 . .1820599 42.1 | 5. | Alabama 1974 108.2 9.537163 . .183126 43.1 | |------------------------------------------------------------------| 6. | Alabama 1975 111.7 9.540031 . .1841921 46.6 | +------------------------------------------------------------------+ . . * Declare the panel: state is the unit, and year is the time variable . xtset state year Panel variable: state (strongly balanced) Time variable: year, 1970 to 2000 Delta: 1 unit . . * Summarize the between and within variation of each variable . xtsum Variable | Mean Std. dev. Min Max | Observations -----------------+--------------------------------------------+---------------- state overall | 20 11.25929 1 39 | N = 1209 between | 11.40175 1 39 | n = 39 within | 0 20 20 | T = 31 | | year overall | 1985 8.947973 1970 2000 | N = 1209 between | 0 1985 1985 | n = 39 within | 8.947973 1970 2000 | T = 31 | | cigsale overall | 118.8932 32.7674 40.7 296.2 | N = 1209 between | 26.47924 63.83548 213.0645 | n = 39 within | 19.74754 49.5287 202.0287 | T = 31 | | lnincome overall | 9.861634 .1706769 9.397449 10.48662 | N = 1014 between | .1304321 9.584268 10.20371 | n = 39 within | .1119729 9.588297 10.14454 | T = 26 | | beer overall | 23.4304 4.22319 2.5 40.4 | N = 546 between | 4.012627 13.72143 36.7 | n = 39 within | 1.45541 2.22326 31.60897 | T = 14 | | age15~24 overall | .175472 .0151589 .1294482 .2036753 | N = 819 between | .0071531 .1593386 .1927357 | n = 39 within | .0134118 .1326057 .1989868 | T = 21 | | retprice overall | 108.3419 64.38199 27.3 351.2 | N = 1209 between | 8.751969 88.31936 127.3258 | n = 39 within | 63.79926 23.21613 339.6194 | T = 31 . . * Identify the state code of California . label list state: 1 Alabama 2 Arkansas 3 California 4 Colorado 5 Connecticut 6 Delaware 7 Georgia 8 Idaho 9 Illinois 10 Indiana 11 Iowa 12 Kansas 13 Kentucky 14 Louisiana 15 Maine 16 Minnesota 17 Mississippi 18 Missouri 19 Montana 20 Nebraska 21 Nevada 22 New Hampshire 23 New Mexico 24 North Carolina 25 North Dakota 26 Ohio 27 Oklahoma 28 Pennsylvania 29 Rhode Island 30 South Carolina 31 South Dakota 32 Tennessee 33 Texas 34 Utah 35 Vermont 36 Virginia 37 West Virginia 38 Wisconsin 39 Wyoming . . di _newline . di "Panel: 39 states over 31 years (1970–2000), 1,209 observations" Panel: 39 states over 31 years (1970–2000), 1,209 observations . di "Treatment unit: California (state == 3)" Treatment unit: California (state == 3) . di "Treatment period: 1989, the first year with Proposition 99 in effect" Treatment period: 1989, the first year with Proposition 99 in effect . di "Donor pool: 38 control states" Donor pool: 38 control states . di _newline . di "Variables:" Variables: . di " cigsale - Cigarette sales per capita (packs)" cigsale - Cigarette sales per capita (packs) . di " lnincome - Log of state GDP per capita" lnincome - Log of state GDP per capita . di " age15to24 - Share of the population aged 15–24 (a fraction)" age15to24 - Share of the population aged 15–24 (a fraction) . di " retprice - Average retail price of cigarettes" retprice - Average retail price of cigarettes . di " beer - Beer consumption per capita" beer - Beer consumption per capita . . . *--------------------------------------------------- . * Section 2: Visualize raw trends . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 2: RAW TRENDS" SECTION 2: RAW TRENDS . di " California vs. Donor Pool Average" California vs. Donor Pool Average . di "========================================" ======================================== . . preserve . . * Create an indicator for California . gen california = (state == 3) . . * Collapse to mean sales by year and group . collapse (mean) cigsale, by(year california) . . twoway (connected cigsale year if california==1, /// > msymbol(O) mcolor("106 155 204") lcolor("106 155 204") /// > lwidth(medthick)) /// > (connected cigsale year if california==0, /// > msymbol(T) mcolor("128 128 128") lcolor("128 128 128") /// > lwidth(medium) lpattern(dash)), /// > xline(1989, lcolor("217 119 87") lpattern(dash) lwidth(medium)) /// > ytitle("Cigarette Sales (packs per capita)") xtitle("Year") /// > legend(order(1 "California" 2 "Donor Pool Average") /// > position(6)) /// > title("Cigarette Sales: California vs. Donor Pool") /// > note("Source: Abadie, Diamond, and Hainmueller (2010)." /// > "The vertical line marks Proposition 99 (1989).") /// > graphregion(color(white)) plotregion(color(white)) /// > name(raw_trends, replace) . . graph export "stata_sc_raw_trends.png", replace width(2400) (file stata_sc_raw_trends.png not found) file stata_sc_raw_trends.png written in PNG format . . restore . . di "Figure saved: stata_sc_raw_trends.png" Figure saved: stata_sc_raw_trends.png . di _newline . di "Sales in California started near the donor average in 1970." Sales in California started near the donor average in 1970. . di "By 1988, they were about 24 packs per capita below that average." By 1988, they were about 24 packs per capita below that average. . di "After Proposition 99, the gap widened further." After Proposition 99, the gap widened further. . di "A simple average is therefore a poor counterfactual for California." A simple average is therefore a poor counterfactual for California. . di "The SCM instead weights the control states to match the path of" The SCM instead weights the control states to match the path of . di "sales in California before 1989." sales in California before 1989. . . . *--------------------------------------------------- . * Section 3: Baseline synthetic control estimate . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 3: BASELINE SCM ESTIMATE" SECTION 3: BASELINE SCM ESTIMATE . di "========================================" ======================================== . . di _newline . di "Command: synth2 with nested optimization and allopt" Command: synth2 with nested optimization and allopt . di _newline . di "Predictors:" Predictors: . di " lnincome - Log GDP per capita (a demand factor)" lnincome - Log GDP per capita (a demand factor) . di " age15to24 - Share of the population aged 15–24 (demographic)" age15to24 - Share of the population aged 15–24 (demographic) . di " retprice - Retail price of cigarettes (a price factor)" retprice - Retail price of cigarettes (a price factor) . di " beer - Beer consumption per capita (a complementary good)" beer - Beer consumption per capita (a complementary good) . di " cigsale(1988), cigsale(1980), cigsale(1975)" cigsale(1988), cigsale(1980), cigsale(1975) . di " - Cigarette sales in three pre-treatment years" - Cigarette sales in three pre-treatment years . di _newline . di "Options:" Options: . di " trunit(3) - Treated unit: California (state==3)" trunit(3) - Treated unit: California (state==3) . di " trperiod(1989) - Treatment onset: January 1989" trperiod(1989) - Treatment onset: January 1989 . di " xperiod(1980(1)1988) - Covariate averaging window: 1980–1988" xperiod(1980(1)1988) - Covariate averaging window: 1980–1988 . di " nested - Nested optimization over V and W" nested - Nested optimization over V and W . di " allopt - Three starting points for the nested search" allopt - Three starting points for the nested search . di _newline . di "Running baseline SCM... (this may take a few minutes)" Running baseline SCM... (this may take a few minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested allopt sav > egraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7556 > 7 Number of Covariates = 7 R-squared = 0.9743 > 4 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0000 10.0766 9.8588 -2.16% 9.8292 -2.45 > % age15to24 | 0.5459 0.1735 0.1735 -0.01% 0.1725 -0.59 > % retprice | 0.0174 89.4222 89.4108 -0.01% 87.2661 -2.41 > % beer | 0.0031 24.2800 24.2278 -0.21% 23.6553 -2.57 > % cigsale(1988) | 0.0049 90.1000 91.6677 1.74% 113.8237 26.33 > % cigsale(1980) | 0.0066 120.2000 120.5017 0.25% 138.0895 14.88 > % cigsale(1975) | 0.4221 127.1000 127.1112 0.01% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3340 Nevada | 0.2350 Montana | 0.2020 Colorado | 0.1610 Connecticut | 0.0680 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.9945 -7.5945 1990 | 77.8000 87.5039 -9.7039 1991 | 68.7000 82.1751 -13.4751 1992 | 67.5000 81.6075 -14.1075 1993 | 63.4000 81.1897 -17.7897 1994 | 58.6000 80.7295 -22.1295 1995 | 56.4000 78.5023 -22.1023 1996 | 54.5000 77.4827 -22.9827 1997 | 53.8000 77.7123 -23.9123 1998 | 52.3000 74.3976 -22.0976 1999 | 47.2000 73.5711 -26.3711 2000 | 41.6000 67.3550 -25.7550 ------+---------------------------------------------------- Mean | 60.3500 79.3518 -19.0018 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -19.0018. (file stata_sc_bias.gph not found) file stata_sc_bias.gph saved (file stata_sc_weight_vars.gph not found) file stata_sc_weight_vars.gph saved (file stata_sc_weight_unit.gph not found) file stata_sc_weight_unit.gph saved (file stata_sc_pred.gph not found) file stata_sc_pred.gph saved (file stata_sc_eff.gph not found) file stata_sc_eff.gph saved Finished. . . * Display stored results . ereturn list scalars: e(N) = 1209 e(T) = 31 e(T0) = 19 e(T1) = 12 e(K) = 7 e(J) = 39 e(rmse) = 1.755672369538799 e(r2) = .9743364199596987 e(att) = -19.0017671585083 macros: e(panelvar) : "state" e(timevar) : "year" e(varlist) : "cigsale lnincome age15to24 retprice beer cigsale(.." e(depvar) : "cigsale" e(indepvars) : "lnincome age15to24 retprice beer cigsale(1988) ci.." e(unit_all) : "Alabama Arkansas California Colorado Connecticut .." e(unit_tr) : "California" e(unit_ctrl) : "Alabama Arkansas Colorado Connecticut Delaware Ge.." e(time_all) : "1970 1971 1972 1973 1974 1975 1976 1977 1978 1979.." e(time_tr) : "1989" e(time_pre) : "1970 1971 1972 1973 1974 1975 1976 1977 1978 1979.." e(time_post) : "1989 1990 1991 1992 1993 1994 1995 1996 1997 1998.." e(graph) : " bias weight_vars weight_unit pred eff" matrices: e(V_wt) : 7 x 7 e(U_wt) : 5 x 1 e(bal) : 7 x 6 . . * Save matrices before graph operations clear e() . matrix X_balance = e(bal) . matrix W_weights = e(U_wt) . . * Convert .gph files to .png (batch mode compatible) . foreach g in pred eff bias weight_unit weight_vars { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_pred.png not found) file stata_sc_pred.png written in PNG format Figure saved: stata_sc_pred.png (file stata_sc_eff.png not found) file stata_sc_eff.png written in PNG format Figure saved: stata_sc_eff.png (file stata_sc_bias.png not found) file stata_sc_bias.png written in PNG format Figure saved: stata_sc_bias.png (file stata_sc_weight_unit.png not found) file stata_sc_weight_unit.png written in PNG format Figure saved: stata_sc_weight_unit.png (file stata_sc_weight_vars.png not found) file stata_sc_weight_vars.png written in PNG format Figure saved: stata_sc_weight_vars.png . . . *--------------------------------------------------- . * Section 4: Interpret predictor balance and weights . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 4: PREDICTOR BALANCE AND WEIGHTS" SECTION 4: PREDICTOR BALANCE AND WEIGHTS . di "========================================" ======================================== . . * Predictor balance: California vs. Synthetic California . di _newline . di "Predictor Balance (California vs. Synthetic California):" Predictor Balance (California vs. Synthetic California): . di "-------------------------------------------------------" ------------------------------------------------------- . matrix list X_balance X_balance[7,6] Vweight Treated ValueSynth~l BiasSynthe~l lnincome .00004916 10.076559 9.8587684 -2.161355 age15to24 .54587094 .17353238 .17352193 -.00602001 retprice .01741005 89.422223 89.4108 -.01277453 beer .0031354 24.28 24.2278 -.21499293 cigsale(1988) .00490342 90.099998 91.667698 1.7399554 cigsale(1980) .00655685 120.2 120.5017 .25100113 cigsale(1975) .42207419 127.1 127.1112 .00881275 ValueAvera~l BiasAverag~l lnincome 9.8291968 -2.4548245 age15to24 .17251011 -.5890933 retprice 87.266082 -2.411192 beer 23.655263 -2.5730524 cigsale(1988) 113.82368 26.330395 cigsale(1980) 138.08947 14.883093 cigsale(1975) 136.93158 7.7353113 . . di _newline . di "The balance table compares the predictors of California with those" The balance table compares the predictors of California with those . di "of synthetic California, and close values indicate a good match." of synthetic California, and close values indicate a good match. . di "The weights W minimize the V-weighted gap in these predictors." The weights W minimize the V-weighted gap in these predictors. . di "The nested search chooses V to minimize the pre-1989 error in sales." The nested search chooses V to minimize the pre-1989 error in sales. . di "V is poorly identified, so its values are not measures of importance." V is poorly identified, so its values are not measures of importance. . . * Unit weights: which states form synthetic California . di _newline . di "Unit Weights (Donor Pool Contributions):" Unit Weights (Donor Pool Contributions): . di "-----------------------------------------" ----------------------------------------- . matrix list W_weights W_weights[5,1] Weight Utah .334 Nevada .235 Montana .202 Colorado .161 Connecticut .068 . . di _newline . di "The unit weights show which control states form synthetic California." The unit weights show which control states form synthetic California. . di "Five states receive positive weights, and the other 33 receive zero." Five states receive positive weights, and the other 33 receive zero. . di "The weights are nonnegative and sum to one." The weights are nonnegative and sum to one. . di "No single donor needs to resemble California on its own." No single donor needs to resemble California on its own. . di "Only the weighted combination must match California before 1989." Only the weighted combination must match California before 1989. . . . *--------------------------------------------------- . * Section 5: Interpret treatment effects . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 5: TREATMENT EFFECTS" SECTION 5: TREATMENT EFFECTS . di "========================================" ======================================== . . di _newline . di "Treatment Effect Interpretation:" Treatment Effect Interpretation: . di "================================" ================================ . di _newline . di "The 'pred' graph plots actual and synthetic sales in California." The 'pred' graph plots actual and synthetic sales in California. . di "Before 1989, the series are close, with the largest gap in 1970." Before 1989, the series are close, with the largest gap in 1970. . di "This close fit supports the synthetic path as a counterfactual." This close fit supports the synthetic path as a counterfactual. . di "A good fit is necessary for credibility, but it is not sufficient." A good fit is necessary for credibility, but it is not sufficient. . di _newline . di "After 1989, actual sales fall well below the synthetic path." After 1989, actual sales fall well below the synthetic path. . di "The synthetic path estimates sales in California without the law." The synthetic path estimates sales in California without the law. . di "The difference is the estimated effect of Proposition 99 on sales." The difference is the estimated effect of Proposition 99 on sales. . di _newline . di "The 'eff' graph plots this gap, actual minus synthetic, by year." The 'eff' graph plots this gap, actual minus synthetic, by year. . di "The gap grows more negative through the 1990s, but not in every year." The gap grows more negative through the 1990s, but not in every year. . di "Its average over 1989–2000 is the ATT reported in the table above." Its average over 1989–2000 is the ATT reported in the table above. . di _newline . di "The estimand is the ATT, the average effect on the treated unit." The estimand is the ATT, the average effect on the treated unit. . di "The treated unit is California, so the estimate applies to it alone." The treated unit is California, so the estimate applies to it alone. . di "It is not an average effect across all states." It is not an average effect across all states. . . . *=================================================== . * PART 2: INFERENCE AND ROBUSTNESS . * Placebo tests and leave-one-out analysis . *=================================================== . . . *--------------------------------------------------- . * Section 6: In-space placebo test . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 6: IN-SPACE PLACEBO TEST" SECTION 6: IN-SPACE PLACEBO TEST . di "========================================" ======================================== . . di _newline . di "The in-space placebo test applies the SCM to each control state." The in-space placebo test applies the SCM to each control state. . di "Each run treats one control state as if it had adopted the law." Each run treats one control state as if it had adopted the law. . di "The placebo gaps show how large a gap can be without any treatment." The placebo gaps show how large a gap can be without any treatment. . di "A gap for California that is unusual among them is evidence of" A gap for California that is unusual among them is evidence of . di "a real effect rather than a statistical artifact." a real effect rather than a statistical artifact. . di _newline . di "Options:" Options: . di " placebo(unit) - Run the SCM for each control state" placebo(unit) - Run the SCM for each control state . di " cut(2) - Keep placebos with a pre-MSPE at most twice" cut(2) - Keep placebos with a pre-MSPE at most twice . di " that of California (drops poorly fitted ones)" that of California (drops poorly fitted ones) . di " sigf(6) - Six significant figures (default 7) for convergence" sigf(6) - Six significant figures (default 7) for convergence . di " The placebo run omits allopt to save computation time." The placebo run omits allopt to save computation time. . di _newline . di "Running in-space placebo test... (this takes several minutes)" Running in-space placebo test... (this takes several minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested placebo(un > it cut(2)) sigf(6) savegraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7795 > 5 Number of Covariates = 7 R-squared = 0.9741 > 1 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0001 10.0766 9.8527 -2.22% 9.8292 -2.45 > % age15to24 | 0.0020 0.1735 0.1737 0.07% 0.1725 -0.59 > % retprice | 0.0147 89.4222 89.3766 -0.05% 87.2661 -2.41 > % beer | 0.0083 24.2800 24.2236 -0.23% 23.6553 -2.57 > % cigsale(1988) | 0.0142 90.1000 91.3866 1.43% 113.8237 26.33 > % cigsale(1980) | 0.1923 120.2000 120.2357 0.03% 138.0895 14.88 > % cigsale(1975) | 0.7683 127.1000 127.0999 -0.00% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3450 Nevada | 0.2410 Montana | 0.2070 Colorado | 0.1480 Connecticut | 0.0590 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.8201 -7.4201 1990 | 77.8000 87.3789 -9.5789 1991 | 68.7000 81.9182 -13.2182 1992 | 67.5000 81.4061 -13.9061 1993 | 63.4000 81.0228 -17.6228 1994 | 58.6000 80.5678 -21.9678 1995 | 56.4000 78.3083 -21.9083 1996 | 54.5000 77.3429 -22.8429 1997 | 53.8000 77.6174 -23.8174 1998 | 52.3000 74.1877 -21.8877 1999 | 47.2000 73.3950 -26.1950 2000 | 41.6000 67.1478 -25.5478 ------+---------------------------------------------------- Mean | 60.3500 79.1761 -18.8261 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -18.8261. Implementing placebo test using fake treatment unit Alabama...Arkansas...Colora > do...Connecticut...Delaware...Georgia...Idaho...Illinois...Indiana...Iowa...K > ansas...Kentucky...Louisiana...Maine...Minnesota...Mississippi...Missouri...M > ontana...Nebraska...Nevada...NewHampshire...NewMexico...NorthCarolina...North > Dakota...Ohio...Oklahoma...Pennsylvania...RhodeIsland...SouthCarolina...South > Dakota...Tennessee...Texas...Utah...Vermont...Virginia...WestVirginia...Wisco > nsin...Wyoming... In-space placebo test results using fake treatment units: ------------------------------------------------------------------------------- Unit | Pre MSPE Post MSPE Post/Pre MSPE Pre MSPE of Fake Unit/ | Pre MSPE of Treated Unit ---------------+--------------------------------------------------------------- California | 3.1668 391.2533 123.5490 1.0000 Alabama | 5.4170 8.2108 1.5157 1.7106 Arkansas | 4.5587 28.8239 6.3228 1.4395 Colorado | 17.6103 68.6736 3.8996 5.5609 Connecticut | 20.6396 118.9189 5.7617 6.5175 Delaware | 30.3949 499.0694 16.4195 9.5980 Georgia | 1.4610 116.8893 80.0074 0.4613 Idaho | 5.8142 39.1830 6.7392 1.8360 Illinois | 4.3146 89.0552 20.6406 1.3624 Indiana | 14.4145 469.4150 32.5654 4.5518 Iowa | 14.6527 31.5816 2.1553 4.6270 Kansas | 14.1121 9.0349 0.6402 4.4563 Kentucky | 431.7229 1475.7975 3.4184 136.3284 Louisiana | 2.0183 94.1070 46.6279 0.6373 Maine | 8.6989 115.5412 13.2822 2.7469 Minnesota | 14.1736 52.5217 3.7056 4.4757 Mississippi | 4.0894 37.2754 9.1151 1.2913 Missouri | 1.2009 85.1794 70.9308 0.3792 Montana | 5.2861 54.8978 10.3853 1.6692 Nebraska | 4.8287 36.5597 7.5713 1.5248 Nevada | 40.6500 83.4186 2.0521 12.8364 NewHampshire | 3436.5980 134.9018 0.0393 1085.2007 NewMexico | 5.0577 63.7459 12.6036 1.5971 NorthCarolina | 90.2241 67.3144 0.7461 28.4907 NorthDakota | 8.0725 72.3200 8.9588 2.5491 Ohio | 2.9585 12.3535 4.1757 0.9342 Oklahoma | 5.7128 267.8078 46.8786 1.8040 Pennsylvania | 2.7029 6.7073 2.4815 0.8535 RhodeIsland | 87.9904 242.6697 2.7579 27.7854 SouthCarolina | 2.1997 41.2941 18.7727 0.6946 SouthDakota | 7.5688 32.2367 4.2592 2.3901 Tennessee | 5.2043 123.3097 23.6938 1.6434 Texas | 4.6691 239.8559 51.3707 1.4744 Utah | 593.7643 223.2758 0.3760 187.4975 Vermont | 15.3860 116.8473 7.5944 4.8585 Virginia | 2.7825 219.8136 78.9994 0.8786 WestVirginia | 8.1492 242.1734 29.7175 2.5733 Wisconsin | 2.8950 75.2425 25.9901 0.9142 Wyoming | 83.7717 31.8266 0.3799 26.4532 ------------------------------------------------------------------------------- Note: (1) Using all control units, the probability of obtaining a post/pretreatment MSPE ratio as large as California's is 0.0256. (2) Excluding control units with pretreatment MSPE 2 times larger than the treated unit, the probability of obtaining a post/pretreatment MSPE ratio as large as California's is 0.0500. (3) The pointwise p-values below are computed by excluding control units with pretreatment MSPE 2 times larger than the treated unit. (4) There are total 19 units with pretreatment MSPE 2 times larger than the treated unit, including Colorado Connecticut Delaware Indiana Iowa Kansas Kentucky Maine Minnesota Nevada NewHampshire NorthCarolina NorthDakota RhodeIsland SouthDakota Utah Vermont WestVirginia Wyoming. In-space placebo test results using fake treatment units (continued, cutoff = 2 > ): ---------------------------------------------------------------- Time | Treatment Effect p-value of Treatment Effect | Two-sided Right-sided Left-sided ------+--------------------------------------------------------- 1989 | -7.4201 0.0500 1.0000 0.0500 1990 | -9.5789 0.1000 0.9500 0.1000 1991 | -13.2182 0.1500 0.9000 0.1500 1992 | -13.9061 0.1000 0.9500 0.1000 1993 | -17.6228 0.0500 1.0000 0.0500 1994 | -21.9678 0.0500 1.0000 0.0500 1995 | -21.9083 0.0500 1.0000 0.0500 1996 | -22.8429 0.0500 1.0000 0.0500 1997 | -23.8174 0.0500 1.0000 0.0500 1998 | -21.8877 0.1000 0.9500 0.1000 1999 | -26.1950 0.0500 1.0000 0.0500 2000 | -25.5478 0.0500 1.0000 0.0500 ---------------------------------------------------------------- Note: (1) The two-sided p-value of the treatment effect for a particular period is defined as the frequency that the absolute values of the placebo effects are greater than or equal to the absolute value of treatment effect. (2) The right-sided (left-sided) p-value of the treatment effect for a particular period is defined as the frequency that the placebo effects are greater (smaller) than or equal to the treatment effect. (3) If the estimated treatment effect is positive, then the right-sided p-value is recommended; whereas the left-sided p-value is recommended if the estimated treatment effect is negative. (file stata_sc_bias.gph not found) file stata_sc_bias.gph saved (file stata_sc_weight_vars.gph not found) file stata_sc_weight_vars.gph saved (file stata_sc_weight_unit.gph not found) file stata_sc_weight_unit.gph saved (file stata_sc_pred.gph not found) file stata_sc_pred.gph saved (file stata_sc_eff.gph not found) file stata_sc_eff.gph saved (file stata_sc_eff_pboUnit.gph not found) file stata_sc_eff_pboUnit.gph saved (file stata_sc_ratio_pboUnit.gph not found) file stata_sc_ratio_pboUnit.gph saved (file stata_sc_pvalTwo_pboUnit.gph not found) file stata_sc_pvalTwo_pboUnit.gph saved (file stata_sc_pvalRight_pboUnit.gph not found) file stata_sc_pvalRight_pboUnit.gph saved (file stata_sc_pvalLeft_pboUnit.gph not found) file stata_sc_pvalLeft_pboUnit.gph saved Finished. . . * Convert placebo .gph files to .png . foreach g in eff_pboUnit ratio_pboUnit pvalTwo_pboUnit pvalRight_pboUnit pval > Left_pboUnit { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_eff_pboUnit.png not found) file stata_sc_eff_pboUnit.png written in PNG format Figure saved: stata_sc_eff_pboUnit.png (file stata_sc_ratio_pboUnit.png not found) file stata_sc_ratio_pboUnit.png written in PNG format Figure saved: stata_sc_ratio_pboUnit.png (file stata_sc_pvalTwo_pboUnit.png not found) file stata_sc_pvalTwo_pboUnit.png written in PNG format Figure saved: stata_sc_pvalTwo_pboUnit.png (file stata_sc_pvalRight_pboUnit.png not found) file stata_sc_pvalRight_pboUnit.png written in PNG format Figure saved: stata_sc_pvalRight_pboUnit.png (file stata_sc_pvalLeft_pboUnit.png not found) file stata_sc_pvalLeft_pboUnit.png written in PNG format Figure saved: stata_sc_pvalLeft_pboUnit.png . . di _newline . di "In-Space Placebo Results:" In-Space Placebo Results: . di "=========================" ========================= . di _newline . di "eff_pboUnit: Gaps of California and of the 19 retained placebos." eff_pboUnit: Gaps of California and of the 19 retained placebos. . di " The line of California is purple, and the placebo lines are gray." The line of California is purple, and the placebo lines are gray. . di " A real effect should make the purple line an outlier after 1989." A real effect should make the purple line an outlier after 1989. . di _newline . di "ratio_pboUnit: Ranks all 39 units by the post/pre MSPE ratio." ratio_pboUnit: Ranks all 39 units by the post/pre MSPE ratio. . di " A high ratio means a large post-1989 gap relative to the fit" A high ratio means a large post-1989 gap relative to the fit . di " before 1989. A real effect should place California at or near" before 1989. A real effect should place California at or near . di " the top." the top. . di _newline . di "pvalTwo/Right/Left: Placebo-based p-values by year." pvalTwo/Right/Left: Placebo-based p-values by year. . di " Two-sided: tests for an effect of either sign" Two-sided: tests for an effect of either sign . di " Right-sided: tests for a positive effect on sales" Right-sided: tests for a positive effect on sales . di " Left-sided: tests for a negative effect, the relevant case here" Left-sided: tests for a negative effect, the relevant case here . di " With California and 19 retained placebos, p cannot fall below 0.05." With California and 19 retained placebos, p cannot fall below 0.05. . di " The left-sided p-value of California is 0.05 in 8 of 12 years." The left-sided p-value of California is 0.05 in 8 of 12 years. . . . *--------------------------------------------------- . * Section 7: In-time placebo test . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 7: IN-TIME PLACEBO TEST" SECTION 7: IN-TIME PLACEBO TEST . di "========================================" ======================================== . . di _newline . di "The in-time placebo test moves the treatment date back to 1985." The in-time placebo test moves the treatment date back to 1985. . di "This fake date falls four years before the real policy." This fake date falls four years before the real policy. . di "A sound design should show small gaps between 1985 and 1988." A sound design should show small gaps between 1985 and 1988. . di "Large gaps in those years would signal a problem with the design." Large gaps in those years would signal a problem with the design. . di _newline . di "Key changes from the baseline:" Key changes from the baseline: . di " - cigsale(1988) is dropped from the predictors, because 1988" - cigsale(1988) is dropped from the predictors, because 1988 . di " falls after the fake treatment date." falls after the fake treatment date. . di " - xperiod(1980(1)1984) replaces 1980(1)1988, so the covariate" - xperiod(1980(1)1984) replaces 1980(1)1988, so the covariate . di " averages end before the fake treatment date." averages end before the fake treatment date. . di " - placebo(period(1985)) sets the fake treatment year." - placebo(period(1985)) sets the fake treatment year. . di _newline . di "Running in-time placebo test..." Running in-time placebo test... . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1980) cigsale(1975), > trunit(3) trperiod(1989) xperiod(1980(1)1984) nested placebo(period(1985)) sa > vegraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 2.2053 > 0 Number of Covariates = 6 R-squared = 0.9525 > 3 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0000 10.0372 9.8639 -1.73% 9.7892 -2.47 > % age15to24 | 0.0000 0.1815 0.1825 0.55% 0.1814 -0.06 > % retprice | 0.0384 76.2200 76.1523 -0.09% 71.8353 -5.75 > % beer | 0.0000 25.0000 23.0089 -7.96% 23.6947 -5.22 > % cigsale(1980) | 0.9597 120.2000 120.0846 -0.10% 138.0895 14.88 > % cigsale(1975) | 0.0019 127.1000 126.8324 -0.21% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3600 Nevada | 0.2880 Connecticut | 0.1990 Colorado | 0.1020 NewMexico | 0.0500 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Montana Nebraska NewHampshire NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 93.0322 -10.6322 1990 | 77.8000 89.4297 -11.6297 1991 | 68.7000 82.4727 -13.7727 1992 | 67.5000 80.6731 -13.1731 1993 | 63.4000 79.5929 -16.1929 1994 | 58.6000 78.1272 -19.5272 1995 | 56.4000 75.6207 -19.2207 1996 | 54.5000 74.8372 -20.3372 1997 | 53.8000 74.5395 -20.7395 1998 | 52.3000 71.1561 -18.8561 1999 | 47.2000 71.4380 -24.2380 2000 | 41.6000 65.8382 -24.2382 ------+---------------------------------------------------- Mean | 60.3500 78.0631 -17.7131 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -17.7131. Implementing placebo test using fake treatment time 1985... In-time placebo test results using fake treatment time 1985: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1985 | 102.8000 106.1262 -3.3262 1986 | 99.7000 103.2850 -3.5850 1987 | 97.5000 106.1524 -8.6524 1988 | 90.1000 98.4873 -8.3873 1989 | 82.4000 96.5237 -14.1237 1990 | 77.8000 91.9127 -14.1127 1991 | 68.7000 83.7156 -15.0156 1992 | 67.5000 81.4730 -13.9730 1993 | 63.4000 79.7911 -16.3911 1994 | 58.6000 77.9078 -19.3078 1995 | 56.4000 76.2193 -19.8193 1996 | 54.5000 75.2010 -20.7010 1997 | 53.8000 75.1958 -21.3958 1998 | 52.3000 71.9437 -19.6437 1999 | 47.2000 72.2260 -25.0260 2000 | 41.6000 67.1861 -25.5861 ------+---------------------------------------------------- Mean | 69.6437 85.2092 -15.5654 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -15.5654. file stata_sc_bias.gph saved file stata_sc_weight_vars.gph saved file stata_sc_weight_unit.gph saved file stata_sc_pred.gph saved file stata_sc_eff.gph saved (file stata_sc_pred_pboTime1985.gph not found) file stata_sc_pred_pboTime1985.gph saved (file stata_sc_eff_pboTime1985.gph not found) file stata_sc_eff_pboTime1985.gph saved Finished. . . * Convert in-time placebo .gph files to .png . foreach g in pred_pboTime1985 eff_pboTime1985 { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_pred_pboTime1985.png not found) file stata_sc_pred_pboTime1985.png written in PNG format Figure saved: stata_sc_pred_pboTime1985.png (file stata_sc_eff_pboTime1985.png not found) file stata_sc_eff_pboTime1985.png written in PNG format Figure saved: stata_sc_eff_pboTime1985.png . . di _newline . di "In-Time Placebo Results:" In-Time Placebo Results: . di "========================" ======================== . di _newline . di "pred_pboTime1985: Actual and synthetic sales with the fake 1985 date." pred_pboTime1985: Actual and synthetic sales with the fake 1985 date. . di " The fit statistics printed first belong to the reduced model at" The fit statistics printed first belong to the reduced model at . di " the real date, 1989. The command does not print the fit of the" the real date, 1989. The command does not print the fit of the . di " 1985 model, and the in-time table lists its yearly gaps." 1985 model, and the in-time table lists its yearly gaps. . di _newline . di "eff_pboTime1985: Gaps of the 1985 model over time." eff_pboTime1985: Gaps of the 1985 model over time. . di " Gaps in 1985–1988 measure effects at the fake date, before the law." Gaps in 1985–1988 measure effects at the fake date, before the law. . di " They should be small relative to the gaps after 1989." They should be small relative to the gaps after 1989. . di " Gaps after 1989 should stay large, as in the baseline results." Gaps after 1989 should stay large, as in the baseline results. . di _newline . di "Large gaps at the fake date would signal a problem with the design." Large gaps at the fake date would signal a problem with the design. . di "They could reflect overfitting or shocks that hit California early." They could reflect overfitting or shocks that hit California early. . di "Small gaps, in contrast, support the timing of the estimated effect." Small gaps, in contrast, support the timing of the estimated effect. . . . *--------------------------------------------------- . * Section 8: Leave-one-out robustness check . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 8: LEAVE-ONE-OUT ROBUSTNESS" SECTION 8: LEAVE-ONE-OUT ROBUSTNESS . di "========================================" ======================================== . . di _newline . di "The leave-one-out (LOO) test re-estimates the SCM five times." The leave-one-out (LOO) test re-estimates the SCM five times. . di "Each run drops one of the five donors with a positive weight." Each run drops one of the five donors with a positive weight. . di "Results that change sharply without one state would be fragile." Results that change sharply without one state would be fragile. . di "Stable results suggest that no single donor drives the estimate." Stable results suggest that no single donor drives the estimate. . di _newline . di "Options:" Options: . di " loo - Drop each weighted donor in turn" loo - Drop each weighted donor in turn . di " frame(california) - Store the results in a Stata frame" frame(california) - Store the results in a Stata frame . di " savegraph(california, replace) - Save each graph as a .gph file" savegraph(california, replace) - Save each graph as a .gph file . di _newline . di "Running leave-one-out test... (this takes several minutes)" Running leave-one-out test... (this takes several minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested loo frame( > california) savegraph(california, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7832 > 9 Number of Covariates = 7 R-squared = 0.9736 > 5 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0002 10.0766 9.8509 -2.24% 9.8292 -2.45 > % age15to24 | 0.0124 0.1735 0.1736 0.03% 0.1725 -0.59 > % retprice | 0.0113 89.4222 89.3331 -0.10% 87.2661 -2.41 > % beer | 0.0416 24.2800 24.2554 -0.10% 23.6553 -2.57 > % cigsale(1988) | 0.0231 90.1000 91.2783 1.31% 113.8237 26.33 > % cigsale(1980) | 0.4378 120.2000 120.1872 -0.01% 138.0895 14.88 > % cigsale(1975) | 0.4735 127.1000 126.9907 -0.09% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3420 Nevada | 0.2380 Montana | 0.2170 Colorado | 0.1450 Connecticut | 0.0580 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.7304 -7.3304 1990 | 77.8000 87.3001 -9.5001 1991 | 68.7000 81.8829 -13.1829 1992 | 67.5000 81.4287 -13.9287 1993 | 63.4000 81.0450 -17.6450 1994 | 58.6000 80.6229 -22.0229 1995 | 56.4000 78.4191 -22.0191 1996 | 54.5000 77.4316 -22.9316 1997 | 53.8000 77.7288 -23.9288 1998 | 52.3000 74.3255 -22.0255 1999 | 47.2000 73.4654 -26.2654 2000 | 41.6000 67.2107 -25.6107 ------+---------------------------------------------------- Mean | 60.3500 79.2159 -18.8659 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -18.8659. Implementing leave-one-out robustness test that excludes one control unit with > a nonzero weight Utah...Nevada...Montana...Colorado...Connecticut... Leave-one-out robustness test results in the posttreatment period: ------------------------------------------------------- Time | Outcome Synthetic Outcome (LOO) | Actual Synthetic Min Max ------+------------------------------------------------ 1989 | 82.4000 89.7304 88.3892 92.3509 1990 | 77.8000 87.3001 83.5373 89.2205 1991 | 68.7000 81.8829 80.8905 82.4889 1992 | 67.5000 81.4287 80.6239 81.8815 1993 | 63.4000 81.0450 79.7801 82.0592 1994 | 58.6000 80.6229 78.6141 83.3112 1995 | 56.4000 78.4191 75.9901 81.3864 1996 | 54.5000 77.4316 75.0801 80.5833 1997 | 53.8000 77.7288 71.7877 84.4150 1998 | 52.3000 74.3255 71.1668 79.0314 1999 | 47.2000 73.4654 71.5421 77.5396 2000 | 41.6000 67.2107 65.0850 69.9503 ------------------------------------------------------- Note: The last two columns report the minimum and maximum synthetic outcomes when one control unit with a nonzero weight is excluded at a time. ------------------------------------------------------- Time | Treatment Effect Treatment Effect (LOO) | Min Max ------+------------------------------------------------ 1989 | -7.3304 -9.9509 -5.9892 1990 | -9.5001 -11.4205 -5.7373 1991 | -13.1829 -13.7889 -12.1905 1992 | -13.9287 -14.3815 -13.1239 1993 | -17.6450 -18.6592 -16.3801 1994 | -22.0229 -24.7112 -20.0141 1995 | -22.0191 -24.9864 -19.5901 1996 | -22.9316 -26.0833 -20.5801 1997 | -23.9288 -30.6150 -17.9877 1998 | -22.0255 -26.7314 -18.8668 1999 | -26.2654 -30.3396 -24.3421 2000 | -25.6107 -28.3503 -23.4850 ------------------------------------------------------- Note: The last two columns report the minimum and maximum treatment effects when one control unit with a nonzero weight is excluded at a time. (file california_bias.gph not found) file california_bias.gph saved (file california_weight_vars.gph not found) file california_weight_vars.gph saved (file california_weight_unit.gph not found) file california_weight_unit.gph saved (file california_pred.gph not found) file california_pred.gph saved (file california_eff.gph not found) file california_eff.gph saved (file california_pred_loo.gph not found) file california_pred_loo.gph saved (file california_eff_loo.gph not found) file california_eff_loo.gph saved Finished. . . * Combine the seven graphs of the LOO run into one figure . graph combine `e(graph)', cols(2) altshrink /// > title("Leave-One-Out Robustness: Synthetic California") /// > note("The last two panels add the five leave-one-out fits in gray.") /// > graphregion(color(white)) /// > name(loo_combined, replace) . . graph export "stata_sc_loo_combined.png", replace width(2400) (file stata_sc_loo_combined.png not found) file stata_sc_loo_combined.png written in PNG format . di "Figure saved: stata_sc_loo_combined.png" Figure saved: stata_sc_loo_combined.png . . di _newline . di "Leave-One-Out Results:" Leave-One-Out Results: . di "======================" ====================== . di _newline . di "The combined graph has seven panels from this run." The combined graph has seven panels from this run. . di "The first five show the fit with all 38 donors." The first five show the fit with all 38 donors. . di "The last two add the five leave-one-out fits as gray lines." The last two add the five leave-one-out fits as gray lines. . di _newline . di "Similar gaps across the five refits indicate robust results." Similar gaps across the five refits indicate robust results. . di "A large change after dropping one state would reveal that the" A large change after dropping one state would reveal that the . di "synthetic control relies heavily on that state." synthetic control relies heavily on that state. . di "The min and max tables above summarize the range by year." The min and max tables above summarize the range by year. . . . *--------------------------------------------------- . * Section 9: Inspect leave-one-out frame . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 9: LOO FRAME INSPECTION" SECTION 9: LOO FRAME INSPECTION . di "========================================" ======================================== . . di _newline . di "The LOO results are stored in a Stata frame named 'california'." The LOO results are stored in a Stata frame named 'california'. . di "A frame holds an additional dataset in memory." A frame holds an additional dataset in memory. . di "Frames require Stata 16 or later." Frames require Stata 16 or later. . . frame change california . describe Contains data Observations: 1,209 Tobacco Sales in 39 US States Variables: 23 ------------------------------------------------------------------------------- Variable Storage Display Value name type format label Variable label ------------------------------------------------------------------------------- state long %14.0g state state no year float %9.0g year cigsale float %9.0g cigarette sale per capita (in packs) age15to24 float %9.0g percent of state population aged 15-24 years beer float %9.0g beer consumption per capita lnincome float %9.0g log state per capita gdp retprice float %9.0g retail price of cigarettes pred·cigsale float %9.0g prediction of cigsale tr·cigsale float %9.0g treatment effect on cigsale pred·cigsale·~h float %9.0g prediction of cigsale (Utah excluded) generated by 'robustness test' tr·cigsale·rm~h float %9.0g treatment effect on cigsale (Utah excluded) generated by 'robustness test' pred·cigsale~da float %9.0g prediction of cigsale (Nevada excluded) generated by 'robustness test' tr·cigsale·r~da float %9.0g treatment effect on cigsale (Nevada excluded) generated by 'robustness test' pred·cigsale~na float %9.0g prediction of cigsale (Montana excluded) generated by 'robustness test' tr·cigsale·r~na float %9.0g treatment effect on cigsale (Montana excluded) generated by 'robustness test' pred·cigsale·~o float %9.0g prediction of cigsale (Colorado excluded) generated by 'robustness test' tr·cigsale·rm~o float %9.0g treatment effect on cigsale (Colorado excluded) generated by 'robustness test' pred·cigsale·~t float %9.0g prediction of cigsale (Connecticut excluded) generated by 'robustness test' tr·cigsale·rm~t float %9.0g treatment effect on cigsale (Connecticut excluded) generated by 'robustness test pred·cigsale·~n float %9.0g min prediction of cigsale generated by 'robustness test' pred·cigsale·~x float %9.0g max prediction of cigsale generated by 'robustness test' tr·cigsale·lo~n float %9.0g min treatment effect on cigsale generated by 'robustness test' tr·cigsale·lo~x float %9.0g max treatment effect on cigsale generated by 'robustness test' ------------------------------------------------------------------------------- Sorted by: Note: Dataset has changed since last saved. . frame change default . . di _newline . di "The frame stores predictions and effects for the full-pool fit and" The frame stores predictions and effects for the full-pool fit and . di "for each refit, and variable labels name the excluded donor." for each refit, and variable labels name the excluded donor. . di "Four more variables hold the minimum and maximum across refits." Four more variables hold the minimum and maximum across refits. . di "Researchers can use these variables for further analysis." Researchers can use these variables for further analysis. . . . *--------------------------------------------------- . * Section 10: Closing summary . *--------------------------------------------------- . . di _newline(2) . di "============================================" ============================================ . di " ANALYSIS COMPLETE" ANALYSIS COMPLETE . di "============================================" ============================================ . di _newline . di " Estimand: ATT (average treatment effect on the treated)" Estimand: ATT (average treatment effect on the treated) . di " Treated unit: California (state==3)" Treated unit: California (state==3) . di " Treatment: Proposition 99 (effective January 1989)" Treatment: Proposition 99 (effective January 1989) . di _newline . di " Key Findings:" Key Findings: . di " 1. Five control states form a synthetic California that" 1. Five control states form a synthetic California that . di " closely matches its sales before 1989." closely matches its sales before 1989. . di " 2. After 1989, actual sales fall well below the synthetic" 2. After 1989, actual sales fall well below the synthetic . di " path, which indicates a substantial reduction in" path, which indicates a substantial reduction in . di " cigarette sales per capita." cigarette sales per capita. . di " 3. California has the largest post/pre MSPE ratio of all" 3. California has the largest post/pre MSPE ratio of all . di " 39 units, so its gap is unusual among the placebos" 39 units, so its gap is unusual among the placebos . di " (permutation p-value 1/39 = 0.026)." (permutation p-value 1/39 = 0.026). . di " 4. The in-time placebo test finds smaller gaps at the" 4. The in-time placebo test finds smaller gaps at the . di " fake 1985 date than after 1989, but not zero gaps." fake 1985 date than after 1989, but not zero gaps. . di " 5. Dropping any one of the five weighted donors keeps" 5. Dropping any one of the five weighted donors keeps . di " every post-1989 gap negative." every post-1989 gap negative. . di _newline . di " Figures:" Figures: . di " stata_sc_raw_trends.png" stata_sc_raw_trends.png . di " stata_sc_pred.png" stata_sc_pred.png . di " stata_sc_eff.png" stata_sc_eff.png . di " stata_sc_bias.png" stata_sc_bias.png . di " stata_sc_weight_unit.png" stata_sc_weight_unit.png . di " stata_sc_weight_vars.png" stata_sc_weight_vars.png . di " stata_sc_eff_pboUnit.png" stata_sc_eff_pboUnit.png . di " stata_sc_ratio_pboUnit.png" stata_sc_ratio_pboUnit.png . di " stata_sc_pvalTwo_pboUnit.png" stata_sc_pvalTwo_pboUnit.png . di " stata_sc_pvalRight_pboUnit.png" stata_sc_pvalRight_pboUnit.png . di " stata_sc_pvalLeft_pboUnit.png" stata_sc_pvalLeft_pboUnit.png . di " stata_sc_pred_pboTime1985.png" stata_sc_pred_pboTime1985.png . di " stata_sc_eff_pboTime1985.png" stata_sc_eff_pboTime1985.png . di " stata_sc_loo_combined.png" stata_sc_loo_combined.png . di _newline . di " Reference:" Reference: . di " Abadie, A., Diamond, A., and Hainmueller, J. (2010)." Abadie, A., Diamond, A., and Hainmueller, J. (2010). . di " Synthetic control methods for comparative case studies." Synthetic control methods for comparative case studies. . di " Journal of the American Statistical Association 105(490):" Journal of the American Statistical Association 105(490): . di " 493–505. https://doi.org/10.1198/jasa.2009.ap08746" 493–505. https://doi.org/10.1198/jasa.2009.ap08746 . di "============================================" ============================================ . di _newline . di "=== Script completed successfully ===" === Script completed successfully === . . log close name: log: /Users/carlosmendez/Documents/GitHub/starter-academic-v501/content > /post/stata_sc/analysis.log log type: text closed on: 5 Oct 2026, 12:16:03 -------------------------------------------------------------------------------