Evaluate Donors

Run donor evaluation on pre-treatment data after applying substantive eligibility exclusions.

geosc donors --config data-config/donor_eval_config.yaml --jobs -1

The evaluator aligns each treatment-donor pair by time, then calculates Pearson correlation and its p-value, root mean squared error (RMSE), percentage RMSE, mean absolute percentage error (MAPE), and normalised dynamic time-warping (DTW) distance. It normalises metrics within each treated geography, combines them using adaptive weights, and ranks complete rows.

Review:

  • overlap_count, treatment_coverage, and donor_coverage before scores;
  • metrics_complete and metric_failure_reasons;
  • correlation, rmse, mape, and dtw, not only composite_score;
  • warning_flags, quality bands, concentration, and effective donor count;
  • maps only as a geographic diagnostic, not as proof of comparability.

selected_weight is normalised from positive composite scores among the recommended donors. Its role is design_stage_recommendation_weight; it is not a fitted SparseSC counterfactual weight.

The evaluator’s output is a screening artefact. If it changes the donor pool, filter the canonical panel and rerun power and inference. To render maps, see Use Shapemaps.

Keep the source file pre-treatment only

Pairwise metrics use pre-treatment rows. However, the adaptive threshold and metric-weight initialisation reads the whole donor source CSV before that filter. Supplying post-treatment rows can therefore affect screening thresholds and scores. For an outcome-blind design, give the donor stage a separate pre-treatment-only CSV with the same approved geography universe. Retain its hash and cutoff. The bundled full-panel examples exercise software paths and do not establish outcome-blind donor selection.

Metric weights are adaptive implementation settings, not user-configurable YAML weights. Review the donor quality explanation.