Skip to content

A node-plus-generators curtailment metric with member-wise ENS inputs

The curtailment metric can now take ensemble members and it reproduces experiment 255, but 34% of the stored solar-farm forecasts it was checked on list their quantiles out of order. The metric prices what it would cost to cap the output of solar farms whenever a forecast says the substation they feed would export more power than a limit. It is step 2 of the load-shift decision program, which asks what knowing about load shifts is worth to the forecast. We asked whether the metric, studies.node_curtailment, can read ensemble members (the 51 runs that make up one ECMWF weather forecast) for a substation's demand and for each solar farm, and whether it then gives the answer that experiment 255 found. We added two functions. We re-scored experiment 255's rows at 0.5 times each substation's earlier maximum export twice: once from the stored quantiles, and once from 101 members drawn from them. We compared both with the avoidable cost (the cost above a perfect forecast) that experiment 255 stored for each forecast, and with its five comparisons between pairs of forecasts.

Verdict: partly supported. The scoring from stored quantiles matches experiment 255 exactly. The scoring from members keeps the sign and the 95% interval verdict of all five comparisons (largest difference 0.58 points of the baseline's avoidable cost, under the 1 point allowed). It misses the 1% allowed on two forecasts' avoidable costs, which differ by 2.0% and 1.1% (the climatology and manual-heuristic forecasts). The cause is in the stored forecasts: 26,344 of 77,897 solar-farm forecast rows (34%) store quantiles that cross, and a set of members cannot reproduce a crossing row. The metric now sorts each row's quantiles by default.

Design

  • Scoring nodes. The step-1 mapping search found no file linking a farm to a Primary, and the six farms are probably connected at the bulk supply point. The generator-aware nodes are therefore the bulk supply points D13 (farms A, C, E) and D17 (farms B, D, F), with their child Primaries as demand. Primary-level curtailment is a demand-only relative proxy: it ranks forecasts and states no money saved.
  • Member path. net_flow_members pairs the demand's and each generator's members by member number and returns the node's metered flow per member. quantiles_from_members reduces members to the q<percent> columns that node_curtailment reads, interpolating linearly between sorted members. rearrange_quantiles sorts a stored row's quantiles; NodeCurtailmentAssumptions switches it on by default, and the result counts the rows that crossed.
  • Reproduction check. Pre-registered in the plan: tolerance 0.01% for the quantile path and 1% per arm for the member path, contrasts within 1 point, same sign, same 95% interval verdict. The script draws 101 members (probabilities 0, 0.01, ..., 1), because only then does linear interpolation between sorted members return the stored quantile at every whole-percent level. Post hoc: the quantile path with sorting off, and the member path on sorted quantiles.

Numbers

  • Quantile path, sorting off. Every arm's avoidable cost equals experiment 255's, with largest difference 0.0000%, and every contrast equals it (e.g. H5 minus XGBoost GBP -62,407).
  • Member path on the stored quantiles. Avoidable cost differs from experiment 255 by -2.00% (climatology floor), -1.09% (manual-heuristic floor), -0.79% (plant), -0.78% (H5 with zero history), -0.76% (H5), and -0.69% (XGBoost). Contrasts differ by at most 0.58 points of the baseline's avoidable cost, keep their signs, and keep their 95% interval verdicts. The 1% arm limit is missed by the two floor arms.
  • Why it misses. 26,344 of 77,897 farm-forecast rows cross in every forecast arm (158,064 rows summed over the six arms). With the quantiles sorted first, the member path and the quantile path agree to 0.0000% on every arm.
  • Sorting moves the costs. With sorting on (the default), avoidable cost differs from experiment 255's by -3.66% (climatology floor), -1.90% (manual-heuristic floor), +1.08% (XGBoost), +0.72% (H5), +0.62% (plant), and -0.59% (H5 with zero history). Contrasts differ by at most 0.81 points; H5 minus XGBoost is GBP -69,766 against -62,407.
  • Rows. Experiment 255's causal rows, 2025-09-01 to 2026-06-30, 12 to 36 hour leads, headline price GBP 2,000 per MWh, Curtailment Limit 0, f = 0.5.
  • Tests. 88 tests in test_node_curtailment.py pass, including hand-worked member cases: members pair by number not by row order, a missing generator member drops only that member, and a 20-member case gives exactly 3.25 MWh planned and GBP 6,500.

Checks

  • Median minus mean forecast per site, and the mean-forecast error beside each contrast: not applicable, because the experiment trains no forecast.
  • Triggers. Trigger 1 fired (a pre-registered tolerance was missed). The explanation, crossing stored quantiles, was tested directly by sorting first, which removes the whole gap. No audit was run for the member path's miss; the maintainer or manager can ask for one. The other six triggers did not fire.

Caveats and next steps

  • We found no member-wise inputs for a real run. In the experiment folders searched, the stored farm outputs of experiments 229 and 246 hold no member column, and the stored 51-member demand forecasts of experiment 125 cover 3 of D13's 7 child Primaries and 3 of D17's 9. A member-wise run at D13 and D17 therefore needs new frames.
  • Step 3 needs no member frames. Its forecasts (climatology, the manual heuristic, XGBoost best guess, and the GAM stack) are quantile forecasts, so step 3 uses the quantile path. Member-wise frames are needed only for step 5's comparison of ensemble-driven forecasts.
  • Sorting changes costs by up to 3.7%, and it swaps H5 and H5 with zero history as the cheapest forecast (GBP 1,878,600 and 1,861,933 sorted; 1,865,144 and 1,872,880 stored).
  • The metric reads each generator's median only, so a better farm tail still does not show in its cost.

Data-processing note

The crossing rows counted on this page belong to three generators that are not solar farms. Their stored forecasts have no 0.01 or 0.99 quantile, and experiment 255's capacity cap turned the null padding into the generator's capacity, so the 0.01 quantile sat above the 0.05 quantile in all 26,344 of their rows. The six solar farms' own 51,553 rows have none (see experiment 255).