Demand inputs: experiments
This page lists every experiment tried in the theme, newest first. An experiment that belongs mainly to another theme but bears on this one is marked "also relevant".
| Experiment | Hypothesis | Verdict | Production status | Take-home message | Relevance |
|---|---|---|---|---|---|
| 00103 differentiable-physics-1-data-look | The trial area's power records hold enough clean, demand-only history for the series to go ahead as designed: 6 MW Primaries with at least 12 months before July 2025, and 8 MVA Primaries with over 3 site-years each of rows with no sunlight. | supported | not-for-production | The data lets the series go ahead, with three changes: start 13 of the 16 Primaries later than their first record (19.4 months later at D4), fix the lockdown window in advance because the data does not isolate it from drift, and count eligible long-history MW sites at each tuning year's first day (5 at 2024-07-01, so 4 of 5 must improve in Part 2's tuning contrast). The power timestamps show no 30-minute error on either side of the 2026-03-26 fix: at the 6 metered photovoltaic series the median offsets are +0.9 minutes before it and +4.7 after, and the correction is right to within 3 minutes, but offsets at the Primaries are not resolved beyond 0 to +8 minutes. | also relevant |
| 00094 explicit-heating-offset-primary-demand | For the 16 Primary substation demand series, adding a per-series heating term to the XGBoost model's base margin lowers the forecast error in cold hours (ENS 2 m temperature below 3 C), because the term extrapolates linearly beyond the coldest training hour and cannot be damped at long leads, whereas trees predict a flat response there. The term is the heating degree-hours (15.5 C minus the temperature, floored at zero) times a per-series, per-hour-of-day coefficient fitted by least squares on the training window only. | small gain | candidate | The heating term lowered the quantile score by 0.143 points of capacity in cold hours at leads up to 48 hours (98.75% interval -0.203 to -0.034), under the 0.3 threshold, and by 0.022 over all rows (-0.042 to +0.000). | primary |
| 00074 an1-same-time-analogues-vs-explicit-solar-forecast-demand | At the 5 Primary substations (substations that feed 11 kV circuits from the 33 kV network) with a clean solar-capacity estimate, when the XGBoost model is refitted every month on the latest 12 months of data (P-roll), an explicit forecast of the unmetered solar output, given to the model as a starting offset (experiment 64's arm am), lowers the daylight April-to-September quantile score by more than same-time-of-day analogues do. An analogue is a recent day, at the same half-hour of the day and of the same day type (working day, weekend, or bank holiday), whose satellite-based sunshine was closest to the sunshine forecast for the target. The model receives the observed power on the K nearest such days (K is 1, 3, or 5) as input columns. Once the explicit offset is in the model, the analogue columns add less than 0.05 points of capacity. |
inconclusive | not-for-production | Both ways of telling the monthly-refitted model about sunshine lowered the daylight error: the explicit forecast of the panels' output by 0.149 points of capacity (99.17% interval -0.201 to -0.104) and the 3 analogues by 0.095 (-0.185 to -0.007) against the reference. | primary |
| 00032 substation-fingerprints-pooled-demand | For demand series of Primary substations, adding a numeric fingerprint of each substation (its location, its mean level and daily and weekly shape over the training window, and its level over the 14 days before the forecast origin) to a by-type pooled XGBoost quantile model closes part of the 0.15-point gap between the pooled model with the series identifier and per-series models, and lets a substation held out of the pool be forecast better than a pooled model with no fingerprint. | inconclusive | not-for-production | Adding numeric substation fingerprints to the pooled model changed the Primary score by at most 0.02 points of capacity, and training-window statistics made a held-out substation worse by 1.39 (inconclusive overall; recent power statistics alone helped it by 0.23, post hoc). | also relevant |
| 00031 embedded-pv-features-primary-demand | For the 16 Primary substation demand series, giving the single-stage ENS-trained quantile model N1 trailing estimates of each substation's embedded (unmetered) solar capacity, built from the Embedded Capacity Register (ECR) as a prior for Primaries and a floor for BSP and GSP series, plus an amount learned from recent power and irradiance, lowers the quantile score on sunny days, and a linear PV offset or a sunshine-shaped correction does so more reliably than extra tree columns. | inconclusive | not-for-production | C3, A3 (PV products with the register) minus B0, quantile score over all leads: +0.006 points of capacity (99.17% interval -0.022 to +0.033). | primary |
| 00030 lagged-power-feature-sets-primary-demand | For the 16 Primary substation demand series, giving the single-stage ENS-trained quantile model N1 the 13 power lags that manual_heuristic uses (the same weekday and time of day 1 to 6 weeks back, and 49 to 55 weeks back), as separate columns beside N1's three lags measured from the forecast origin, lowers the quantile score. |
inconclusive | candidate | C3, all 13 lags minus N1 (seed 0 only): -0.01 points of capacity on the quantile score (99.375% interval -0.45 to +0.43). | primary |
| 00028 pvlive-forecast-in-stage2-gsp-demand | For the two grid supply points (GSPs) Walpole and Bicker Fen, giving the stage 2 calibrator a forecast of the GSP's solar generation at the target time, from a simple XGBoost model of Sheffield Solar's PV_Live estimate applied to ECMWF ensemble (ENS) weather, lowers the quantile score below that of the same calibrator without a solar forecast. | inconclusive | not-for-production | C1, solar forecast minus no solar forecast, over all leads: -0.14 points of capacity (95% interval -0.34 to +0.04). | primary |
| 00027 pvlive-lags-in-stage2-gsp-demand | For the two grid supply points (GSPs) Walpole and Bicker Fen, giving the stage 2 calibrator Sheffield Solar's PV_Live estimate of the GSP's solar generation at the lag times, next to the lagged delta (metered demand minus the stage 1a expectation), lowers the quantile score below that of the same calibrator without PV_Live. | refuted | not-for-production | C3, delta plus PV_Live minus no PV_Live, over all leads: -0.03 points of capacity (98.33% interval -0.13 to +0.09). | primary |
| 00025 price-change-features-stage1-demand | Giving the ERA5-trained stage 1 demand model the change in the intra-day price over the last half-hour, 1 hour, 2 hours, and 6 hours, in addition to the price at the target half-hour, lowers its error at observed power for Primary substations, and by more than the same columns built from a price taken from another week. Batteries and other dispatchable generation respond to how the price moves, not only to its level. | small gain | not-for-production | On Primary substations, adding price changes to the price level lowered the error by 0.037 points of capacity in the selection year (97.5% interval -0.056 to -0.018) and 0.050 in the confirmation year (-0.083 to -0.018). | primary |
| 00024 pooling-groupings-demand-xgboost | For demand series of Primary substations, one XGBoost quantile model trained across a group of series (all series, one model per substation type, or one model per size band), given the series identifier as a categorical column, scores no more than 0.3 points of capacity worse than one model per series, so that the number of models can fall from one per series to one per group. | supported | candidate | Pooling the 16 Primary substations into 3 or 4 XGBoost models (with the series identifier) scored 0.15 points of capacity worse than 20 per-series models (99.44% interval +0.03 to +0.26), inside the 0.30 margin; one model for all 20 series was 0.29 worse, and dropping the identifier cost 0.8 to 1.0. | also relevant |
| 00015 same-time-price-stage1-demand | Adding the intra-day price at the same half-hour as the target (Elexon Market Index Data, observed at time t, not forecast and not lagged) to the ERA5-trained stage 1 demand model, which already sees the weather at time t, lowers its error at observed power across all fitted series pooled, and by more than a price taken from another week does. | inconclusive | not-for-production | C1, price against no price, pooled: -0.52 points (97.5% interval -0.88 to -0.11) in the selection year and -0.18 (-0.50 to +0.08) in the confirmation year. | primary |
| 00014 cams-trained-stage1-with-stage0-ens-correction-demand | A stage 0 that maps the ECMWF ensemble (ENS) member's shortwave irradiance onto the irradiance of the Copernicus Atmosphere Monitoring Service (CAMS) lowers the Primary substation quantile score of the CAMS-trained stage 1a XGBoost demand model, relative to feeding that model the raw ENS shortwave, because the ENS shortwave is about 7% above CAMS and stage 1a never saw that offset in training. | inconclusive | candidate | C2, stage 0 minus raw ENS for the CAMS-trained arm, Primary quantile score over all leads: -0.067 points of capacity (97.5% interval -0.094 to -0.039), better than the planned -0.03. | also relevant |
| 00011 school-term-and-return-to-work-columns-stage1a-demand | For the ERA5-trained stage 1a XGBoost demand model at Primary substations in Lincolnshire, columns for the return to work after New Year and for Lincolnshire's school terms lower the mean absolute error of the median in the weeks after Christmas, at Easter, and in the half-term holidays, against the same XGBoost model with a Christmas column that is set only within 14 days of 25 December. | refuted | not-for-production | Return and term columns against the base arm, Primary substations: +0.06 points from 2 to 20 January (99.2% interval -0.17 to +0.30), +0.20 over the Easter window (-0.19 to +0.59), and -0.29 over the half-term holidays (-0.68 to +0.07). | primary |
| 00010 elexon-price-lags-stage1a-demand | Adding the Elexon Market Index Data price from exactly 7 days earlier to the ERA5-trained stage 1a demand model lowers its error at the substations that host price-responsive batteries, and does so by more than adding the same-lag national demand and wind output, which price follows. | inconclusive | not-for-production | C1 at B2, price against base: -0.92 points (98.3% interval -2.10 to +0.07) in the selection year and -0.06 (-0.38 to +0.23) in the confirmation year. | primary |
| 00009 cams-trained-stage1a-vs-era5-trained-demand | A stage 1a XGBoost demand model trained with irradiance from the Copernicus Atmosphere Monitoring Service (CAMS) in place of ERA5 shortwave, and applied to the shortwave of ECMWF ensemble (ENS) members, has a lower quantile score at Primary substations than the same model trained on ERA5 shortwave, because the satellite-derived irradiance is closer to the irradiance that the substations' embedded solar generation responded to. | refuted | not-for-production | C1, CAMS-trained minus ERA5-trained, Primary quantile score over all leads: -0.035 points of capacity (97.5% interval -0.070 to +0.001), a sixth of the planned 0.2-point gain. | also relevant |
| 00007 cams-static-correction-stage1a-residual | Most of the daytime residual of the stage 1a demand model at Primary substations is a fixed error in the ERA5 irradiance input and not a growing embedded-solar capacity, so a correction with fixed coefficients on irradiance from the Copernicus Atmosphere Monitoring Service (CAMS), fitted on one year of out-of-fold residuals, removes most of the residual in the next year. | inconclusive | not-for-production | The static correction removes 46.6% of the next year's daylight squared residual, just below the planned 50%, and a fixed bias alone removes 39.1%. | also relevant |
| 00006 stage1a-demand-input-feature-sweep | For a weather-to-power XGBoost model trained on ERA5 for substation demand, the local clock (hour of day and day of week) is the input that lowers the held-out error most, and the other non-lag inputs tested here (holiday encodings, time of year, trailing temperature means, sun position beyond zenith and azimuth, UTC offset, windchill, and a COVID indicator) each lower it by less. | partly supported | candidate | P1, hour and weekday against the weather block alone, Primary substations: -0.55 (99.4% interval -0.71 to -0.41) on the selection year and -0.46 (-0.61 to -0.30) on the confirmation year, the largest single input but about a third of the predicted size. | primary |