Retraining and training history: experiments
This page lists every experiment tried in the theme, newest first. An experiment that belongs mainly to another theme but bears on this one is marked "also relevant".
| Experiment | Hypothesis | Verdict | Production status | Take-home message | Relevance |
|---|---|---|---|---|---|
| 00360 bg2-member-drop-one-at-d8 | BG2's lead over the GAM stack at D8 is spread across its five-member blend, so no single member explains it. | supported | not-for-production | BG2's lower score than the GAM stack at substation D8 does not come from blending five forecasts. Four of its members (weekly XGBoost, monthly XGBoost, analogue, pooled by substation type) each score 7.47 to 7.69 alone, against 8.338 for the GAM stack, and dropping any one member changes the blend's score by less than 0.1 points. The XGBoost stack member alone scores 8.562, so the lower score more likely comes from what the other four members learn. This is one substation and one scored year. | also relevant |
| 00357 wind-default-2-monthly-head-longer-retest | Retraining Wind farms research default 2's mixture head on the first day of every month (experiment 356's arm u3) gains over default 2 at true days 1 to 10, because a recent head tracks the farms' drift. Experiment 356 measured -0.038 points on 53 Tuesday runs (interval -0.104 to +0.024) and -0.120 on about 16 Tuesday runs of 2026-07 onwards. | partly supported | candidate | Retraining Wind farms research default 2's head every month improves the forecast slightly, by about 0.04 to 0.05 points over the scored year and about 0.10 points in the latest 2.5 months, and no scope shows harm to the score. The scored-year interval still touches zero, so the change is a candidate and is not adopted. The upper tail stays too heavy (1.7% to 2.0% of observations above the 0.99 quantile), so retraining does not fix it. | also relevant |
| 00356 wind-default-2-blend-and-monthly-refit-retest | Wind farms research default 2 (experiment 347's arm y2) gains 0.234 points over Wind farms research default 1 at true days 1 to 10 but puts 2.28% of observations above q99 over every lead (2.15% at true days 1 to 10). Either a per-quantile, per-lead-band blend of the two defaults (u2), experiment 348's arm z5 (u1), a monthly offset layer on default 2 (u4), or a monthly retrained head (u3) lowers the 0.99 pinball loss and the share above q99 without losing the gain. | not supported | not-for-production | Blending Wind farms research default 2 with research default 1, or refitting it every month, does not fix its upper tail: the share of observations above the 0.99 quantile only falls from 2.15% to between 1.77% and 2.04%, against 1% intended, and no arm clearly beats default 2 over days 1 to 10. The blend's small gain (0.05 points) is the same with shuffled weights, so it is averaging, not learning. The top quantile needs new information, not a blend or a refit; the monthly retrained head is the one arm worth a longer retest. | also relevant |
| 00355 d8-weekly-refit-and-median-band-swap | (H1) Refitting D8's stack weekly instead of monthly is not the main reason BG2 beats the stack at D8. (H2) Most of the stack's remaining gap to BG2 sits in the band, not the median. | partly supported | not-for-production | Refitting D8's forecast every week instead of every month lowers its score by only 0.045 points of capacity, so BG2's weekly refit is not why it beats the stack at D8. The remaining 0.79-point gap is not mostly the band or mostly the median: BG2's band on the stack's median closes 14% of it and BG2's median under the stack's band closes 27%. The next step is to look at how BG2's median and spread are fitted together rather than at either part alone. | primary |
| 00354 d8-own-residual-bands-and-longer-fit-history | (Fix 1) At D8 (Marsh Lane), a forecast band built from the quantiles of D8's own past residuals scores much better than the band the stack borrows from six other Primaries, and costs little at Primaries with no gas engine. (Fix 3) A GAM fitted on D8's history since 2023-09, with a free level step per hour band at 2025-04-01, scores better than the GAM fitted from 2025-04 only, and the step is needed. | partly supported | not-for-production | A forecast band built from D8's own recent errors lowers D8's score by 2.3 points of capacity and closes about 60% of the gap to BG2, with no change to the median forecast. The same rule raises the score by 1% to 6% at three Primaries that have no engine, so it should apply to D8 only until a version that widens with lead time exists. A longer fit history helps only in the first three scored months, when the stored fits had 3 to 5 months of data, and a level step for the April 2025 change adds nothing measurable. | also relevant |
| 00346 wind-research-default-1-offset-table-refit-cadence | Experiment 342 found that the default's offset table, fitted once on the 442 daily runs of 2024-04-01 to 2025-06-16, leaves the upper tail too low in the scored year (observed power above q99 in 1.8% of half-hours against 1% intended) although it fits the hindcast, and blamed drift between the hindcast years and the scored year. The ensemble table holds daily runs up to 2026-10-24, so the table can be refitted each month on runs that had already finished. A table refitted monthly on a trailing 6-month window (x2) scores lower than the table fitted once. | partly supported | candidate | Refitting the wind forecast's offset table every month on recent runs makes the forecast a little better (a 0.4% to 1.1% lower quantile score), using only data the service already stores. It does not fix the upper quantiles, which still miss in 1.7% of half-hours against 1% intended. So the change between the training years and the scored year is a minor cause, and the upper-tail miss needs another explanation. A monthly job that refits the table on all earlier daily runs passes the adoption conditions and is a candidate. | also relevant |
| 00331 primary-demand-recency-weights-on-gam-and-kalman | When the additive (GAM) model with a Kalman filter on its daily residual is refitted monthly, giving older training rows exponentially less weight forecasts the next month better than equal weights on the full history, and no worse than training on the last 24 months only. | not supported | not-for-production | Training the monthly refit mostly on recent data improves the next month's forecast by about 0.025 points of capacity, but a plain 24-month window gets the same gain, so smooth recency weights add nothing and no weights should go into the research default. Half of the gain comes from one Primary substation whose demand shifted in 2023. Giving extra weight to rows from the same time of year in earlier years gave a further 0.012 points and is worth its own test. This is a lab result on past weather with a Kalman filter that already adapts to level changes. | primary |
| 00328 lsc-regime-changes-detect-then-model-part-1 | A permanent change in the configuration of an electricity network (a large new generator, a new factory) can be found offline from a Primary's own history, and a forecast that keeps the effects of the old behaviour fitted on the full history and adds new effects that exist only from the change onward scores better after the change than the same forecast refitted on a recent window only. | not supported | not-for-production | Adding effects for a detected permanent change does not forecast better than refitting the additive model on the last 24 months, in a lab setting where the model is given past weather. The detectors find a 10% step of a Primary's load within 60 days only 26% of the time at an acceptable false-alarm rate, so there is little time after the alarm for the extra effects to help. On this base, handling a regime change comes down to how often the model is refitted. The rolling-GAM detector might still help when building substation cards, which this experiment did not test. | also relevant |
| 00298 lsc-long-deep-detector-and-refit-cleaning | A trivial detector on the daily mean residual of the lagless convex additive model (GAM_ONLY) finds load shifts that last days to weeks and cover at least 10% of a Primary substation's load, which the existing hour-scale detectors miss, and excluding the days it flags from the monthly refit stops the forecast learning those shifts as the normal pattern. | not supported | not-for-production | A simple detector on the daily mean error of the forecast that uses no recent power finds about as many of the long, deep real load shifts as the learned score LR_b20 (10 to 12 of 19 for four of the six variants). The cumulative-sum variant B also finds 27 of the 29 multi-day drops at D8, which are probably a gas engine and not load shifts; variant C finds 11. Leaving the days a detector flags out of the monthly refit cuts the share of a 30-day transfer that the forecast learns as normal from about 19% to about 10% to 13%. Excluding the true shift days instead cuts the share to 2.8%. The detectors recover less because each needs 1 to 3 days to fire and misses about 40% of long injected transfers and over half of the real ones. Tell NGED that transfers of 14 days or more are partly learned as normal, and try a detector that fires on the first day next. | also relevant |
| 00295 regime-segmentation-part-b-persistent-regimes-without-labels | A segmentation of a Primary's whole history with hindsight, using only changes in the distribution of the residual of a lagless convex additive model (GAM), finds regime changes without labels. A regime is the whole period, without gaps, during which one configuration holds. For D8's gas engine, the regime is the whole period in which the engine is installed and included in the meter, including the spells when the engine is off. Whether the engine is running, and whether a load shift is in force, are states inside a regime. A boundary is a regime change when the distribution does not return to its earlier form and no neighbouring Primary shows the opposite change, and a state change otherwise. | not supported | not-for-production | Looking only at the residual of a forecast that uses no recent power, a segmentation finds a lasting change of 25% of a Primary's load and dates it with a median error of 0 days, but it misses about half of 10% changes and cannot reliably tell a 14 to 60 day load shift from a lasting change. At D8 it finds the 2023-08-15 break exactly, but only at penalties that also find 2.5 or more regime changes per Primary-year at the other Primaries. An offset for the latest regime lowers the forecast error after a step by 2.2 points of capacity, but a plain 60-day mean residual does as well, so the segmentation is not worth adding for that purpose. | also relevant |
| 00290 lsc-long-shift-cut-date-robustness | The 19% of a 30-day transfer that experiment 274 found learned as normal is stable across fit-cut dates, so that the share learned lies within 15% to 25% at every cut date. | not supported | not-for-production | The forecast with no recent power learns about a fifth of a 30-day load transfer as normal, but the share depends on the date: it ranges from 9% to 29% across 12 end dates, so experiment 274's 2.49-point bias is a typical value, not a fixed one. Quote the share as a range of about 9% to 29%. The cause of the spread is not located. | also relevant |
| 00278 stack-retraining-cadence | The best-guess stack (arm D, the neural network head, and the two-number solar summary) needs its neural network head retrained at least quarterly, but its convex model (a generalised additive model with a Kalman filter on its coefficients) tolerates quarterly instead of monthly refits, because the filter absorbs the drift between refits. | not supported | not-for-production | Retraining the small neural network that corrects the demand forecast's median, every quarter or every month, scored no better than training it once before the scored year, so it needs no retraining schedule over a year. The convex demand model should keep its monthly refits, which take 0.4 s per substation: refitting quarterly instead scored 0.010 points worse overall (not shown to be acceptable) and weekly refits gained nothing. Refitting the two solar factors quarterly instead of monthly changed nothing. | also relevant |
| 00275 metered-farm-long-history-neural-head | Experiment 243 fitted H5 (the solar-farm power model plus a neural network that outputs all 11 quantile offsets) on about 17 months of ensemble forecasts. Adding each farm's rows built from past weather (Copernicus Atmosphere Monitoring Service irradiance and ERA5 temperature, back to 2019-09) as extra training rows, with a flag that marks them, lowers H5's quantile score on the ensemble-driven scored year, and the neural head gains more from the extra rows than XGBoost does. | not supported | not-for-production | Training the neural network that corrects a metered solar farm's power forecast on nearly six years of past-weather rows, as well as on 17 months of ensemble forecasts, lowers the quantile score by only 0.5%, far under the 2% we predicted. The same rows improve XGBoost by 1.7%, so the neural network is not the better way to use extra history. The extra rows do lower the loss at the 99% quantile (the upper edge of the forecast band, which matters for curtailment) by 4.6%. | also relevant |
| 00274 lsc-long-arrangement-audit-gam-refit | A forecast with no lagged power learns a long-lasting change of running arrangement as the normal pattern, because the monthly refit trains on whatever the meter recorded. The learned share grows with the duration of the arrangement, and the duration beyond which the forecast of the normal running arrangement (NRA) is wrong by more than a stated tolerance sets the limit that can be told to NGED. | not supported | not-for-production | A forecast that uses no recent power still learns a load transfer as normal, and the forecast learns the transfer quickly: a transfer lasting only 7 days leaves the forecast 1.2 points of effective capacity too high for the next 2 days, and a transfer lasting 30 days leaves it 2.5 points too high (about a fifth of the transfer). No tested duration, down to 7 days, was short enough to avoid this. Masking known transfers out of training is the obvious remedy, but this experiment did not test it, and the next experiment should test it together with transfers of 1 to 6 days. | also relevant |
| 00167 dp-full-history-gam-24-months-or-all-years-with-drift | With the PV capacity pinned to the as-of table (explicit PV) and the heating split by day type (T-all), the convex demand model (the GAM) trained on every year of history back to late 2019 with a drift term scores better than the same model trained on the last 24 months, when both are refitted at 12 quarterly origins on past weather (ERA5 and CAMS). The mechanism is that explicit PV removes the changing solar capacity from the target, so the old years teach the heating and calendar terms without the capacity growth that made the all-years fits of experiments 104 and 107 unstable, and the drift term absorbs the remaining slow change in the demand level. | supported | candidate | Adding a drift term (a slow adjustment to the demand level) improved the forecast at 11 of 12 quarterly refits. The quantile score, where lower is better, fell by 0.40 points when training on the last 24 months and by 0.55 points when training on all years back to late 2019. Training on all years instead of 24 months adds only 0.10 points on top of a drift term, and gives no gain without one, so extra years of history help little. | also relevant |
| 00149 differentiable-physics-full-history-gam-era5-cams | When the convex model's solar term is explicit (a pinned capacity times a unit solar shape in both arms), training on every year of a substation's history back to 2019 gives a lower quantile score than training on the last 24 months, once a drift term lets the model follow the slow level change. The mechanism is that more years pin down the temperature, calendar, and weekday terms better, and the drift term absorbs the level change that made all-years training unstable in experiments 104, 106, and 107. | inconclusive | not-for-production | Training the demand model on every year back to 2019, instead of the last 24 months, lowered its forecast error by only about 3% (0.08 points of substation capacity), and the difference could be chance. The drift term helped much more, lowering the error by 0.38 points on 24 months and 0.50 points on all history, but a post hoc audit found that a plain correction by the mean error of the last 28 training days recovers 86% to 94% of that gain, so the gain is a recent-level correction and the drift form is not shown to beat the simpler fix. We did not test whether explicit heating, switching, or pooled terms would make the longer history help. | also relevant |
| 00107 differentiable-physics-rolling-origin-history-and-tails | Refitted quarterly from 2023-07 to 2026-04 and scored on the next 3 months, the convex model's two history contrasts (all years against 12 months, drift against none) keep one sign at almost every origin. A second claim, labelled separately as C5 and not part of the verdict: a level-dependent spread brings its 95% and 99% quantiles within 1 point of nominal coverage without worsening the quantile score. | partly supported | not-for-production | Giving the convex demand model a drift term lowered its score at 10 of 12 quarterly refits, by 0.45 points when trained on all years (interval 0.23 to 0.68). In a post hoc split, about three-fifths of that size is one substation (D10) at the 2023-24 refits, and the 2025-26 refits give 0.30, as in experiment 104. All years without drift scored worse than 12 months at 8 of 12 refits, and all years with drift against 12 months has no stable answer: winter-quarter refits favour more years, summer ones are level. The spread that depends on the forecast level made the 95% and 99% quantiles worse, so the tail coverage still needs a different method. The verdict is partly supported, because only the drift contrast is stable; a second spread (the conformal spread) is not interpreted, because it was built with a flaw. | also relevant |
| 00106 differentiable-physics-4-all-years-drift-and-recent-level | The differentiable-physics model trained on every usable year from 2019-09 with a drift term forecasts Primary demand better than the same model on its best window, and assimilating recent residuals through a Kalman filter makes it better again, by more than a simple moving-average correction does. | partly supported | not-for-production | Correcting each day-ahead demand forecast with the errors the model made up to 3.5 hours earlier lowered the quantile score by 0.61 points (from 2.90 to 2.29) in the scored year and by 1.02 points in the year before, at all five long-history substations. The gain relies on estimates of past weather that a live service would not have, and against lagged XGBoost at the same lead the filtered model scored 2.349 against 2.487, a difference that is not significant. Whether old years of history help is not settled, because the contrast changed sign between the two years, so the verdict is partly supported. | also relevant |
| 00104 differentiable-physics-2-simple-structural-model | A convex demand-minus-solar model, trained on every usable year with a slowly drifting level per hour band, forecasts Primary demand better than the same model trained on the last 12 months. | inconclusive | not-for-production | A simple model of demand minus solar, trained on 12 months, scored 0.25 points better than lagless XGBoost with its default start (post hoc; a lower score is better). Most of that lead is in the forecast spread, because the error of the mean forecast differs by only 0.08 points, and four of the five pre-registered positive controls failed, so the verdict is inconclusive. More years of history did not clearly help: the scored year and the tuning year gave opposite signs for two history-window contrasts, and a first version of this page had the wrong sign for both because its spread carried the previous year's mean bias. | also relevant |
| 00095 age2-old-year-penalty-with-the-same-weather-product-demand | The extra forecast error of an old training year (experiment 93) comes mostly from the change of weather product between the old years (18 km grid spacing) and the recent year (9 km), not from the age of the training data, so an old year and a recent year that use the same weather product score about the same. | refuted | not-for-production | The old-year penalty does not shrink when both years use ERA5: the 2021-07 year is 1.71 points worse than the 2024-07 year on ERA5 (99% interval 1.15 to 2.29) and 1.57 worse on the control member (1.12 to 2.03), so the weather product is not the main explanation and the penalty goes with how old the training year is. | primary |
| 00093 age1-old-training-year-after-level-normalisation-demand | A year of training data hurts a Primary substation demand forecast with its age only because the substation's average demand level differed in that year, so subtracting a trailing 365-day level from the targets and the power lags removes most of the penalty for an old training year. | refuted | not-for-production | Subtracting a trailing 365-day level removes 42% of the old year's penalty (C1 1.57, C2 0.91 points), under the half the prediction required, and without the one series that carries most of the penalty it removes none (0.61 to 0.68). | primary |
| 00092 lv0-drift-diagnostics-on-stored-predictions-demand | Most of the drift in Primary substation demand since 2022 is a per-series level or smooth trend, not a change of daily shape, so a slow per-series level term, not a new weather response, is what a long-history or pooled XGBoost model is missing. | inconclusive | not-for-production | Drift is large and differs by series, but a level shift alone does not explain the loss from extra history or the pooling gap. | primary |
| 00085 cheaper-retraining-schedules-rank-arms-like-weekly-demand | For the 16 Primary substation demand series, a cheaper retraining schedule ranks two pairs of arms the way full weekly retraining ranks them: the cadence pair (monthly minus weekly retraining) and the starting-value pair (a zero starting value minus XGBoost's estimated starting value, the contrast of experiment 80). The cheaper schedules are (a) a monthly refit on a rolling 12-month window with 150 rounds and eta 0.15 (P-roll), (b) the same refit every quarter, (c) a monthly refit at the full setting (whole history, 500 rounds, eta 0.05), and (d) a weekly refit on the whole history with 150 rounds and eta 0.15. A schedule that passes can replace weekly retraining in research experiments. |
partly supported | not-for-production | Monthly minus weekly retraining is +0.125 (95% interval +0.036 to +0.229) at the full setting, +0.118 (+0.033 to +0.216) with a 12-month window, and +0.125 (+0.038 to +0.227) with 150 rounds. | primary |
| 00081 r11-control-member-second-validation-year-demand | The Primary substation demand findings of experiments 48 and 49 keep their sign on forecast runs the project has not scored: the daily 00:00 UTC ECMWF ensemble (ENS) control-member runs of 2023-07-01 to 2024-03-31 (July to March, with no April to June). For the 13 Primary substations that have power data throughout, weekly refitting of the XGBoost demand model N1 without error correction scores below the stack (frozen N1 with the 14-day correction and the spread factor 1.15), and weekly refitting scores below monthly refitting. | supported | candidate | C1, weekly minus the stack: -0.240 points (99.17% interval -0.360 to -0.145). | primary |
| 00080 bm0-zero-base-margin-xgboost-quantile-models | XGBoost starts a quantile model from a base score that it estimates from the training labels, one value for each of the nine quantile levels (experiment 78 found 0.403 at the 5% level to 0.899 at the 95% level for one series, with 500 boosting rounds and no early stopping). Replacing that start with zero (a base margin of zero in every row, which is the same as a base score of zero) lowers the quantile score of the production-style models for Primary substation demand (retrained weekly), bulk supply point and grid supply point demand (retrained weekly), solar farms, and wind farms. The gap is not a symptom of too few boosting rounds, so four times as many rounds, or early stopping, does not remove it. | inconclusive | not-for-production | Starting XGBoost from zero did not lower the quantile score of any production-style XGBoost model: Primary substations -0.017 points of capacity (98.75% interval -0.072 to +0.054), bulk and grid supply points +0.006 (-0.091 to +0.092), the frozen Primary stack -0.033 (-0.084 to +0.031), solar farms -0.010 (-0.036 to +0.028). | also relevant |
| 00049 vp1-screening-reference-under-p-roll-demand | For the 16 Primary substation demand series, once the XGBoost quantile model N1 is refitted every month without the 14-day error correction, (a) a spread factor chosen on earlier months only lowers the quantile score below the fixed factor 1.15, (b) an average of the quantiles of retrained and stored XGBoost models, with one weight vector chosen per month on earlier months, still lowers it below the best single member, and (c) each adopted input (the Christmas column, the COVID indicator, a half-life of 0.5 years) changes the score when dropped, and the substation fingerprints of experiment 32 change it when added to a pooled XGBoost model, all under the cheap rolling-window refit P-roll. | supported | candidate | B1, the blend minus its best member: -0.076 points (99.17% interval -0.135 to -0.023). | primary |
| 00048 weekly-retraining-confirmation-and-extension-demand-generators | On the 292 daily ENS forecast runs of the validation year that experiment 34's 73 runs leave out, weekly retraining of the Primary substation demand model N1, with no error correction, lowers the quantile score below the best-guess stack (frozen N1 with the 14-day correction and spread factor 1.15) by more than zero points of capacity; monthly retraining of the ERA5-trained solar and wind stage 1a models changes the score by less than 0.05 points; and, once the model retrains weekly, neither a 7-day nor a 14-day error correction helps. | supported | candidate | C1, weekly retraining minus the stack: -0.20 points (99.44% interval -0.40 to -0.035). | primary |
| 00035 lead-weighted-quantile-blend-of-stored-models | For Primary substation demand, solar farms, and wind farms, averaging the quantiles of two or three forecast models that experiments 19, 20, 30, and 33 already scored (N1 is the ENS-trained single-stage XGBoost model, and the stack is N1 with the 14-day correction and a spread factor of 1.15), with one weight per lead band chosen only on earlier months, lowers the quantile score below the best single member by more than the threshold set for each technology (0.05 points of capacity for Primary demand and solar, 0.10 for wind). | supported | candidate | Averaging the quantiles of three stored demand models lowered the Primary demand error by 0.10 points of capacity against the best single model (98.33% interval -0.19 to -0.03), so the estimate clears the 0.05 threshold for moving it into production. For solar and wind the gains were too small to resolve on 6 solar farms and 3 wind farms. | also relevant |
| 00034 retraining-cadence-and-cheap-proxies-demand | For the 16 Primary substation demand series, retraining the single-stage ENS-trained quantile model N1 monthly (half-life 0.5 years, no 14-day error correction) lowers the quantile score below the best-guess stack (frozen N1 with the 14-day correction and spread factor 1.15) by more than 0.10 points of capacity; the gain from retraining more often flattens between monthly and weekly retraining; and the 14-day correction stops helping once the model is retrained. A second, separately reported claim is that at least one cheap proxy for monthly retraining ranks arms the way full monthly retraining ranks them. | inconclusive | candidate | C1, monthly retraining minus the stack: -0.20 points (99.375% interval -0.44 to +0.08, 95% interval -0.38 to -0.01). | primary |
| 00023 lagged-power-and-error-correction-solar-wind | For the 6 solar farms and 3 wind farms, giving the stage 2 calibrator the farm's recent observed power (lagged power, or summaries of the latest day), or shifting the forecast by its own recent error (the 14-day correction of experiments 00017 and 00019), lowers the continuous ranked probability score (CRPS) below that of the same calibrator without recent power. | inconclusive | not-for-production | C1, lagged power added to the calibrator: solar -0.012 (99.58% interval -0.122 to +0.088), wind +0.012 (-0.090 to +0.146). | also relevant |
| 00021 error-correction-variants-bsp-gsp-demand | A recent-error correction of N1's quantiles, measured with power that is 2 days old, lowers the quantile score of the 4 bulk supply point (BSP) and grid supply point (GSP) demand series as it does for the 16 Primary substations, and some variant of the correction (a 7-day or 28-day window, multiplicative, per-half-hour-of-day, or weighting recent days more) beats the plain 14-day additive correction. | inconclusive | not-for-production | The 14-day additive correction lowered the Primary quantile score by 0.99 points of capacity (99.375% interval -1.82 to -0.18) when the newest power was 2 days old, as it did with 3.5-hour-old power in experiment 19. On the 4 bulk and grid supply point series it raised the score by 0.36 (-0.19 to +0.98), so it is unproven there. No variant beat the plain additive 14-day form: a 7-day window was worse (+0.19) and a multiplicative form far worse (+2.28). | also relevant |
| 00019 primary-demand-stack-vs-manual-heuristic | For the 16 Primary substation demand series, stacking three changes on the single-stage ENS-trained quantile model N1 (a 14-day correction from the model's own recent errors, a recency half-life of 0.5 years, and a spread factor of 1.15) beats manual_heuristic on ENS weather, which N1 alone does not. |
supported | candidate | C1, the full stack minus manual_heuristic, quantile score over all leads: -0.56 points of capacity (99.44% interval -0.93 to -0.26). | also relevant |
| 00017 training-window-and-retraining-rule-drift-demand | For the stage 1a XGBoost demand model trained on ERA5 weather, Primary substation demand drifts (experiment 13 saw every arm under-forecast its validation year by 1.3 to 3.1 points of capacity), so a model trained on a recent window, weighted towards recent rows, retrained monthly, or corrected by the error of the last 14 days, has a lower error than the same model trained on the whole history and never retrained. | supported | candidate | C9, the 14-day correction against A: -1.52 points (99.44% interval -1.95 to -1.13). | primary |