Lagged power and error correction: experiments
This page lists every experiment tried in the theme, newest first. An experiment that belongs mainly to another theme but bears on this one is marked "also relevant".
| Experiment | Hypothesis | Verdict | Production status | Take-home message | Relevance |
|---|---|---|---|---|---|
| 00222 dp-network-before-filter-order | Experiments 200 and 201 correct arm D after the Kalman filter. Their gain is mostly a systematic error of the convex model at some temperatures and loads, not a recent-error effect, and that systematic error currently sits in the filter's level state. A network that reads only the convex model's own output and inputs, and corrects the convex model before the filter, leaves the filter a cleaner residual stream and so beats the same kind of network applied after the filter. | inconclusive | not-for-production | The gain of the network after the filter does not come from the convex model's own systematic error as seen in its past residuals. A network trained on those residuals learned patterns that do not carry to the next year, and when it was stopped on a later year it learned nothing and left the score unchanged. The test could not settle the order question, because the network failed to recover a known injected error. | also relevant |
| 00205 dp-gamk-gap-robustness-coast-or-frozen-gam | When power readings go missing, a Kalman filter that coasts (keeps running with no update, so the transient decays and the level holds) beats the frozen convex model (GAM) alone for short gaps and loses to it for long gaps, so a crossover gap length exists between 6 and 72 hours. The mechanism is that the level is a random walk, so a short gap leaves it nearly right, while a long gap lets it drift away from the demand. Widening the quantile bands by the filter's own growth in predictive variance keeps the 5% and 95% quantiles calibrated at the gap lengths where coasting is still better than the frozen model. | inconclusive | candidate | When readings stop for 3, 6, 24, or 72 hours, keep running the Kalman filter, which corrects the convex demand model using its recent forecast errors, without feeding it new readings, and widen the quantile bands. At each of those four gap lengths that forecast scored 0.25 to 0.44 points of quantile score better than the convex demand model with no correction (a quantile score is an error measure in percent of the substation's effective capacity, and lower is better; the gain is 8% to 14% of the uncorrected model's score). At 72 hours the 99.4% bootstrap interval for the gain runs from 0.04 to 0.48 points, so the gain may be small. The data cannot show whether dropping only the filter beats coasting after a gap of about 72 hours: at every gap length the difference between the two forecasts is smaller than the variation between months. The forecast with the filter, winter night offset, and recipe R was still 0.025 points worse than its no-gap score 48 hours after a 24-hour gap ended. The filter alone was within 0.02 points of its no-gap score 24 hours after the gap ended. | also relevant |
| 00201 dp-gam-residual-mlp | A small neural network that reads the convex model's own design features, and later the calendar, the ensemble-mean weather, and lagged power, can learn an interaction the additive convex model lacks, and so lowers the quantile score of the best-guess convex forecast (arm D of experiment 155) when it adds a correction to the median and rescales the band widths. | partly supported | not-for-production | A small neural network that reads the convex demand model's forecast lowers that forecast's quantile score by about 0.09 points (3%), and adding weather, calendar, or recent power does not help further. The neural network corrects an under-forecast on very hot and very cold days. That points to a missing temperature term in the convex demand model. The correction also scores worse on bank holidays and, in point estimate, on Christmas. | also relevant |
| 00200 dp-pooled-correction-head-on-kalman-state | Experiment 142's small network failed its known-answer gate with about 1,100 samples from one year at five substations. The same kind of network, trained on rows pooled across many Primary substations and years, learns to correct arm D's forecast from the recent errors, the calendar, the lead, and the ensemble-mean weather, and so beats recipe R (subtract 0.75 times the mean error of the last 14 days) at day D+1. | partly supported | not-for-production | A small neural network trained on arm D's own past errors lowers the day-ahead score by 0.07 to 0.12 points at five substations, but a copy that sees no recent error gets most of that gain. The gain comes from correcting how arm D's error depends on its forecast level and the calendar, and a pinball-loss table comes within 0.01 to 0.02 points of the network on one year of data. | also relevant |
| 00197 dp-kalman-level-adaptation-retest | Experiment 190's known-answer gate could not be passed by a filter with one state per hour band, because the target's band is usually unread since an offset began. Scored only on pairs the filter can know about, (a) a faster per-band filter (A1) recovers at least 50% of a 0.10-size, 16-hour offset on target bands already read, and (b) a filter whose bands share one level while a same-sign run lasts (A3) recovers at least 50% on target bands not yet read, because it moves bands that no reading has reached. | not supported | not-for-production | Making the Kalman filter's level state faster does not fix offsets that last for hours. Faster settings recover a long offset only late, and only by scoring worse on quiet days, and a level shared across the four hour bands recovered under a fifth of the offset, because its run counter cannot tell an offset from noise. No setting on the grid tried recovered an injected offset without scoring worse on quiet days, so level adaptation is dropped. | also relevant |
| 00190 dp-kalman-level-adaptation-gated-boost | The Kalman filter of the convex demand model (GAM-K) has a level state that is a random walk with one shared process-noise setting. A single setting must trade speed on a real level shift against noise on ordinary days. A gate that multiplies the level's process noise by m only while the last k one-step innovations (pooled over the four hour bands) share one sign and their mean exceeds z times the day-before innovation spread gives speed when a shift looks real and keeps the quiet setting otherwise. The mechanism predicts that the gate lowers the excess loss on days where the residual keeps one sign for hours (the 15 smooth-offset days of experiment 175), without harming ordinary days. | inconclusive | not-for-production | A Kalman filter whose level state speeds up when the residual keeps one sign scored the same as the current filter (+0.0002 points), and an always-fast version scored 0.023 points worse. The test cannot say that filter speed is useless: its known-answer gate could not have been passed by any filter with one state per hour band, and a faster filter did recover offsets where they were visible. A fair test needs a redesigned gate and a wider grid of multipliers. | also relevant |
| 00168 dp-gamk-production-cadence-and-3-5-hour-delay | The GAM-K forecast of experiment 155's arm D (the convex demand model with a Kalman filter on recent errors, the winter night offset, and recipe R) keeps its score when it is issued the way production would issue it. Production refreshes the forecast every 6 hours from the newest ECMWF run, so the weather forecast is 12 to 30 hours old, and the filter reads only power readings that are at least 3.5 hours old. Experiments 136 and 155 issued one forecast a day, 9 hours after the run. The mechanism is that a fresher filter state makes up for an older weather forecast, so the score at a given lead changes little. | inconclusive | not-for-production | Issued every 6 hours from the newest weather forecast with power readings 3.5 hours old, the stage 2 forecast scores about the same as the once-a-day forecast over the next 24 hours (-0.007 points, 99% interval -0.076 to +0.037). The largest loss, 0.073 points, falls between 09:00 and 12:00 UTC, when the new weather run has arrived and the 6-hourly schedule has not yet used it. The pre-registered same-lead contrast read +0.076 but is confounded with the hour of the day, so the formal verdict is inconclusive. | also relevant |
| 00155 dp-bg-stage2-assembly-night-offset-and-error-correction | A GAM-K forecast (the convex demand model with a Kalman filter on recent errors, experiment 111) that already has the heating split by day type, one shared filter noise setting, the pinned solar capacity, and the ensemble mean as its weather input (experiment 136's arm A2) improves further when two post-forecast corrections are added: a winter night offset (experiment 137) and the 14-day mean error profile of recipe R (experiment 115). The mechanism is that the two corrections remove different errors, a level error that repeats on winter nights and a half-hour-of-day shape error that repeats day to day, so their gains add. The assembled arm then scores below experiment 111's GAM-K and BG2 (experiment 87) over leads 0.5 to 168 hours, and its tails on cold nights and on holidays are no worse than the uncorrected arm's. | inconclusive | candidate | Adding recipe R, which subtracts 0.75 times the mean error of the last 14 days by half-hour, lowers the stage 1 assembly's score by about 0.04 points (1.3%), and the winter night offset adds nothing once R is in. The assembly with R scores 2.754 against BG2's 2.780, which is better by 0.026 points with an interval too wide to rank the two. The pre-registered placebo control failed, so the formal verdict is inconclusive, but the choice of winning arm (R alone) does not depend on it. | also relevant |
| 00145 differentiable-physics-analog-day-error-correction-on-gam-k | Averaging GAM-K's past day-ahead errors over the days most similar to the target day (same day type, with holiday identity for holidays, similar forecast daily mean temperature, and a season window of a few weeks either side of the date) gives a better correction than averaging the last 14 days alike (recipe R of experiment 115) or the same-day-type days of the last few weeks (experiment 134). The reasoning is that demand errors repeat on days that resemble each other. The errors that repeat most, on holidays and on cold days, are the ones that a 14-day average dilutes. | small gain | candidate | Averaging the errors of the 20 most similar past days (same day type, within 13 weeks of the same date, last year's days included) beat recipe R (the 0.75-weighted mean error of the last 14 days) by 0.041 points at day D+1, the calendar day after the forecast origin (about 1.7% of R's score of 2.436). That gain is larger than the 0.011 points that the weekday term of experiment 134 gained. Matching on forecast temperature helped by only 0.005 points, and copying the error of the same holiday last year made holidays worse, so the next step is a day-type profile with a season window and no correction copied from an earlier holiday. | also relevant |
| 00144 differentiable-physics-weather-forecast-bias-persistence-on-gam-k | The day-ahead error of the ensemble weather forecast (temperature and irradiance) repeats from one day to the next, so the trailing mean of recent weather errors can correct GAM-K's demand forecast beyond what the Kalman filter already does. | refuted | not-for-production | In this test, the day-ahead temperature and irradiance forecasts did not repeat their errors from one day to the next: the correlation between one day's error and the next day's was at most +0.11, against a stop threshold of 0.2. A correction that feeds recent weather errors into the demand forecast therefore has little signal to use, so no demand forecast was fitted. Two caveats apply: wind was not tested, because experiment 111's ensemble store does not hold it, and the reference was a later forecast (the analysis proxy), not observed weather, so a bias against observations is not ruled out. | also relevant |
| 00143 differentiable-physics-holiday-aware-updating-of-the-gam-k-filter | The Kalman filter of GAM-K lets a holiday's reading error teach it a wrong level, which carries into the days after the holiday; leaving holiday readings out of the filter's update (and nothing else) lowers the day-ahead score on those following days, and does not hurt the holiday targets or the rest of the year. | not supported | not-for-production | Leaving holiday readings out of the day-ahead filter's update does not reliably help the days after a holiday: the effect had the opposite sign after Christmas 2024 and Christmas 2025, and its 99% interval in the scored year (-0.309 to +0.036 points) includes zero. In 10 of 12 holidays, the holiday readings' error had the same sign as the error on the following days, so these readings are not usually contamination. Setting the correction to zero on holiday days, as experiment 118 did, is the action that hurts (+0.62 points on holiday days). Two Christmases give no basis for changing the filter this Christmas. | also relevant |
| 00142 differentiable-physics-sequence-model-on-gam-k-residuals | A small neural network that reads GAM-K's last 14 days of residuals, the calendar, and the target day's forecast weather predicts the day-ahead residual better than the hand-built Kalman filter with the 14-day error profile (recipe R of experiment 115), and gives better tails when it is trained with the pinball loss at the 11 quantiles. | inconclusive | not-for-production | A small neural network that reads the last 14 days of GAM-K's errors did not learn to adapt to changes in them: when we added a known step or a Saturday-evening shift to real errors, it recovered none of it, so we did not score it on the scored year. Nothing was scored, so this result does not compare it with the hand-built Kalman filter and recipe R. | also relevant |
| 00135 differentiable-physics-n10-joint-kalman-and-demand-fit | Fitting the convex demand model (GAM-K) so that its coefficients minimise the day-ahead error after the Kalman correction, instead of the plain error before it, lowers the day-ahead quantile score at 12 to 36 hours below the two-stage fit (fit the demand model, then filter its residuals), without moving the fitted shapes to explain recent-level structure that the filter already tracks. | not supported | not-for-production | We fitted the demand model so that its coefficients minimise the error left after a Kalman filter (a running correction of recent errors) has adjusted the forecast. That made the day-ahead forecast much worse, because the filter can absorb a shift in the slow parts of the demand model, so the fit barely constrains those parts. Tying the fit to the two-stage coefficients removed the damage and also removed any gain: the quantile score was within 0.013 points of the two-stage score, in either direction. The two-stage design (fit the demand model, then filter its residuals) stays. | also relevant |
| 00134 differentiable-physics-same-weekday-error-profile-on-gam-k | Adding a same-weekday component to the 14-day all-days error profile (recipe R of experiment 115) lowers GAM-K's day-ahead quantile score, because demand errors repeat from week to week (Saturdays differ from Tuesdays) and the all-days profile averages that structure away. | not supported | not-for-production | The gain is too small to count: using the same weekday's recent errors, instead of the last 14 days' errors alike, lowers the day-ahead error by about 0.011 points (0.5%) at five Primary substations, and the scored-year interval includes zero. The gain is on Mondays and Fridays and is absent midweek. Grouping days into five day types did better than seven separate weekdays, so day types are the version to test next, with the profile limited to day D+1. | also relevant |
| 00133 differentiable-physics-vk-learned-noise-and-observation-variance-in-gam-k | On GAM-K's filter (a level and a transient per hour band, per substation), noise settings chosen per substation by day-ahead score lower the day-ahead quantile score below the fixed settings of experiment 111, which were chosen by marginal likelihood on 184 non-contiguous days. An online observation variance in the manner of VIKING (de Vilmarest and Wintenberger, 2022), used to scale the spread, then improves the coverage of the 0.95 quantile on cold nights, which experiment 125 found at 73% to 78%, because recent large errors widen the spread. The mechanism under test is that the size of recent proxy-driven residuals predicts the size of the next day's error on cold nights. | inconclusive | not-for-production | Choosing the filter's noise settings for each substation by how well they forecast the next day did not improve the forecast on later months: the gain of 0.044 points on the months used to choose shrank to 0.009 on the scored year, inside the noise. A single setting shared by every substation did at least as well as the per-substation settings, so tuning the filter's noise per substation looks unnecessary, although the grid did not find a minimum. Letting the observation variance follow recent errors did not widen the forecast where it matters, on cold nights, where the 0.95 quantile still covered only 86% to 88% of readings. | also relevant |
| 00128 differentiable-physics-kfd-shared-day-anomaly-and-sparse-q-in-gam-k | Adding to experiment 111's level-and-anomaly filter (GAM-K) one day-level error state shared by all four hour bands, and estimating the level noise sparsely from zero, makes the filter's midday correction track an injected solar-panel-shaped change in real residuals better than GAM-K, and lowers the day-ahead score on ENS. Real residuals carry a day-level error shared across the four hour bands: the daily means of GAM-K's one-step innovations in different hour bands correlate at 0.18 to 0.23 (control P-premise). An earlier diagnosis of experiment 118's residuals gave 0.58 on a different quantity, the daily means of the residuals, which cannot separate a day-level error from a slow drift shared by all bands. | inconclusive | not-for-production | The day-ahead forecast was not scored, because the filter with one shared day-level error state failed its test on injected changes in real recorded errors, and the experiment stopped there. The errors at different times of day do move together (a correlation of about 0.2 between their daily innovations), but a single shared state probably also soaks up errors that belong to one time of day, so it cannot follow a slow, solar-panel-shaped change. Options for a next experiment are a day-level state for each time of day with a shared part, an explicit solar-panel term, and a test with an injected change large enough to detect; the verdict is inconclusive. | also relevant |
| 00126 differentiable-physics-s1-cheap-screens-added-to-gam-k | Each of six families of cheap additions to the day-ahead convex model GAM-K lowers its score after the Kalman filter, winter first: a free or monotone temperature curve (A-free, A-mono), exponentially smoothed temperatures (S-ewma) or smoothed daily extremes (S-mm), wind chill (C-wind) or apparent temperature (C-hw) inside the heating hinge, a school-term term (D-sch), lighting from solar elevation (E-sol), and a filter whose noise differs by regime (F-mix). A second family of four exploratory screens: a switching Kalman filter with three hidden regimes (G-skf), a learned periodic basis for the year cycle (W-basis), a soft cold/mild mixture of heating slopes (W-mix), and a cross-substation anomaly from weather-station residuals (W-stn). One hypothesis per arm, in "Arms" and "Four exploratory screens". | inconclusive | not-for-production | None of the thirteen cheap additions to the day-ahead GAM-K lowered the winter score by a margin we can trust. E-sol (lighting from solar elevation) came closest, at -0.036 points on dark winter evenings, but its interval (-0.122 to +0.024) is too wide to call, because six of the thirteen screens had a half-width above their 0.03 to 0.05-point margin. Only W-basis (learned year cycle) was clearly worse, in every month, site, and year. F-mix and W-stn looked worse too, but each rests on 2 to 3 calendar months. G-skf found injected load transfers 4 to 7 times as often at 25% and 40% transfers as the plain filter, but failed its own power gate and recovered no faster. | also relevant |
| 00121 differentiable-physics-mh-multi-hinge-heating-curve-and-night-step | In the convex model of Primary demand in the form Part 6 made the DP's candidate (GAM-K), on past weather, replacing the single fixed 15.5 C heating hinge by five hinges (10, 12, 14, 16, and 18 C) with non-negative slopes, so the heating curve never falls as it gets colder and stays convex, lowers the day-ahead quantile score after the recent-error filter. Second question (NS): GAM-K leaves a 4 to 7 point step at 01:00 on winter nights at three substations (a pattern like storage heating switching on), and a night-step term lowers the score on winter nights without worsening the other rows by more than 0.02 points. Indexed in UTC against local time, the winning indexing is reported. | inconclusive | candidate | Adding night-time levels indexed on the UTC clock lowers the day-ahead error on winter nights by about 0.23 points after the recent-error filter, at all 5 substations in both winters, and the forecast then shows the 01:00 jump in demand that the current forecast misses on cold winter nights. Five heating bends instead of one gain only 0.016 points, within noise. The synthetic tests could not show that a step this small can be detected, so the night result needs a test on weather forecasts and an audit before it enters the model. | also relevant |
| 00118 differentiable-physics-kfe-kalman-filter-on-the-gam-effects-on-ens | Freezing the convex model's fitted effects and letting a weight on each follow a random walk, with a diagonal noise matrix fitted by likelihood on the tuning months, forecasts day-ahead demand on ENS better than experiment 111's filter of one level and one anomaly per hour band (GAM-K), because heating slopes, PV capacity, profile scale, and holiday effects drift faster than a 24-month fit follows. The fitted noise matrix beats one step variance shared by every weight, so the gain comes from how each effect drifts and not from the weights' freedom alone. | inconclusive | not-for-production | Letting the convex model's PV, heating, profile, and holiday weights drift did not improve the day-ahead forecast over experiment 111's level-and-anomaly filter (0.015 points worse at D+1, interval -0.013 to +0.048), and did no better in winter or on cold days. Switching the correction off on holidays is much worse (+1.24 points) and halving it is not better, so the filter still helps on holidays. The weights cannot be read as measurements, because the known-answer gate failed on real residuals, although two of its bars were mis-set. Keep experiment 111's filter, and the next step is a filter that fixes the PV weight and moves the others; the verdict is inconclusive. | also relevant |
| 00116 differentiable-physics-nra-edge-flows-between-primaries | Flows on the electricity network's interconnection edges, fitted to a frozen convex model's residuals with a sparse-step penalty, recover injected conserved transfers between MW-metered Primaries; removing them gives a normal running arrangement (NRA) model trained without hand labels that forecasts no worse outside events; and fed causally into the Kalman filter as a known input, they improve the metered forecast during and after a transfer. | inconclusive | not-for-production | The method that looks for load moved between neighbouring substations did not work: with the settings we fixed in advance it found under a quarter of the load we added artificially, and it changed no forecast. Looser settings found more, but they also found load moving between substations that are not connected, probably the slow drifts of demand that our model does not describe. The gate failed for a structural reason: at the settings we fixed in advance, the fit cannot find any transfer shorter than 7 to 9 days, and the longest test transfer lasts under 5 days, so the failure is a limit of the method as specified, not a bug in the solver. The zero flow we first saw on the unconnected pairs was also an artefact of the order the fit visits its paths, so those pairs were never clean. Adding a drift term to the model, or accepting a flow only where the two substations are connected, is the next thing to test. | also relevant |
| 00115 differentiable-physics-w1-diurnal-shape-state-on-ens | GAM-K's remaining day-ahead error on ENS is mostly in the shape of the day inside each hour band, and that shape drifts faster than the convex model's monthly knots follow. A 48-half-hour shape state with zero mean in each hour band, added to the Kalman filter and updated from every reading the telemetry delay allows, lowers GAM-K's day-ahead quantile score at most decision substations, in months that played no part in choosing the idea's form and in the scored year. | inconclusive | not-for-production | Teaching the physics-based forecast the shape of the day from recent readings lowers its day-ahead error by 0.075 points at five Primary substations (about 3%), in two separate periods and more in winter (0.11, a post hoc breakdown), but averaging the last 14 days' errors by half-hour gains as much, so the learned shape state is not needed on this evidence. The shape state's own check failed: it learns an injected change too slowly (0.35 against a floor of 0.4), so the verdict is capped at inconclusive. A faster shape state, tried afterwards and labelled post hoc, passed that check and beat both K0 and the recipe, which a rerun with the check fixed in advance would need to confirm. | also relevant |
| 00114 differentiable-physics-n3-n4-interactions-and-heating-threshold | Two labelled claims about the convex model of Primary demand in the form Part 6 made the DP's candidate (GAM-K: 24 months, no drift, pinned capacity, Kalman filter), on past weather. N3: interaction terms (heating by day-type group, heating on eight sub-band blocks, lighting by day-type group), each shrunk towards the base by a difference penalty, lower the day-ahead quantile score after the filter. N4: a per-site heating threshold, chosen by profiling the training loss, is recoverable on synthetic substations at real noise, is identified away from 15.5 C at one or more real substations under the per-site printing rule, and lowers the score after the filter. | partly supported | not-for-production | Splitting the convex model's heating terms between the first and second half of each part of the day lowered its day-ahead error after the recent-error filter by about 0.04 points (97.5% interval 0.014 to 0.070), at all 5 substations in both years, but the gain label rests on one substation (D5), and the gain is a temperature-dependent shape of the day more than a heating effect. Letting heating or lighting differ between weekdays and weekends adds nothing, and choosing a heating threshold separately for each substation did not clearly help (0.01 points, within noise). Every number uses past temperature, not forecasts, so the gain needs a test on weather forecasts before GAM-K's convex model changes; the PV panel-orientation mix (N13) was dropped because earlier experiments found it unidentified. |
also relevant |
| 00111 differentiable-physics-6-ens-forecast-engine-against-bg2 | Driven by ECMWF ENS forecasts, the convex DP demand model forecasts Primary demand up to 7 days ahead better than lagless XGBoost given the same recent-level correction, and, used as the base margin of the BG2 recipe's lagged XGBoost, makes the production recipe better like for like. | partly supported | candidate | Driven by real ensemble weather forecasts, the physics-based demand model with a filter on its recent errors (GAM-K) scored 2.80 points against 2.78 for the production recipe (BG2) at five Primary substations. The 0.02 difference cannot be told from zero (-0.109 to +0.157), and the comparison is not like for like. GAM-K scored 0.13 points better than lagless XGBoost with the same filter (-0.305 to +0.014, 4 of 5 substations better), so the verdict is partly supported. The filter's gain is the firmly supported part: 0.15 points, 0.39 in the first day, with the sign held in the replicate months. Giving lagged XGBoost the physics model's forecast as a starting point changed nothing measurable. The known-answer gate passed 22 of 22 readings on its second run, after a sign error in the gate's own test data (both runs are kept). The departures were that the XGBoost compute estimate was 20 times too low, so the margin arms refitted monthly; lagless XGBoost for the scored year reused SIMP1's refits and had no spread factor; and the confirmation comparison was not run. GAM-K is a candidate for Flexpectation v2's roughly 1,300 substations on the measured filter gain, with four caveats: five substations over one year, a short XGBoost history, the undecided XGBoost comparison, and a margin that gave no gain. | also relevant |
| 00108 differentiable-physics-shared-anomaly-factor-across-substations | The residuals of the frozen convex demand model carry a component shared across Primaries, and a shared state estimated from the other substations' residuals, available at the forecast origin, lowers the day-ahead quantile score by more than the per-site Kalman filter of Part 4 already does. | inconclusive | not-for-production | The experiment stopped at its known-answer gate, where three of seven tests failed, so it cannot say whether a shared factor improves the forecast. On injected anomalies the shared filter, tuned on 2023-24, carried under 5% of the injected points (the plan asked for 40% or more), and it corrected no better than the per-site filter (2.402 against 2.397 points of mean absolute error over days 2 and 3 of the episodes). Part of the test was mis-specified, so a corrected test would be needed before any real score. | primary |
| 00106 differentiable-physics-4-all-years-drift-and-recent-level | The differentiable-physics model trained on every usable year from 2019-09 with a drift term forecasts Primary demand better than the same model on its best window, and assimilating recent residuals through a Kalman filter makes it better again, by more than a simple moving-average correction does. | partly supported | not-for-production | Correcting each day-ahead demand forecast with the errors the model made up to 3.5 hours earlier lowered the quantile score by 0.61 points (from 2.90 to 2.29) in the scored year and by 1.02 points in the year before, at all five long-history substations. The gain relies on estimates of past weather that a live service would not have, and against lagged XGBoost at the same lead the filtered model scored 2.349 against 2.487, a difference that is not significant. Whether old years of history help is not settled, because the contrast changed sign between the two years, so the verdict is partly supported. | also relevant |
| 00102 simp1-simplifying-the-bg2-best-guess-before-production | A single weekly-refitted XGBoost model without the three power lags, the solar offset, the 14-day correction, or the blend scores within 0.15 points of capacity of BG2 at the 16 Primary substations, within 0.30 at the 4 bulk and grid supply points, within 0.10 at the 6 solar farms, and within 0.30 at the 3 wind farms. We expect the Primary claim to fail and the other three to be close calls (see the prediction). Separately, the solar offset helps most where it should, at midday in summer at the 5 Primary substations that have one. | partly supported | candidate | Removing the three recent-power inputs from the best-guess XGBoost forecast makes it worse at Primary substations (by 0.36 points of capacity once one substation's commissioning months are cut, and most of that comes from one substation whose demand level fell in March 2026) and at supply points (by 0.28). The wind farm forecasts lost nothing measurable and the solar farm forecasts little. The blend of 5 models can shrink to 3, and the solar offset lowers the score by 0.18 points at midday in summer at the 5 substations that use it. | also relevant |
| 00100 drop1-random-lag-dropout-widens-intervals | An XGBoost quantile model for Primary substation demand that is trained with labelled switching half-hours removed from its targets and with its lagged power inputs set to missing at random intervals, and that has those lags set to missing when they lie inside a labelled switching event, widens its 5% to 95% band on the half-hours whose lags are missing, so that the band's coverage moves from 0.784 towards the nominal 0.90, without widening the band elsewhere. | inconclusive | not-for-production | Training the demand forecast with lags hidden at random did not make its band wider on half-hours where a lag is missing: on real rows the band covered 0.80 (hidden in 10% of training rows) to 0.84 (30%) of actuals, against a nominal 0.90. Hiding lags in 30% of rows did remove the accuracy loss that masking alone causes on injected transfers (-0.02 points, 95% interval -0.35 to +0.38 against the plain model). Hiding lags in 10% of rows left +0.26 (98.75% interval -0.15 to +0.85), not separable from zero or from the loss. An input that tells the XGBoost model a lag is missing, or a separate XGBoost model of the quantile spread, are untested routes to a wider band. | also relevant |
| 00021 error-correction-variants-bsp-gsp-demand | A recent-error correction of N1's quantiles, measured with power that is 2 days old, lowers the quantile score of the 4 bulk supply point (BSP) and grid supply point (GSP) demand series as it does for the 16 Primary substations, and some variant of the correction (a 7-day or 28-day window, multiplicative, per-half-hour-of-day, or weighting recent days more) beats the plain 14-day additive correction. | inconclusive | not-for-production | The 14-day additive correction lowered the Primary quantile score by 0.99 points of capacity (99.375% interval -1.82 to -0.18) when the newest power was 2 days old, as it did with 3.5-hour-old power in experiment 19. On the 4 bulk and grid supply point series it raised the score by 0.36 (-0.19 to +0.98), so it is unproven there. No variant beat the plain additive 14-day form: a 7-day window was worse (+0.19) and a multiplicative form far worse (+2.28). | primary |
| 00018 staged-chain-vs-n1-serving-latency-lag-anomalies-demand | For the 16 Primary substation demand series, the staged chain (an ERA5-trained stage 1 and the stage 2 calibrator, with lag anomalies built only from weather that serving has at the forecast origin, and NGED's real telemetry delay) beats the single-stage ENS-trained quantile model N1 and manual_heuristic on the quantile score. |
inconclusive | not-for-production | With lagged inputs built only from data a live service has, including NGED's real telemetry delay, the two-stage chain was not distinguishable from the single-stage N1 over all leads: 6.41 against 6.33, a difference of +0.08 points of capacity (99.17% interval -0.11 to +0.25; positive means the chain is worse). Up to 48 hours the chain was 0.27 lower (-0.62 to +0.03). It scored 0.52 above the stored manual_heuristic baseline (-0.25 to +1.25), so the staged design is not supported for demand on this evidence. | primary |
| 00008 lag-summaries-in-stage2-calibrator-demand | For Primary substation demand series, giving the stage 2 calibrator the minimum, mean, and maximum of the most recent 24 hours of power, and of the most recent day on the same weekday as the target, lowers the probabilistic forecast's error below that of the calibrator with only the raw power lags (experiment 00005's L3). | inconclusive | not-for-production | Adding the minimum, mean, and maximum of the last 24 hours of power, and of the same weekday, to a calibrator that already had raw power lags changed the quantile score by -0.02 points of capacity up to 48 hours ahead (98.75% interval -0.14 to +0.13), which cannot be told from no change. Replacing the raw lags with the summaries worsened the score by 0.29 (+0.11 to +0.51), so keep the raw lags. Summaries alone, with no estimate-of-past-weather residual, beat no recent power by 1.18 (-1.83 to -0.58). | primary |
| 00005 lag-placement-stage1b-vs-stage2-demand | For substation demand series, recent observed power improves a two-stage probabilistic forecast more when it enters stage 1 through a residual-persistence XGBoost model (stage 1b) than when it enters only the stage 2 calibrator, when both placements are given identical inputs and both outputs are calibrated. | refuted | not-for-production | For Primary substations, recent power lowered the forecast error more when it went into the second-stage calibrator than when it went into a stage 1b correction model. Up to 48 hours ahead the calibrator scored 5.31 against 6.14, so stage 1b was worse by 0.84 points of capacity (99% interval +0.28 to +1.50); using both added at most 0.12 points. These are ceilings, because the lagged inputs used estimates of past weather that a live service lacks, and stage 1b was better at 8 of the 16 series. | primary |