Generator forecasting: experiments
This page lists every experiment tried in the theme, newest first. An experiment that belongs mainly to another theme but bears on this one is marked "also relevant".
| Experiment | Hypothesis | Verdict | Production status | Take-home message | Relevance |
|---|---|---|---|---|---|
| 00357 wind-default-2-monthly-head-longer-retest | Retraining Wind farms research default 2's mixture head on the first day of every month (experiment 356's arm u3) gains over default 2 at true days 1 to 10, because a recent head tracks the farms' drift. Experiment 356 measured -0.038 points on 53 Tuesday runs (interval -0.104 to +0.024) and -0.120 on about 16 Tuesday runs of 2026-07 onwards. | partly supported | candidate | Retraining Wind farms research default 2's head every month improves the forecast slightly, by about 0.04 to 0.05 points over the scored year and about 0.10 points in the latest 2.5 months, and no scope shows harm to the score. The scored-year interval still touches zero, so the change is a candidate and is not adopted. The upper tail stays too heavy (1.7% to 2.0% of observations above the 0.99 quantile), so retraining does not fix it. | primary |
| 00356 wind-default-2-blend-and-monthly-refit-retest | Wind farms research default 2 (experiment 347's arm y2) gains 0.234 points over Wind farms research default 1 at true days 1 to 10 but puts 2.28% of observations above q99 over every lead (2.15% at true days 1 to 10). Either a per-quantile, per-lead-band blend of the two defaults (u2), experiment 348's arm z5 (u1), a monthly offset layer on default 2 (u4), or a monthly retrained head (u3) lowers the 0.99 pinball loss and the share above q99 without losing the gain. | not supported | not-for-production | Blending Wind farms research default 2 with research default 1, or refitting it every month, does not fix its upper tail: the share of observations above the 0.99 quantile only falls from 2.15% to between 1.77% and 2.04%, against 1% intended, and no arm clearly beats default 2 over days 1 to 10. The blend's small gain (0.05 points) is the same with shuffled weights, so it is averaging, not learning. The top quantile needs new information, not a blend or a refit; the monthly retrained head is the one arm worth a longer retest. | primary |
| 00348 wind-plant-mdn-upper-tail | Experiment 347's best arm (y2: Stage 3 frozen physics plant, 51 ensemble members through it, a three-component Gaussian mixture head on the summary of the resulting power values) beats Wind farms research default 1 by 0.234 points over true days 1 to 10 but is over-confident in the upper tail (2.3% of observations above q99 over every lead, and a 0.99 pinball loss up to 0.084 worse in experiment 347's report). One of five tail fixes (z1 tail-weighted loss, z2 capped quantiles with a point mass at the plant's scale, z3 monotone increments above q90, z4 z1 plus z2, z5 the solar recipe of a plant residual band blended with the default's XGBoost quantiles) brings the share above q99 under 1.5% and the 0.99 pinball loss to within the tolerance, while keeping the headline gain. | partly supported | candidate | The frozen physics plant with a mixture head stays about 0.23 points of capacity better than the research default over days 1 to 10, but none of the four tail fixes inside the head brings the top quantile into calibration without losing score at that quantile, and none of their intervals excludes zero. Blending the plant's residual band with the default's XGBoost quantiles gains less (0.15 points) but is the arm of the five whose gain is clear and whose 0.99 quantile is no worse, so that recipe, not a better head, is the next step. | primary |
| 00347 wind-physics-hybrid-plant-then-mdn | A physics plant model of each metered wind farm (a smooth cubic speed-to-power curve with cut-in, rated, and cut-out speeds, a direction-dependent loss, and priors that make the untrained model already sensible), fitted on ERA5 before 2024-04-01 and frozen, turns each of the 51 ensemble members into a power value. A three-component Gaussian mixture-density network (MDN) fed the summary of those 51 values, the lead, and the farm then scores within 0.15 points of Wind farms research default 1 on the kept rows at 1 to 10 days, without a neural adapter. A second plant fitted after masking the blocks where the farm fell short of the first plant (a healthy-farm plant) lets the head learn the shortfall. | partly supported | candidate | A physics model of each wind farm, run on every ensemble member and corrected by a small neural network, forecasts 0.27 points of capacity (2.4%) better than the research default over leads of 0.5 hours to 15 days, and 0.23 points over true days 1 to 10, but the neural network given only the plant's median and the farm already gets 83% of the gain, and the upper tail is still over-confident. The next step is to fix the upper tail (the neural network caps its quantiles at capacity, and 0.9% of observations exceed it) before any plant-based arm is adopted. | primary |
| 00346 wind-research-default-1-offset-table-refit-cadence | Experiment 342 found that the default's offset table, fitted once on the 442 daily runs of 2024-04-01 to 2025-06-16, leaves the upper tail too low in the scored year (observed power above q99 in 1.8% of half-hours against 1% intended) although it fits the hindcast, and blamed drift between the hindcast years and the scored year. The ensemble table holds daily runs up to 2026-10-24, so the table can be refitted each month on runs that had already finished. A table refitted monthly on a trailing 6-month window (x2) scores lower than the table fitted once. | partly supported | candidate | Refitting the wind forecast's offset table every month on recent runs makes the forecast a little better (a 0.4% to 1.1% lower quantile score), using only data the service already stores. It does not fix the upper quantiles, which still miss in 1.7% of half-hours against 1% intended. So the change between the training years and the scored year is a minor cause, and the upper-tail miss needs another explanation. A monthly job that refits the table on all earlier daily runs passes the adoption conditions and is a candidate. | primary |
| 00345 pooled-solar-wind-mixture-density-network | One mixture-density network (MDN) trained once for all leads on the six metered solar farms and the wind farms together, with a fuel-type indicator and weather inputs common to both fuels, scores as well on each fuel as that fuel's best single-family model, and scores better for a farm with no history. | partly supported | candidate | One neural network trained on all six solar farms forecasts each farm as well as a separate network on the physical plant model (within 0.8%), so a production forecast could use one neural network per fuel. Adding wind farms to the same network does not help either fuel and makes solar 0.7% to 2.3% worse, so solar and wind are better trained separately. In this experiment, a wind farm with no history scored 18% worse at day 1 when trained on public wind farms alone than when trained on other metered wind farms. | also relevant |
| 00344 wind-research-default-1-speed-dependent-offsets | Experiment 340 found that a constant upward shift of the wind quantiles gains 0.09 to 0.12 points on the kept rows but worsens the rows below cut-in by 0.51 to 0.78, and that the default's median is too low at 8 to 15 m/s and too high below 5 m/s. An offset that depends on the ensemble-mean 100 m wind speed at forecast time lifts the quantiles where the power curve is steep and leaves them alone where the forecast is near zero, so it keeps the gain without the low-wind harm. The starting point is Wind farms research default 1, the three metered wind farms W1 to W3 are scored, and the lead range is 1 to 10 days. | not supported | not-for-production | Letting the wind forecast's quantile offsets depend on the forecast wind speed gains only about 0.02 points of capacity when fitted on the same retrospective forecasts (hindcast runs) that fit the quantile offset table, and the gain cannot be told from zero. Fitted on the six months just before the scored year (July 2025 to June 2026), the same speed-dependent correction gains 0.146 points and does not worsen the half-hours where the forecast wind speed is below the turbines' cut-in speed. The speed-dependent bias is therefore real, and it changes from year to year. Refitting the correction on recent months is the untested option that could capture it. | primary |
| 00343 wind-research-default-1-train-on-ensemble-mean | Wind farms research default 1 trains XGBoost on ERA5 and runs it on each of the 51 ensemble members, then corrects with the offset table. The ensemble's 100 m wind speed at the grid cell is lower than at the turbines (a factor of 1.13 closed 78% of the gap to the ERA5 oracle in experiment 307). An XGBoost model trained on the 51-member mean of the ensemble's own history (arm M, 2024-04 to 2025-06, no calendar columns) would learn that bias and need less offset-table correction. Arm MS adds the member standard deviation of the 100 m and 10 m speeds. The three metered wind farms W1 to W3 are scored at 1 to 10 days. | not supported | not-for-production | A wind model trained on the ensemble mean needs almost no offset table (the table changes its score by 0.017 points, against 0.345 for the default), but its forecast is no better than the default's: 0.08 points worse on the headline rows, an amount the data cannot distinguish from zero, and worse at low winds and in the 0.99 quantile. The default's recipe (ERA5 training plus the table) stays; mean training is a simpler recipe that scores about the same. | primary |
| 00342 wind-research-default-1-upper-tail-offsets-and-head | The upper quantiles of Wind farms research default 1 are too low: in experiment 324 the observed power exceeded q99 in 1.8% of half-hours (1% nominal) and q95 in 6.7% (5% nominal). Three corrections fitted on the hindcast can fix that without harming the headline score. x1 is a table of offsets for q90 to q99 that also depends on the ensemble-mean wind speed. x2 is a small boosted-tree head, fed the default's quantiles and summaries of the ensemble members' wind speed, that adds offsets to q90, q95, and q99. x3 lets the share of ensemble members above the farm's saturation speed set a probability mass at the farm's scale. | not supported | not-for-production | The wind forecast's upper quantiles miss too often (observed power exceeds q99 in 1.8% of half-hours against 1% intended), and corrections fitted on past forecasts do not fix that: the offsets keyed by wind speed, the tail head, and the mass at the farm's cap all leave the share at 1.8% to 1.9% and change the upper-tail loss by about 1% or less. A head that sees the ensemble members does no better than one that sees only the default's quantiles, so in these 3 farms the members add no information about the upper tail. The remaining miss is consistent with the scored year differing from the hindcast years, which the next experiment could attack by refitting more often or by training on the most recent months. | primary |
| 00341 wind-capacity-steps-from-strong-wind | For the three metered wind farms (W1, W2, W3), a change-point search on the upper quantiles (99.9th percentile and maximum) of observed power in the half-hours when ERA5 wind is well above the speed at which the fitted power curve saturates finds a loss of 25% of capacity within 60 days at W1 and W2, with fewer false alarms than experiment 233's ratio monitor under a 9% seasonal swing. Steps come in whole turbines. Weighting lower-wind half-hours by the inverse of their speed-error variance speeds detection up. A neighbouring farm in the same ERA5 cell cancels the shared speed error. | not supported | not-for-production | A change in a wind farm's usable capacity shows up in time only when it is large: a loss of half the capacity is found within 60 days at 91% to 93% of start dates at all three farms, and a loss of a quarter at 26% to 77%. Looking only at the strongest winds, as proposed, does not help, because strong wind fills 0.18% to 1.1% of half-hours and most 28-day blocks hold too few of them. The real records show repaired episodes (a 2-week outage at W1, a loss of 17% and of 31% at W2, a loss of about one of two turbines at W3 from October 2024 to January 2025) and no permanent step. | primary |
| 00340 wind-research-default-1-constant-shift-then-low-wind-drop | Experiment 336 found that dropping the ERA5 training rows below the turbines' cut-in speed (x2) gains 0.117 points on the kept rows, and its audit traced about 85% of the gain to a constant upward shift of the quantiles. If that is right, a constant shift fitted on the 442-run hindcast (arm S) reproduces most of the gain without changing the training rows. Arm S+X2, which also drops the low-wind rows, then adds little beyond the shift. Wind farms research default 1 is the starting point, the three metered wind farms W1 to W3 are scored, and the lead range is 1 to 10 days. | not supported | not-for-production | A single upward shift of the wind quantiles, fitted on past forecasts, does nothing, because the forecast's offset table already fits those runs. Moving the quantiles up by 0.8 to 1.2 points of capacity does reproduce most of the gain of dropping low-wind training rows, but it also worsens the forecast at low winds by 0.5 to 0.8 points, the same harm that stops X2 being adopted. An untested option that could lift the forecast at 8 to 15 m/s and leave it alone below 5 m/s is a correction that depends on wind speed. | primary |
| 00339 wind-research-default-1-adapter-plus-mixture-head | A three-component Gaussian mixture head on the hindcast, fed the quantiles of the physics core plus adapter and the ensemble members' summaries of speed, direction, and temperature (experiment 324's W4), lowers the quantile score of Wind farms research default 1 on the kept rows of the three metered wind farms W1 to W3 at 1 to 10 day leads. The head loses on the idle half-hours because it is trained on kept rows only, and the maintainer's view is that a forecast-time rule (predict the idle floor where the ensemble-mean wind speed is below cut-in) needs no idle detector. Three arms are tested: x1 the head as in experiment 324, x2 x1 plus the rule, x3 the head trained on all hindcast half-hours. | partly supported | candidate | A small neural-network head that re-draws the physics-plus-adapter wind forecast's quantiles gains 0.27 points of capacity (2.4%) on the farms' normal half-hours and keeps that gain on the replicate months, but the same head given no weather summaries does as well, so the gain is a recalibration that a simpler correction partly reproduces. A simple rule that predicts the idle floor at low wind recovers none of the head's loss on the idle-band readings, and training the head on those readings removes the gain. | primary |
| 00338 wind-adapter-column-without-neural-network | The gain of experiment 326's arm (c), the neural adapter's effective wind speed as an extra XGBoost column, can be kept without a neural network: a lookup table of speed scale by direction sector, speed ratio bin, and temperature bin (x1) keeps at least half of the gain, and a small monotone gradient-boosted tree (x2) keeps at least 80%. | not supported | not-for-production | A lookup table of wind-speed corrections keeps almost none (5%) of the gain that the neural adapter gave the wind forecast, and boosted trees keep about half (0.051 of 0.099 points of capacity, scored at 1 to 10 day leads). Neither passes the adoption rule, so the neural adapter of experiment 326 is the only arm tested that does, and the maintainer decides whether it belongs in the production service. A non-neural column would need to reduce the physics-based wind-power model's error on ERA5 as much as the neural network does. Boosted trees with more inputs might approach that. | primary |
| 00336 wind-research-default-1-below-cut-in-zeroing | The maintainer proposes that, for wind, rows where the wind speed is below the turbines' cut-in speed carry only noise, so zeroing them lowers the quantile score of Wind farms research default 1 on the kept rows of the three metered wind farms W1 to W3 at 1 to 10 day leads. Three readings are tested, each added alone: x1 sets the forecast to zero power whenever the ensemble-mean 100 m speed is below the farm's cut-in speed, x2 drops the ERA5 training rows below cut-in from the XGBoost model and the offset table, and x3 sets the training target to zero on those rows. At most one is adopted, in the order x1, x2, x3. | partly supported | candidate | Setting the wind forecast, or the training target, to zero below the turbines' cut-in speed does not improve the forecast, because the farms still produce small real output there. Leaving the low-wind rows out of training gains 0.12 points of capacity (about 1% of the default's score) on the kept rows, but mostly by lifting the quantiles upward, which a constant upward shift may reproduce (not yet tested). | primary |
| 00326 wind-research-default-1-change-arms | Each of four changes, added alone to Wind farms research default 1, lowers the quantile score on the kept rows of the three metered wind farms W1 to W3 at 1 to 10 day leads: (a) 2 m temperature and an air-density column as XGBoost inputs, (b) hour of day and day of year as XGBoost inputs, (c) the ERA5-trained neural adapter's effective wind speed as an extra XGBoost column, and (d) one wind-speed scale factor, fitted on the hindcast to the implied wind, applied to every member's 100 m speed before the XGBoost model. At most one is adopted, in the order c, b, a, d. | supported | candidate | Feeding XGBoost the neural adapter's corrected wind speed lowers the wind forecast's score by 0.10 points of capacity, about 0.9% of the default's score, at 1 to 10 day leads (99.75% interval -0.19 to -0.01). The change is negative at every farm and at short and long leads. The three other changes were not separable from zero. Only a port that puts the adapter in the production service can use the gain, so the maintainer decides. | primary |
| 00324 wind-farm-solar-cross-breeds-long-run | The solar-farm findings (a mixture-density network (MDN) fed by a physical plant forecast beats the plant alone, lead features help it, and one pooled head across leads matches separate heads per lead band) carry over to the three metered wind farms W1 to W3, when the plant is the physics core with the experiment 285 wind-speed adapter and the weather is the 51-member ensemble. | partly supported | not-for-production | A small neural network that corrects the quantiles of the wind farms' physics-based forecast lowers the quantile score by 0.27 points of capacity against XGBoost plus a lookup table of offsets, on the half-hours when the farm is not idle, at days 1 to 2 and at days 3 to 10. The gain disappears when the idle half-hours are included. The upper tail gets no better. The network needs the farm's own forecast history to learn from. | primary |
| 00313 metered-farm-pure-xgboost-and-mdn-vs-plant | A predictor built only from weather and calendar inputs, with no plant model anywhere in the pipeline (not even as an input column), scores close to the plant-based forecast in own history and falls behind it where the plant carries what missing history or a long lead cannot. | partly supported | candidate | A forecast that sees only weather and the calendar, with no physical plant model anywhere, scores almost as well as the plant-based forecast for the six solar farms, up to 10 days ahead, even when the farm has no history. The plant's added value is small, between 0% and 3.4%, and its intervals include zero except for XGBoost. For wind, the same kind of network also matches the best plant-based forecast, so a plant model is not needed to be competitive on these farms. | also relevant |
| 00307 wind-farm-power-as-wind-sensor | The wind that the three metered wind farms' own power implies (the fitted physics core's power curve run backwards) is closer to a high-resolution analysis than to ERA5 at 9 to 14 m/s, and at least half of the ensemble's shortfall (observed power below the forecast's median on 34.1% of rows instead of 50%) is in the spread and timing of the members, not in their mean speed. | not supported | not-for-production | The ensemble's low wind-power forecast comes mostly from wind speeds that are too slow, by about 1.1 m/s on average and more at longer leads. Multiplying every member's wind speed by a factor fitted on the 14 months before the scored year closed about 78% of the gap to the ERA5 oracle, while widening the members or shifting them in time closed almost none. The median is slightly better calibrated when the factor is fitted to the wind that the farms' power implies than when it is fitted to ERA5 on the same rows (1.7 percentage points), but the quantile score is not better. | primary |
| 00303 wind-adapter-in-production-setting | Experiment 285 found that, with ERA5 as the weather, a small neural adapter in front of the physics core of a wind farm scored better than XGBoost. On the 51 ensemble members in the scored year, an adapter fine-tuned on the ensemble hindcast beats XGBoost plus experiment 259's offset table (the benchmark that matters), because the adapter corrects the wind speed itself and the table corrects only the power. | partly supported | not-for-production | A small neural correction of the wind speed, fitted to ERA5 and run on the 51 ensemble members, does about as well as XGBoost with a lookup table of offsets (0.09 points better, too little to justify porting). Fitting the correction on the one year of ensemble hindcasts made the forecast worse, probably because that year differs from the scored year; we did not test the cause. | primary |
| 00288 pooled-head-on-public-wind-farms | A neural head that adds 11 quantile offsets to the XGBoost wind forecast (R0) and was trained on dozens of public onshore wind farms (metered output from Elexon's Balancing Mechanism Reporting Service) lowers the quantile score of the three metered wind farms (W1 to W3) against R0 alone, and a head trained on the pool beats the same head trained on W1 to W3 only, because experiment 259's head had too few farm-years to learn a correction that transfers. A second hypothesis: XGBoost keeps improving as the training history grows from 12 to 48 months while the physics core is flat after 24 months, and a pool of public farms replaces the missing history. | not supported | not-for-production | Training the correction on 40 public wind farms did not help the forecasts for the three NGED wind farms. Applied directly, it made them worse, because the public farms' capacity, turbine rating, and hub height let the head memorise each farm. Without those inputs it did no harm, and after a short fine-tune on the three farms' own data it gained about as much as a simple offset table fitted on those farms, so the public farms added nothing measurable. | primary |
| 00285 wind-effective-wind-adapters | A small bounded correction of the wind speed that the physics core of a wind farm reads, learned from the wind direction, the 10 m to 100 m speed ratio, the 2 m temperature, and the hour of day, closes at least half of XGBoost's lead over the core on the three metered wind farms (W1, W2, W3) when ERA5 is the weather. A correction of each ensemble member's wind speed towards ERA5's removes the ensemble's low bias. | partly supported | candidate | The physics model of a wind farm matched XGBoost once a small neural network adjusted the wind speed it takes as input, so XGBoost's lead can be reproduced by a smooth correction of wind speed. On the ensemble weather, mapping each member's wind speed to ERA5's helped by 0.31 points but fixed only a small part of the ensemble's low bias, and a lookup table of power offsets by lead day helped more. | primary |
| 00276 wind-xgboost-vs-physics-gap-diagnostics | The XGBoost model beats the physics core on the three metered wind farms (W1, W2, W3) because it learns the metering noise (the reactive-power floor of the apparent-power series, the deadband, and the dip near cut-in) that the physics core ignores. | refuted | not-for-production | XGBoost beats the physics model of the wind farms because it corrects the wind speed using the wind direction, the 10 m wind, and the hour of day. It does not learn the metering noise: the lead is as large on rows without idle readings as on all rows. No change to the physics model's power curve, spread, or scale closed more than about a sixth of the gap, so the next step is a flexible correction to the physics model's wind speed. | primary |
| 00262 wind-aifs-members-and-climatology-by-lead | For the three metered wind farms (W1 to W3), a lead-dependent mixture of the ECMWF ensemble's member forecasts, the AIFS ensemble's member forecasts (both run through experiment 230's ERA5-trained XGBoost wind model A0r), and climatology lowers the quantile score and the 0.99 pinball loss against the ECMWF members alone, after both are corrected by a table of offsets per lead day, quantile, and farm; and a small neural gate that reads the two ensembles' spread and disagreement chooses better weights than weights by lead alone. | partly supported | candidate | Adding the AIFS ensemble's wind forecasts to the ECMWF ensemble's improves the wind forecast at 3 to 6 days ahead, by about 5% of the quantile score, but does not improve the high-power tail that matters for curtailment. Climatology and a small neural network that picks the mixing weights add nothing once the table of offsets has corrected the common low bias. The AIFS arms also need testing on a second year, because one year of Tuesday runs sets the weights. | primary |
| 00261 wind-low-bias-decomposition | The low bias in the three metered wind farms' forecast (observed power below the physics core's median on 34% of rows, not 50%) comes mostly from the ensemble's weather input, not from the core, the farms, or the mixing of the 51 members. | partly supported | not-for-production | The wind forecast's low median comes mostly from the ensemble's weather: its 10 m and 100 m wind speeds are slower than ERA5's, and the same physics core fed ERA5 wind has 52.8% of rows below its median. Mixing the 51 members adds about a tenth of the shift, and training the physics core and the XGBoost model on 9 months of ensemble forecasts removes between a fifth and a third of the gap. The fix to test next is a correction of the ensemble's wind speeds before the model reads them. | primary |
| 00259 wind-plant-plus-quantile-offset-head | For the three metered wind farms (W1 to W3), a small neural network that reads the 51-member mixture of a wind model and the ensemble's wind summaries, and adds an offset to each of 11 quantiles, lowers the quantile score against the wind model alone (R1, the physics core), matches the XGBoost best guess (R0), and lowers the pinball loss at the 0.99 quantile, once it can learn from daily runs of the ensemble forecast over 15 months (about 440 runs) instead of the 13 to 40 Tuesday runs that experiment 221's head saw. | partly supported | candidate | A correction learned from 15 months of daily ensemble forecasts lowered the wind forecast's quantile score by 4.5% to 5.5% on the three metered wind farms, but the neural network and the ensemble's wind summaries were not what earned it. The 51 ensemble members' median sits too low: the observed power was below the physics core's median on 34% of rows, not 50%. A lookup table with one offset per lead day, quantile, and farm recovers 92% of the gain. The open question is where that low bias comes from. | primary |
| 00256 wind-temperature-density-shear-columns | Adding 2 m air temperature (arm T1) to the six wind columns of the XGBoost wind model lowers the quantile score on the three metered wind farms (W1 to W3), because air density and stability change the power a turbine produces at a given forecast wind speed. A density column (T2) carries the same information as temperature, and a shear and solar-elevation column set (T3) adds a further gain at night. | partly supported | candidate | Giving the XGBoost wind forecast the air temperature helps a little, about 0.7% of the score, and the evidence just misses the bar we set beforehand. Columns that carry the time of day and the season help more, and their gain sits in daytime and summer rows. That pattern is not explained by the air density we could test (a function of temperature alone) or by shear, and it is consistent with a correction for biases in the ensemble wind at certain hours and seasons. A repeat on other months would show whether the time-of-day and season columns still help. | primary |
| 00242 wind-physics-core-part-3-member-tails | For the three metered wind farms (W1, W2, W3), running each of the 51 ensemble members through an ERA5-trained wind model and mixing the 51 forecasts gives better 0.01 and 0.99 quantiles than running the model once on the ensemble mean of the weather inputs and applying the same spread factor, because the model is non-linear near cut-in and rated wind, so a model of the mean weather is not the mean of the model. | supported | not-for-production | Keep running all 51 ensemble members through the wind model. Running the ensemble mean once raises the error at the 99% quantile by a factor of 2.1 for the physics model and 3.3 for XGBoost, probably because averaging the members removes the high winds that produce full output, and the single spread factor tried here does not repair that. The physics model and XGBoost differ by under 0.01 points on the tails, so the choice between them does not matter here. | primary |
| 00240 no-xgboost-forecast-sprinkle-part-2 | A small XGBoost residual model on the plant forecast's median error, an XGBoost spread model for the quantile offsets around it, or both, close at least half of the gap in quantile score between the XGBoost-free metered solar forecast (the plant fed the mean and spread of 51 ensemble members, experiment 224's arm 3) and the full XGBoost forecast (experiment 224's arm 5) at 12 to 36 hour leads. | partly supported | candidate | For solar farms, the best way to add XGBoost to the physical plant forecast is to let XGBoost predict the spread around the plant's mean, not to correct the mean: that forecast scores 4.3% better than the plant alone and 1.6% better than full XGBoost, and wins at the upper tail that matters for curtailment. Correcting the mean with XGBoost made the quantile score 0.18 points worse, about as much as a model trained on shuffled residuals. For wind, experiment 230's XGBoost stage on the physics core closes 71% of the gap to full XGBoost at 12 to 36 hours, and 44% over all leads. | also relevant |
| 00234 no-xgboost-forecast-audit-part-1 | For Primary substation demand, a metered solar farm, and a metered wind farm, the best forecast that uses no XGBoost anywhere scores within 2% of the quantile score of the best forecast that uses XGBoost, at 12 to 36 hour leads. | partly supported | candidate | An XGBoost-free forecast of Primary substation demand scores 7.5% better than BG2, the XGBoost best guess, at 12 to 36 hours ahead, on one year and six substations. The XGBoost-free solar and wind forecasts score 2.8% and 5.0% worse, and lose at the 0.01, 0.05, and 0.95 quantiles. Only the solar plant model's 0.99 quantile beats XGBoost, and that is the tail that matters for curtailment. The next experiment tests small XGBoost components inside the XGBoost-free solar and wind forecasts. | also relevant |
| 00233 wind-physics-core-part-2-capacity-availability | For the three metered wind farms (W1, W2, W3), the ratio of observed power to the power that the fitted physics core of experiment 230 expects, taken over the half-hours when the core expects rated power and averaged over a trailing 60 days, shows an injected capacity loss of 25% within 60 days of the loss starting, at every farm, and raises no alarm on a slow drift of the observed power of the size seen in the flat top of metered solar farms (experiment 183). | not supported | not-for-production | A monitor that compares observed power with the wind-turbine model only in strong wind, where the power curve is flat, cannot show a 25% loss of capacity in time at any of the three wind farms, because strong-wind half-hours are rare: a trailing 60-day window holds enough of them to give a reading on only 21% of days at W1, 45% at W2, and 5% at W3. A variant added after the results were seen, which uses every half-hour at 20% of maximum output or more, finds a 25% loss with a median delay of 27 days at W1, 52 at W2, and 34 at W3, but a smooth 9% seasonal swing then raises false alarms, and a 10% loss is found at only 27% to 73% of start dates. The next step to test is a monitor on the whole power curve with a seasonal baseline; the real series also shows single months with low output (W2 in winter 2025 to 2026, W3 in 2024 to 2025) that nothing here explains. | primary |
| 00230 wind-physics-core-part-1 | For the three metered wind farms (W1, W2, W3), a physics core fitted by gradient descent (hub-height wind from the ERA5 100 m and 10 m speeds, an air-density correction, a smooth monotone power curve with a cut-out, and an idle layer for the reactive-power floor of the MVA series) run through the 51 ENS members scores within 0.1 points of capacity of the XGBoost best guess (experiment 33) on the quantile score, and the measured gap between the ERA5-input ceiling and a simple power curve is under 0.02 points, which would mean the 25 km weather cell, and not the turbine physics, limits the forecast. | not supported | not-for-production | A physics model of a wind farm, fitted to the three metered farms, forecasts them slightly worse than the XGBoost best guess, so it should not replace it. Most of the forecast error comes from the error in the ensemble wind forecast, not from the power curve: with ERA5 wind as the forecast, the same physics core cuts the score by 43%. The idle layer and the small neural adapter add nothing, and training the physics core on ERA5 instead of the ensemble control member improves it by 0.4 points. | primary |
| 00223 dp-bsp-generators-as-children-and-double-count-test | The metered generators that NGED lists under a bulk supply point (BSP) are children of that BSP as much as the Primaries are: the BSP's reading is the Primaries' demand plus the unmetered load minus the generators' output. If the Primaries' readings already include a generator's output, adding that generator to the sum double-counts it. This experiment first tests, from the readings alone, which generators the BSP's remainder carries and which a Primary already carries, and then asks how much the BSP's forecast improves when the generators the BSP carries are summed as children. | inconclusive | not-for-production | The two bulk supply points' readings move with the metered generators that NGED lists under them, and the Primaries' readings show only small coefficients on the same generators. The size of the movement is not the clean one-for-one that adding the generator to the sum needs, so the experiment did not test whether adding the generators improves the bulk supply point forecast. A better test is to separate the unmetered solar and the battery's price-driven dispatch first. | also relevant |
| 00221 wind-correction-head-on-stored-predictions | A small neural-network head, fed the nine quantiles of the wind best guess (experiment 33's ERA5-trained mixture over the 51 ENS members, with the power-curve base margin and the spread factor) plus the ensemble mean and spread of the 100 m wind speed and the direction sector, lowers the quantile score at lead days 1 and 2 against the best guess by at least 0.1 points of capacity. The head is pooled over the three farms and trained on earlier months only. | inconclusive | not-for-production | Training a small neural network to correct our wind-farm forecast did not help at 1 to 2 days ahead: the best version scored 0.05 points better and the version that also sees the weather forecast scored 0.18 points worse, with intervals that include zero. With only three farms and 9 scored months, the network may learn the months it trains on and narrow the forecast band too far on the next month. The stored forecast's band was already near the best width. A different kind of correction, or many more farms, is the next thing to try. | primary |
| 00210 wind-idle-noise-mva-artefact | H0: the noise in the three metered wind series (W1 and W3 in MVA, W2 in MW) occurs only at low wind speed, because the meters have a deadband, the reactive power drifts, and reactive power control runs only while the turbines turn. If H0 holds, a rule on ERA5 100 m wind speed alone, with no observed power, finds the rows to drop from training and evaluation. | not supported | not-for-production | The noisy near-zero rows in the three metered wind series do not occur only at low wind. A rule on ERA5 wind speed alone drops about 9% to 15% of half-hours but still misses about 60% of the idle-band half-hours, probably because ERA5's 25 km cell is a poor guide to the wind at each farm (untested). The rows this experiment flags are stored for the main wind experiment to use, and W3's first 2 years are a candidate to drop. | primary |
| 00102 simp1-simplifying-the-bg2-best-guess-before-production | A single weekly-refitted XGBoost model without the three power lags, the solar offset, the 14-day correction, or the blend scores within 0.15 points of capacity of BG2 at the 16 Primary substations, within 0.30 at the 4 bulk and grid supply points, within 0.10 at the 6 solar farms, and within 0.30 at the 3 wind farms. We expect the Primary claim to fail and the other three to be close calls (see the prediction). Separately, the solar offset helps most where it should, at midday in summer at the 5 Primary substations that have one. | partly supported | candidate | Removing the three recent-power inputs from the best-guess XGBoost forecast makes it worse at Primary substations (by 0.36 points of capacity once one substation's commissioning months are cut, and most of that comes from one substation whose demand level fell in March 2026) and at supply points (by 0.28). The wind farm forecasts lost nothing measurable and the solar farm forecasts little. The blend of 5 models can shrink to 3, and the solar offset lowers the score by 0.18 points at midday in summer at the 5 substations that use it. | also relevant |
| 00072 p35-step-loss-monitor-for-solar-farms | For the 5 metered solar farms with a long record (labelled A, B, C, D, and F), a monitor can find a loss of generating capacity, such as a lost inverter, from the farm's own power telemetry. The monitor divides each day's power by the power a physics model expects on clear, unclipped hours, divides that ratio again by the same day's ratio at the other farms, and fits a piecewise-constant level to the result with an L0 penalty (an optimal-partitioning change-point fit). It finds a 25% loss that lasts 30 days, starting from April to September, within 28 days of its onset in at least 80% of injections, at no more than 0.5 false alarms per farm-year. It finds more injected losses of 10% than two simpler monitors that are tuned to the same false-alarm rate: a rolling median of the same signal, and the hysteresis rule of the capacity studies (called HYST here) run on the same signal. | refuted | not-for-production | A change-point monitor of a solar farm's daily output found 0.84 of 25% losses lasting 90 days and 0.45 of those lasting 30 days, below the 0.80 the plan needed, so the verdict is refuted. On 10% losses it did not beat a rolling median by an interval that excludes zero (+0.045, 97.5% interval -0.015 to +0.090). | primary |
| 00057 product-trained-xgboost-solar-farms | For the six metered solar farms, an XGBoost quantile model trained on ENS and given UKV's shortwave radiation, total cloud, and temperature as extra input columns scores lower on the quantile score at lead hours 24 to 54 than the same model trained on ENS alone, with every arm trained on the same window, rows, and seeds. UKV is read from the CEDA archive, which is not the live product, so the contrast is a ceiling read. | inconclusive | not-for-production | Giving UKV forecasts to an ENS-trained solar-farm model changed the quantile score by -0.035 points of capacity at 1 to 2 days ahead (95% interval -0.119 to +0.037), which is too small to resolve. The data cannot show a gain and cannot rule out the 0.10-point gain that would justify a new weather feed. UKV helped the six solar farms far less than it helped the three wind farms. | primary |
| 00054 product-trained-xgboost-wind-farms | For the three metered wind farms, an XGBoost quantile model trained on ENS and given UKV's wind as extra input columns scores lower on the quantile score at lead days 1 and 2 than the same model trained on ENS alone, with every arm trained on the same window, rows, and seeds. UKV is read from the CEDA archive, which is not the live product, so the contrast is a ceiling read. | small gain | not-for-production | Giving the Met Office's UKV wind forecast to an ENS-trained wind-farm model lowered the quantile score by about 0.2 points of capacity at 1 to 2 days ahead (95% interval 0.07 to 0.32), and a shuffled-UKV placebo shows the gain is information and not extra columns. The UKV data come from a research archive and not the live feed, so the gain is an upper limit on what a service could get. | primary |
| 00052 more-nwp-products-for-wind-farms | For the three metered wind farms, averaging the quantiles of the experiment 33 wind best guess (the ERA5-trained mixture over the 51 ENS members, with the power-curve margin and the spread factor) with the quantiles of the same ERA5-trained model fed ICON-EU's and IFS 0.25 degree's forecast wind lowers the quantile score at 1 to 2 days ahead, with weights chosen only on earlier months. | inconclusive | not-for-production | Averaging the wind best guess with two extra weather forecasts, ICON-EU and the ECMWF high-resolution forecast, scored 0.055 points of capacity better at lead days 1 and 2, but the 95% interval (-0.200 to +0.072) includes zero. Three wind farms cannot show whether the extra forecasts help, and the blend scored 0.122 worse at days 3 to 6. | primary |
| 00029 physics-stage1-solar-pytorch | For the 6 metered solar farms, replacing the XGBoost stage 1 with a differentiable PyTorch physics model (priors on tilt, azimuth, DC capacity, and the AC-to-DC ratio, fitted to metered power with CAMS irradiance and ERA5 temperature), applied to each ENS member and followed by a per-farm quantile XGBoost stage 2, gives a lower continuous ranked probability score (CRPS) than the pooled XGBoost mixture of experiment 00020. | refuted | not-for-production | C1, PyTorch physics stage 1 minus the XGBoost stage 1: +0.079 (98.75% interval +0.029 to +0.135), worse. | primary |
| 00026 capacity-features-solar-wind | For the 6 solar farms and 3 wind farms, giving each farm's stage 1a quantile model a feature that states the farm's recent effective capacity (the 99th percentile of its power at the steps with the most irradiance for solar, or the most wind for wind, over the trailing 28 days) lowers the continuous ranked probability score (CRPS) of the pooled mixture below that of the same model without the feature. | inconclusive | not-for-production | C1, the single 28-day feature minus no feature: solar -0.018 (98.75% interval -0.162 to +0.096), wind +0.013 (-0.054 to +0.101). | primary |
| 00023 lagged-power-and-error-correction-solar-wind | For the 6 solar farms and 3 wind farms, giving the stage 2 calibrator the farm's recent observed power (lagged power, or summaries of the latest day), or shifting the forecast by its own recent error (the 14-day correction of experiments 00017 and 00019), lowers the continuous ranked probability score (CRPS) below that of the same calibrator without recent power. | inconclusive | not-for-production | C1, lagged power added to the calibrator: solar -0.012 (99.58% interval -0.122 to +0.088), wind +0.012 (-0.090 to +0.146). | primary |
| 00012 solar-wind-stage1a-inputs-cams-and-training-start | For the ERA5-trained stage 1a XGBoost model of the 6 solar farms and 3 wind farms, applied to the 51 ENS members and averaged, at least one change to the inputs (CAMS irradiance in place of ERA5 shortwave as the training input, a stage 0 model that calibrates ENS irradiance to CAMS, a clear-sky-ratio target for solar, or a power-curve base margin for wind) lowers the mean absolute error of the member average. | inconclusive | not-for-production | C1, CAMS-trained minus ERA5-trained, solar: -0.140 (99.5% interval -0.277 to -0.044), short of the -0.15 that counts as a gain. | primary |