Differentiable physics: the network: experiments
This page lists every experiment tried in the theme, newest first. An experiment that belongs mainly to another theme but bears on this one is marked "also relevant".
| Experiment | Hypothesis | Verdict | Production status | Take-home message | Relevance |
|---|---|---|---|---|---|
| 00223 dp-bsp-generators-as-children-and-double-count-test | The metered generators that NGED lists under a bulk supply point (BSP) are children of that BSP as much as the Primaries are: the BSP's reading is the Primaries' demand plus the unmetered load minus the generators' output. If the Primaries' readings already include a generator's output, adding that generator to the sum double-counts it. This experiment first tests, from the readings alone, which generators the BSP's remainder carries and which a Primary already carries, and then asks how much the BSP's forecast improves when the generators the BSP carries are summed as children. | inconclusive | not-for-production | The two bulk supply points' readings move with the metered generators that NGED lists under them, and the Primaries' readings show only small coefficients on the same generators. The size of the movement is not the clean one-for-one that adding the generator to the sum needs, so the experiment did not test whether adding the generators improves the bulk supply point forecast. A better test is to separate the unmetered solar and the battery's price-driven dispatch first. | also relevant |
| 00204 dp-bsp-full-bottom-up-with-signed-mva-children | A bulk supply point (BSP) is forecast better bottom-up than top-down when the sum covers every child Primary. Experiment 161 summed only the three children per BSP that report real power (MW) and have a long history, which reached 42% of D13's effective capacity and 56% of D17's, and the bottom-up forecast scored 0.5% to 0.6% worse than the top-down forecast. This experiment adds the children that report apparent power (MVA). A child's apparent-power reading is the magnitude of its flow, so a child with reverse flow at midday needs a signed flow before it can enter a sum. The fold-aware model of experiment 152 supplies that signed flow as a latent variable. | inconclusive | not-for-production | Adding every child Primary to the sum did not improve the forecast of either bulk supply point: the bottom-up score matched the top-down score (+0.1% at D13, -0.15% at D17), and the intervals include zero. Converting the children that report apparent power to signed flow changed the score by at most 0.03 points, at the one bulk supply point where a child has reverse flow. Even with every child's reading known exactly the BSP's score gains only 12% to 24%, and the children's forecasts gave no detectable gain over the BSP's own chain, so the next gain may lie in forecasting the generators and the unmetered load beneath each BSP. | also relevant |
| 00161 dp-bsp-top-down-against-bottom-up | A bulk supply point (BSP) is forecast better bottom-up than top-down. The top-down forecast is the best-guess GAM-K chain (experiment 155's arm D) applied to the BSP's reading as a stand-alone series. The bottom-up forecast adds the arm D forecasts of the BSP's child Primary substations that report real power, each times a weight learned from earlier readings, to arm D's forecast of the remainder (the BSP's reading minus those children's weighted readings). The mechanism is that the child Primaries' own forecasts are better than the BSP's chain can infer from the BSP's sum, because each child has its own heating, solar, and calendar response and its own recent errors, and the sum is not exact (losses, unmetered load, children that report apparent power), so the remainder and the weights absorb the difference. | inconclusive | not-for-production | Forecasting a BSP from its children's forecasts plus a remainder did not beat forecasting the BSP as a stand-alone series at either BSP: the bottom-up score was 0.6% worse at D13 and 0.5% worse at D17. The likely reason is that the children this test could use (three real-power Primaries per BSP) cover 42% and 56% of the BSP's capacity, and the BSP's error lives in the rest. The oracle control also missed its pre-registered threshold, which was set too high. The verdict is inconclusive because both intervals include zero. |
primary |
| 00131 differentiable-physics-nra-edge-flows-drift-term | Adding a slowly varying offset to each substation, fitted jointly with the flows on the interconnection edges, lets the flow fit find injected transfers that last hours to days without putting flow on pairs of substations that are not connected, which experiment 116's fit could not do. | inconclusive | not-for-production | Letting each substation drift slowly did reduce the load that the fit wrongly moves between substations that are not connected, by half to three quarters at an equal or higher detection rate. It did not remove that false load, and the settings that keep it near zero find only about half of the transfers we added artificially and report them too small. In the replicate year part of the false load sits in two jumps in one substation's data that nobody had labelled. We therefore cannot yet read these flows as switching events. | primary |
| 00116 differentiable-physics-nra-edge-flows-between-primaries | Flows on the electricity network's interconnection edges, fitted to a frozen convex model's residuals with a sparse-step penalty, recover injected conserved transfers between MW-metered Primaries; removing them gives a normal running arrangement (NRA) model trained without hand labels that forecasts no worse outside events; and fed causally into the Kalman filter as a known input, they improve the metered forecast during and after a transfer. | inconclusive | not-for-production | The method that looks for load moved between neighbouring substations did not work: with the settings we fixed in advance it found under a quarter of the load we added artificially, and it changed no forecast. Looser settings found more, but they also found load moving between substations that are not connected, probably the slow drifts of demand that our model does not describe. The gate failed for a structural reason: at the settings we fixed in advance, the fit cannot find any transfer shorter than 7 to 9 days, and the longest test transfer lasts under 5 days, so the failure is a limit of the method as specified, not a bug in the solver. The zero flow we first saw on the unconnected pairs was also an artefact of the order the fit visits its paths, so those pairs were never clean. Adding a drift term to the model, or accepting a flow only where the two substations are connected, is the next thing to test. | primary |