- Root cause, one bug in three costumes. Leads 2 and 3 are the same defect: the engine reported a loss curve it did not integrate.
anchor_p = min(1.0, 1.0/rps[0])equalsprobs[0]exactly, so the guardanchor_p > probs[0]is dead for every return period ≥ 1 — and every RP inhazard_cache.jsonstarts at 2.0 or 5.0. The bandp ∈ [0.5, 1]therefore carried no loss: on a flat curve that is exactly a third of the answer (L·0.5computed instead ofL·0.75). Separately, a trapezoid across a 2.5x RP gap is a chord, not an integral. Fixed by anchoring atp = 1and integrating on a 16x-refined log-RP grid, which is also the grid now reported — reporting the coarse input while integrating something else is how a number and the curve drawn beside it stop being the same object. - The
ponytail:comment at oldengine.py:105was confessing to the same thing, and its defence was wrong. It argued the protection ramp "errs high, which is the right direction". It does err high, but the size of the error was set by grid spacing, not by physics: tj-priok/riverine moved -31.1% under refinement. An error whose magnitude is an artefact of the input grid is not conservatism. Inserting a zero-loss knot immediately below the standard makes the step a step; INT-STEP went from 46.1% off its knotted reference to inside 2%, and the same knot removed most of the convergence drift. - Convergence and grid density. 16 sub-intervals per modelled interval is comfortably enough: portfolio drift against a 32x-finer grid reads -0.00%, so this is not sitting on the tolerance. Cost is ~4x eval runtime (0.64s → 2.6s), which is the price of
loss_curvedoing ~130 points instead of 9 across ~650 calls per sweep. Don't raise the constant without a measured reason. - Lead 1 is a different root cause — still open, and now confirmed by reading, not guessing.
compute.asset_detaildoeseal = sum(r.lc.eal for r in results);compute.summarydoeseal = sum(r.lc.eal for r in event_results)whereevent_resultsfilters outr.permanent. tj-priok's coastal reading is 0.819 damage at 1-in-2 and near-flat, so it classifies permanent: summary reports 3,452,760 plus a 344,037,120 write-down,asset_detailreports 263,968,456 of "annual" damage. The summary path is right.asset_detailneeds the same split, including awritedownfield, or it is billing standing water annually. (My change moved both numbers; the divergence is untouched.) - VUL-MDF is a one-line fix but not a one-line change.
PerilResult.mean_damage_fractionreturnsmax(damage_fractions)while its docstring promises the EAL as a share of value — off by ~100x at tj-priok (0.8987 vs 0.008221). It is the only input todowntime_days, so correcting it moves business interruption,annual_net_cost, NPV, impairment, DSCR and the covenant breach count on every asset. Worth its own experiment so the anchor movement is attributable. - Looked up: nothing. No web search was needed and none was run. The two domain questions — whether an EAL integrates the full
[0,1]exceedance-probability domain anchored at zero loss, and whether a FLOPROS standard is a step or a ramp — were already settled inside the repo:research/invariants.py:82-93states the anchoring convention in its own docstring, andengine.loss_at_return_periodalready declared log-RP as the module's interpolation rule. Recording this so the next experiment doesn't spend budget re-confirming it. - No invariant looked wrong. INT-DOMAIN pins the engine to a linear-in-p trapezoid over its reported grid to 1e-9, which rules out substituting a closed-form integral of the log-RP interpolant (I derived it:
L_a(p_a - p_b) + m(p_a - (1+S)p_b)withS = ln(p_a/p_b)). That is not a flaw in the check — it forces the reported curve and the integrated curve to be the same object, which is the stronger property. Confirmed dead end; don't retry the analytic route.
- One root cause in two places: an expectation and a per-event quantity swapped for each other at the engine→finance boundary.
PerilResult.mean_damage_fractionreturnedmax(damage_fractions)— the damage of the worst modelled event — where its docstring and every caller wanted the EAL as a share of value. That feddowntime_days, so rotterdam-chem was billed 162 outage days a year and its expected business interruption came out 9.5x its expected physical damage (now 1,792 against 36,061). In the other direction,_insurance_recovery(eal, ...)subtracted a per-event deductible from an annual expectation:max(0, eal - 0.02·value)asks whether the average year clears the deductible, and for any site with an EAL under 2% of value the answer is no, however many modelled events are total losses. Fixing both moved VUL-MDF and INS-PAYS. summaryandasset_detailtookmax(mean_damage_fraction)across perils; that had to becomesum. A max is right for a peak and wrong for an expectation — expected shares of value add, the way the EALs they come from add. Leaving the max would have handedtranslatean outage driver that did not correspond to the EAL passed next to it.- Looked up: catastrophe financial modules apply deductibles, limits and coinsurance to the ground-up loss of each event, and take the AAL afterwards, from the net event losses (NAIC CIPR, Catastrophe Models (Property); CAS reinsurance bootcamp, Sigona 2019). That settles the direction: expected recovery is an expectation over the event distribution, never a subtraction from its mean.
API-SHAPEfreezestranslateto(eal, mean_damage_fraction, fin, a), so the event loss table cannot reach the finance layer. Severity has to be rebuilt from one moment plus a shape. Confirmed dead end for the next experiment: do not try to thread the loss curve intotranslate— the signature is an anchor. Adding a field toAssetFinancialswould work mechanically but gives the eval and the dashboard two different recovery numbers for the same asset, which is the lead-1 sin.- The shape choice is load-bearing and I got it wrong once. A maximum-entropy exponential (mean = EAL) is the tempting one-moment answer and it fixed 4 of 5 assets, but it left hcmc-tower recovering exactly zero: its EAL is 0.02% of value against a 20m worst modelled event, and an exponential prices an event 387x its mean at ~1e-36. Switched to the single-parameter Pareto, the standard severity model for excess-of-loss property pricing (CAS, A Practical Guide to the Single Parameter Pareto; CAS, Sahasrabuddhe, Single Parameter Pareto Revisited), with the shape exposed as
Assumptions.severity_tail_index = 1.5rather than buried, per this module's own rule that nothing is hidden in a constant. - Negative result worth recording: recovery is not monotone in
severity_tail_index. I asserted it was and the self-check caught me. Lowering alpha fattens the tail but also drags the Pareto scale down, so for a layer near the mean (900k EAL, 1m deductible) a fatter tail recovers less; only out where the deductible sits far above the EAL does fatter mean more. The self-check now pins the far-tail case, which is the one the fix exists for. - Known ceiling, and the obvious next move if the signature ever thaws: one moment plus a fixed shape cannot represent "rare and huge". For hcmc-tower a rough integration of the actual loss curve gives an expected recovery near 38k; the Pareto gives 1,711. The number is honest about being a severity assumption and errs toward the retained side, but it is an assumption where the engine already has the answer in
lc.losses. - Lead 1 is untouched and is now the loudest thing left.
asset_detail('tj-priok')runstranslateon 263,968,456 of "annual" damage whilesummaryuses 3,452,760, so the detail page reports a 149,956,862 insurance recovery and a 356,357,417 premium against standing water. The permanent-inundation split exists insummaryand in_asset_eal;asset_detailis the only path missing it, and it needs awritedownfield to say so. - No invariant looked wrong. COV-ATTRIBUTABLE (hcmc-tower breaches at
dscr_before0.73) is a real defect I deliberately left failing:covenant_breachmeans "breaches after climate loss" when a climate report needs "breaches because of climate loss", i.e.breach_after and not breach_before. It is a one-line definitional fix infinance.translatebut a different root cause from this experiment, so it belongs to its own commit rather than riding along unattributed. Breach count fell 7 → 4 as a side effect of the BI correction; hcmc-tower is the one that structurally cannot fall out.
- Root cause of VUL-OVERLAP: the combination was at the wrong layer.
_asset_perilsintegratedinundation_coastalandinundation_riverineas two independent loss curves and every consumer added their damage fractions. At cat-lai that is 0.56 + 1.00 = 1.563 at 1-in-1000: one building, destroyed 1.56 times. Two flood layers at one point are two ways for the same site to end up under the same water; they must be combined before the vulnerability curve is applied, not after. - Looked up, and it settles the method: FEMA Guidance Document 76 §4.4 "Combined Effects: Surge Plus Riverine Runoff". At each flood level Z, add the sources' rates of occurrence of exceeding Z —
R_T(Z) = R_riverine(Z) + R_surge(Z)— then read the combined level off at the rate of interest. Rates add on a common level; damage fractions never add on a common return period. FEMA states the assumption explicitly (independent and non-concurrent, "acceptable if storms that produce extreme rainfall and runoff are not the same as the storms that produce the greatest storm surge") and flags compound surge/runoff events as the case to check.engine.combine_exceedanceis this procedure;_rate_aboveis the inverse of_interp_log_rp, so it lives in the same log-RP space the module already declared. The FEMA combined coastal-riverine floodplain guidance is the mapping companion and points at Doc 76 for the arithmetic. - A defence is a cap on its own source's rate, not on the combined curve. cat-lai has a 1-in-8.375 FLOPROS standard on riverine and none on coastal; a river levee holds back the river and not the sea.
R_i(Z) → min(R_i(Z), 1/sop_i)before summing is exactly equivalent to the loss-zeroing steploss_curveapplies to a single source, expressed on the rate axis, so the merged reading arrives already defended and is integrated withprotection_rp=None. That is also why the merged peril must be namedcombined_flood:prot.sopreturns None for it, which is what lets INT-STEP, INT-CONVERGE-* and INT-MONO-INTENSITY reconstruct the curve from(reading, curve, sop)and get the same number back. Confirmed dead end: naming the merged result after one of its sources breaks INT-STEP, because the reconstruction re-applies a standard that is already baked in. - Numbers: the merge is non-additive, not a deletion. cat-lai riverine alone 32.4m, coastal alone 98.2m, old sum 130.6m, combined 119.1m — strictly between max and sum, as a union of rates must be. The combined 1-in-5 depth (0.729 m) is below riverine's undefended 1-in-5 (1.355 m) and that is correct: the levee stops the 1-in-5 river event, so what the site sees at that rate is sea water.
- Both paths are now one function, so lead 1's class of bug is gone.
_asset_ealis a one-line filter over_asset_perils(a, scenario, variant_rank, curve_rank). The headline is rank (0, 0). The zero-intensity screen, the permanent-inundation split and the flood merge can no longer be added to one path and forgotten in the other. (Lead 1 itself —asset_detailreporting 263,968,456 of "annual" damage for tj-priok wheresummaryreports 3,452,760 — is still open and untouched; it needs theevent_results/perm_resultssplit and awritedownfield inasset_detail.) - Honest regression I could not fix from the three editable files, and it is the most important thing in this log.
accumulation._bookiterateshz.PERILSand matches_asset_perilsoutput by peril name.combined_floodis not in the hazard cache's peril list, so cat-lai's and port-klang's flood units vanish: 4 exclusions of kindunknown("dropped upstream, reason unclassified"),accumulation.demo()fails its "every exclusion must be classified" assertion, and 91.5% of the portfolio EAL is now absent from the accumulation view. This is not cosmetic:FOOTPRINTSgives riverine a 100 km radius (basin synchrony, Berghuijs et al. 2019) and coastal 200 km (alongshore surge, Haigh et al. 2016), so a merged flood unit genuinely has no single correlation length. The fix is to attribute the combined loss back to the sources by each one's share of the combined rate,w_i(rp) = R_i(Z(rp)) / R_T(Z(rp))— the weights sum to 1 by construction, so VUL-OVERLAP stays held. - Confirmed dead end: do not try that attribution by splitting the merged
LossCurveinto per-sourcePerilResults. I worked it through.w_ifalls with return period as riverine takes over, so an allocated coastal loss curve decreases with rarity and fails INT-MONO-RP; and INT-CONVERGE-PERIL / INT-MONO-INTENSITY rebuild each result fromloss_curve(r.reading…), which an allocated slice cannot reproduce. The attribution has to be a separate output consumed byaccumulation, not a re-split of the integrated curve — which means it needsaccumulation.pyandhazard.pyto be editable. - cat-lai is now 38.4% of value a year and 82% of the portfolio EAL, and its coastal reading is the same standing-water signature as tj-priok's, sitting 0.13 outside the classifier. 0.604 m already present at 1-in-2, rising only 1.63x out to 1-in-1000, against
PERMANENT_FLATNESS = 1.5andPERMANENT_MIN_DEPTH = 0.25. That is what an EAL of 98.2m on a 310m asset from the coastal layer alone means. This is the loudest remaining modelling number and it is a threshold question, not an arithmetic one; it wants its own experiment with an argued flatness criterion, not a nudged constant. - No invariant looked wrong. DET-RUNID, SWEEP-DISTINCT and COV-ATTRIBUTABLE are all real defects with diagnoses already recorded (run id hashes a string literal instead of portfolio contents;
n_variantsis the max across perils sohz.readclamps duplicates into 24 of the 54 sweep points;covenant_breachmeans "breaches after" where a climate report needs "breaches because of"). All three are separate root causes and were deliberately left for their own commits so this one's movement stays attributable.
- DET-RUNID root cause and fix.
_run_idhashed"demo", a literal, so every portfolio in the world under the same scenario and assumptions shared a run id.report.py:331already tells the reader the identifier "is a hash of the portfolio"; now it is one. Asset digests are sorted before hashing, because a portfolio is a set of assets and not the order someone listed them in — that keeps DET-ORDER holding. - SWEEP-DISTINCT is unsatisfiable as the harness computes it, and I left it failing. The check counts
len(c.sweep)wherec.sweepis the harness's own re-enumeration ofSPREAD_SCENARIOS × len(hz.variants(p)) × 3through_asset_eal. The store cannot answer 54 distinct requests with 54 distinct datasets. It holds 30: 2 pathways × 5 riverine members × 3 curve configurations. Verified exhaustively, all facts aboutdata/hazard_cache.json, not about the code:- ssp126 and ssp245 are byte-identical. Over all 12 assets × 3 priced perils × 6 ranks, zero differences in path or intensities.
scenario_substitutiondiscloses why: WRI ships no RCP2.6, both are servedinuncoast_rcp4p5_*/inunriver_rcp4p5_*. - Riverine ranks 0 and 1 are the same GCM.
hz.variantslistsWATCHfirst — the observational-forcing member, which exists only underhistorical— sohz.read's provider fallback serves NorESM1-M for rank 0 as well as rank 1. Five usable members, not six. - Wind has exactly one resolvable variant per site. No cached point carries both
irisandwisc; Asia/US are iris-only, Rotterdam and Hamburg wisc-only. The wind axis contributes no ensemble spread at all. - No choice of
SPREAD_SCENARIOScan rescue it: the inner loops alone are 6 × 3 = 18 requests per scenario against 15 available models. Making it pass would require fabricating model diversity the data does not have — the exact failure the anchors exist to catch. The check would hold as written if it readsummary()["headline"]["eal_spread"]instead of re-deriving the sweep, which is what the product now reports: n = 30, all distinct.
- ssp126 and ssp245 are byte-identical. Over all 12 assets × 3 priced perils × 6 ranks, zero differences in path or intensities.
- What the fix actually changes for the user.
portfolio_spreadnow identifies a point by what it read —(pathway, sorted (asset, peril, source path, curve id))— and skips a configuration already swept. Reported n falls 54 → 30,by_driverlevels fall to the true counts (climate model 5 not 6, scenario 2 not 3), low and high are unchanged (a duplicate can never be the min or max), and the median moves to 69,008,549.asset_detailnow callsportfolio_spread([asset])rather than keeping a second copy of the loop. math.fsumis load-bearing, not tidiness. Accumulating peril EALs with+=made the sweep's low and high differ by one ulp under portfolio reversal, which broke DET-ORDER (headline dicts are compared for exact equality). A running float total is only associative to within a rounding error;fsumover a sorted list is exactly rounded and order-independent. The oldsum(_asset_eal(...) for a in assets)had the same exposure and passed by luck.- Open, and small:
vulnerability_curvereports 4 levels for a 3-rank axis. Verified cause — at curve rank 0 the portfolio uses 6 curve ids under rcp8p5 but 5 under rcp4p5, because an asset gains exposure under the hotter pathway and brings its regional curve with it. The label is a set of curve ids, so an exposure change splits one curve level in two and bleeds a little scenario signal into the curve attribution. Not fixed: every level named is a genuinely different set of curves, so nothing is duplicated, only sub-divided. - Deliberately not touched, and still the loudest things left. COV-ATTRIBUTABLE:
covenant_breachmeans "breaches after climate loss" where a climate report needs "breaches because of it" (breach_after and not breach_before); hcmc-tower hasdscr_before0.73 and breaches at zero hazard. One line infinance.translate, different root cause, own commit. And cat-lai remains 38.4% of value a year off a coastal curve sitting 0.13 outsidePERMANENT_FLATNESS— a threshold question that wants an argued criterion, not a nudged constant.
- COV-ATTRIBUTABLE root cause:
covenant_breachmeant "is in breach after climate cost", where a climate report needs "is in breach because of it". hcmc-tower has a DSCR of 0.73 on its own numbers — 14.4m of debt service against 10.5m of NOI — so it is in default before a drop of water falls.translatealready computesdscr_beforeandltv_before, so the pre-existing state needed no second call to anything: the fix is to test both floors before and after and report only the difference. Portfolio breaches 4 → 3, EAL and every other headline byte-identical. - Tested per covenant, not in aggregate, and that is the load-bearing detail.
{"dscr","ltv"} after − before, notbreach_after and not breach_before. hcmc-tower is already through DSCR while its LTV is 0.70; if climate impairment pushes LTV past 0.75 that is a new default trigger a lender acts on, and an aggregate rule would silently swallow it. Thecrushedcase infinance.demo()pins exactly that geometry (1m EAL on the hcmc shape → ltv_after 0.80,covenant_breachandcovenant_breach_beforeboth true). - Nothing is dropped, it is split.
covenant_breach_beforeon every asset row andcovenant_breaches_pre_existingin the headline. A site in default at zero hazard is a real credit finding; it is just not this model's finding, and deleting it to make the count fall would be the hiding the anchors exist to catch. - Open, and needs a file I cannot edit:
report.py:639still labels the headline "Assets breaching a debt covenant after climate cost". That label now under-counts by one — 4 assets do breach after climate cost, 3 because of it. The number and the per-asset column agree with each other and with the API, so the document is internally consistent; only the row's wording is stale. One-word fix ("after" → "because of", plus the pre-existing count beside it) wheneverreport.pyis in scope.src/components/views.tsx:121,722inherit the new meaning correctly without change. FinancialImpacthas exactly one construction site (finance.py:232), so the new field breaks no caller;types.tsis unextended and the extra keys are inert.- Deliberately not touched. SWEEP-DISTINCT: experiment 4's exhaustive account of
data/hazard_cache.jsonstands — 30 datasets against 54 rank requests, ssp126/ssp245 byte-identical, riverine rank 0 falling back to rank 1's GCM, one resolvable wind variant per site. Nothing in the three editable files can create model diversity the store does not hold, so it stays failing honestly. And cat-lai is still 38.4% of value a year off a coastal curve sitting 0.13 outsidePERMANENT_FLATNESS— a threshold question wanting an argued criterion, still the loudest number in the run.