re-run-note-v6.md, sha256 9d81da7841dc). Portfolio edition: Kith Climate cohort case, fictional client. Small grey tags such as J-081 point to entries in the project record (journal, findings, decisions), so every statement can be traced.Re-run note: how to update Northwood's carbon inventory
Version 6, 5 October 2026 (replaces v5; step 8 now uses run-gate-checks-v7.py for every gate, step 4 says to add files under new names, the label check is version 2, the manifest is version 3 with a verifier, and the account of what was tested is corrected, findings F-127 to F-135. Earlier: v5 replaced v4; the mirror section now says the mirror is kept at this engagement and describes the logged no-mirror mode, and step 1 names the handover manifest, findings F-118 and F-119. Earlier: v4 replaced v3; step 7 now builds the dashboard with build-dashboard-v4.py, which prints the mandatory roster label, and adds a label check; the example suffixes continue from the signed set, finding F-111. Earlier: v3 named dashboard v14 and its check report v16. Earlier: v2 replaced v1; adds the traceability pass, in which each statement that rests on a fact, number or rule carries a tag such as [N-07] pointing to its row in the claims register, and a separate check re-tests every row against the record). One of three leave-behind documents, with the methodology note (what is calculated and why) and the reference data pack (where each outside number comes from). This note says how to repeat the work next quarter or next year, using only what is left behind.
What this note is, and is not. The steps below were assembled from the scripts' own instructions and the build record. On 4 October 2026 the builder ran the calculation, checking and dashboard sequence once in a scratch copy of the engagement folder, on the unchanged 2025 data, with the commands of the first version of this note: the figures reproduced (320 of 320 match) and every independent check passed (C-07 to C-10, C-11 to C-13, C-19, C-20). The dashboard step has changed since then, and on 5 October the builder repeated steps 2, 6 and 7 in a scratch copy with the commands as written in version 4 of this note; the run logs of those copies are not part of the record. Steps 3, 4, 5 and 8 and the no-mirror option have not been run in a copy. Nobody has run it on new data, or without the builder's help. Treat your first re-run as a trial: do it in a copy (step 1), and confirm in step 2 that you reproduce the current figures before you change any data. [N-01, N-02, N-03]
What you need
- A computer with Python 3.9 or later (the sequence was tested on Python 3.9). Every script uses only Python's built-in tools except three, which also need the free package
openpyxl: rebuilding the emission factors, the Stage 0 review script (not part of a re-run), and the gate-2 runner, whose check C-03 compares every factor with the EPA workbook. So you needopenpyxlwhenever you run gate 2, not only when the factors change. [N-04, N-05] - The engagement folder as it was handed over (about 120 MB with the reference files). The scripts find their files by their position in that folder, so keep the folder layout. [N-06]
- Someone to type the commands. No coding is needed: every step is one command, with the exact text below. A person also has to look at the dashboard in a browser once (step 7).
- Your own name for the log: set
PIPELINE_OPERATORbefore running anything, so the run log shows who ran it. In a Mac or Linux terminal:export PIPELINE_OPERATOR="Your Name". [N-07]
Two things to know before you start
- The report date lives in two places. The calculation reads
pipeline/pipeline-config-v2.json; the dashboard and tables steps readpipeline/pipeline-config-v3.json. Both havestaleness.report_date(currently 2026-10-02). Set both to your actual report date, or the two halves will disagree about which figures are stale. The date is the one you issue the report on, not a date inside the data. A figure's age counts from the end of the period it covers. [N-08, N-09, N-10, N-11] - Scripts never overwrite. Each one writes a new, labelled output and stops with "refusing to overwrite" if it already exists. Most take
--suffix(for example--suffix v3) to name the new output. A few scripts write fixed file names (the roster, subregion, emission-factor and mirror-loader scripts, and the plausibility file the later steps read). Before re-running those, move the old output into an archive folder; do not delete it. [N-12, N-13, N-14]
Step 1: make a working copy
Copy the whole engagement folder to a new place, for example northwood-carbon-2026q4. Work only in the copy. The original stays as the record of what was signed.
The files you need are listed with their fingerprints in fund-level/leave-behind/handover-manifest-v3.csv. Check your copy against it with python3 verify-handover-manifest-v1.py from my-work/week-3/pipeline/; it lists any file that is missing or changed, and lets the record files (journal, decisions, findings, run log) have grown. [N-52]
Step 2: check the starting point
From my-work/week-3/fund-level/leave-behind/reference-data/, run python3 verify-reference-pack-v1.py. All 14 files should report ok. This confirms the reference files are the ones the signed inventory used. [N-15]
Then confirm your copy reproduces the signed figures before you change any data. From my-work/week-3/pipeline/, run the calculation again on the unchanged inputs with the original report date, and compare:
python3 calculate-emissions-v2.py --suffix v3 python3 ../fund-level/leave-behind/re-run/compare-ledger-figures-v1.py outputs/ledger-figures-v2.csv outputs/ledger-figures-v3.csv
It should report 320 figures matching and none differing. If it does, any later difference comes from the data you change. If not, something other than the data changed: the report date, a settings file or a reference file. This test writes a ledger named v3, so your real update in step 6 uses the next number (v4); the numbers below allow for that. [N-16, N-17]
Step 3: refresh the outside data
Follow the refresh list in the reference data pack readme (reference-data-pack-readme-v2.md): emission factors, eGRID boundaries, leak rates, vehicle fuel use, the three carbon prices and the euro-dollar rate. After any change, update pipeline/reference-sources-v1.csv, then run from pipeline/:
python3 build-reference-pack-v1.py --suffix v2
For new emission factors only, first run python3 build-emission-factors-v3.py (after moving the old emission-factors-us-v3.csv aside), and point the settings file at the new factor file (paths.ef_v3 in config v2 and v3). The carbon prices and their file fingerprints go in config v3 under carbon_prices.
Step 4: put in the new company data
Add, in the data room folder, the new year's files under new names, and point the settings at them. Never overwrite a file the scripts have already read: the gate check C-02 fails for any input whose contents differ between logged runs, and a new freeze does not clear that. The files are:
- carbon collection template (settings key
paths.template; keep the same columns, because the repair rule and the calculation read them by column name), [N-19] - facility locations with latitude and longitude (
paths.geocoded), - spine files for revenue, EBITDA, headcount and ownership (
paths.spine_portfolioandportco_kpis.csv), - company questionnaires and operating records (org charts, status files, board decks).
Update staleness.data_room_vintage and calculation.template_as_of_date in the settings (the end date of the period the data covers), and the ownership months under ownership_months_2025 if ownership changed. Note that these keys are named for 2025; the period label calculation.period and reporting_year also need changing. [N-18]
Step 5: redo the judgment files by hand
These files hold decisions that only a person can make from the evidence. They are not generated. Each row cites the data room line it rests on, and the methodology note says what each rule means. Redo them for the new data:
File (in pipeline/) | What it holds | Evidence it needs |
|---|---|---|
roster-reconciliation-register-v2.csv | One disposition per site (consistent, hold-unverified and so on). Re-run python3 roster-reconciliation-v2.py after archiving the old one; it proposes the dispositions, and you review every change | Each company's own org charts and status files [N-20] |
outputs/site-subregion-map-v2.csv | Each site's eGRID subregion. Re-run python3 assign-egrid-subregion-v2.py after archiving the old one | Site coordinates and the boundary files [N-21] |
data-corrections-v1.csv | Values that evidence shows are wrong, with the corrected treatment (rule R10) | A re-runnable test, or the company's confirmation [N-22] |
refrigerant-evidence-v3.csv | One row per refrigerant unit: type, charge, equipment type, service record or default leak rate | Template notes, questionnaires, EPA Table 1 [N-23] |
questionnaire-totals-v1.csv, questionnaire-fills-v1.csv, data-gaps-v1.csv | Company totals for comparison (rule R2), figures used in place of template figures, and rows with no data | Questionnaires [N-24] |
fleet-vehicle-counts-v1.csv | Vehicles and vehicle class per company, for the plausibility test | Questionnaires and row notes [N-25] |
energy-spend-v2.csv, fy2025-alternatives-v1.csv, scope3-flags-v3.csv | Energy spend, alternative revenue and EBITDA, and Scope 3 flags, each with a quoted source line | Questionnaires and board decks [N-26] |
Save each new version beside the old one with a new version number. Most of these files are found through the settings file (paths keys in config v2 and v3: corrections, data_gaps, questionnaire_fills, questionnaire_totals, refrigerant_evidence, roster_register, subregion_map), so point the key at the new file. Four file names are written into the scripts instead: fleet-vehicle-counts-v1.csv (plausibility test), and energy-spend-v2.csv, fy2025-alternatives-v1.csv and scope3-flags-v3.csv (tables and dashboard). For those, either keep the exact name for the new data (archiving the old file first), or have someone who can edit the scripts point them at new names. [N-27, N-28]
Step 6: run the calculation and its checks
From my-work/week-3/pipeline/, in this order. Replace each version number with the next unused one. The signed set is ledger v2, calculation check report v3, tables v6, dashboard v14 and dashboard check report v16; the step 2 test used ledger v3, so the examples below continue from there. [N-29]
python3 calculate-emissions-v2.py --suffix v4 python3 plausibility-r5-v2.py --ledger v4 --suffix v2 python3 check-calculation-v2.py --ledger v4 --suffix v4
- The calculation writes the ledger (one line per site and fuel), the figures and the figure-to-line links. Each script ends by printing its run ID.
- The plausibility test must be written as
--suffix v2, because the next steps readr5-plausibility-v2.csvby that name. Move the old file into an archive folder first. - The independent check rebuilds every line from the raw template and must agree to 0.001 kg CO2e per facility. If it does not, do not go on; the report names the line that differs. [N-30]
Step 7: tables and dashboard, and the browser check
python3 g34-build-tables-v1.py --ledger v4 --suffix v7 python3 build-dashboard-v4.py --tables v7 --ledger v4 --suffix v15 --prices-confirmed python3 check-g34-v1.py --tables v7 --ledger v4 --dashboard ../fund-level/emissions-dashboard-v15.html --suffix v17 python3 check-roster-label-v2.py --ledger v4 --dashboard ../fund-level/emissions-dashboard-v15.html
--prices-confirmedstates that Northwood confirmed the carbon price files. Add it only when that is true.- The dashboard builder is
build-dashboard-v4.py. It prints the mandatory "roster unverified at site level" label beside every company whose records cannot test its site list.check-roster-label-v2.pyrecomputes from the ledger which companies need the label and fails if the page does not match. The olderbuild-dashboard-v3.pydoes not print the label; do not use it. [N-47, N-48, N-49] - Browser check (a person does this). Open the new dashboard file in a browser. Confirm there are no error messages in the browser console, that nothing scrolls sideways at a phone width (375 pixels) or a desktop width (1280 pixels), and that the facility drilldown opens. Then record the result so the gate can tie it to the exact file: [N-31, N-32]
python3 record-browser-check-v1.py --dashboard ../fund-level/emissions-dashboard-v15.html --console-errors 0 --widths 375,1280 --overflow-px 0 --drilldown-ok yes --note "checked by NAME"
If you rebuild the dashboard again, the browser check must be done again, because it is tied to the file's fingerprint.
Step 8: write up the runs, then run the gate checks
Journal first. The gate check C-16 fails until every script run in the run log is cited in a journal entry, and every changed file is named in one by exact path. Each script prints its run ID; cite it in the entry as run: followed by the first 8 characters. For example: [N-33]
python3 record_tools.py journal --gate G5 --step "Re-run for 2026 Q4" --action "..." --reason "..." --evidence "run:2b989241; run:3b1b8c2c; run:e0b6f0f3" --outputs "my-work/week-3/pipeline/outputs/calculation-ledger-v4.csv; ..." --result "..." python3 record_tools.py decision --decision-id D-... --decision "..." --rationale "..." python3 generate-build-record-v1.py
Add a decision entry for any rule you change. The record files are append-only: never edit an earlier row.
Then the gate checks:
python3 run-gate-checks-v7.py --gate 2 python3 run-gate-checks-v7.py --gate 4
Use run-gate-checks-v7.py for every gate, not the older runners: it runs them with check C-16 corrected so that a failed run cannot satisfy itself (finding F-112), and with the no-mirror option described below. [N-53] The close (gate 5) also needs a handoff note: add --handoff <your handoff note>; every figure and status line in it must cite a journal, decision, finding, disposition or run entry (check C-17). [N-54]
Gate 2 runs the record and calculation checks (C-01 to C-04, C-14, C-16, C-18) and reads the calculation results. Gate 4 runs the dashboard checks, including the browser check (C-15). When you follow the order above, they should all pass, except C-18 if you keep no mirror (next section). In the scratch test, with the steps run but no journal entry written, C-16 failed and C-15 failed until the browser check was recorded: that is the system working as designed. [N-34]
What happens with the Supabase mirror
The inventory is built and checked from local files; the local export stays the authoritative copy. The mirror in Supabase is a convenience copy. At this engagement the mirror is kept after close in Daniel's personal Supabase project (decision D-G5-8); how long it is kept has not been decided. [N-50]
The gate runner's check C-18 fails if there is no record in the run log of a mirror comparison. If you keep no mirror, record a decision in the decision log that accepts running without one, then run the gate checks with the no-mirror option: python3 run-gate-checks-v7.py --gate 4 --no-mirror --no-mirror-decision D-... (it works on gates 2, 4 and 5; the decision must say that you run without a mirror). C-18 is then logged as not applicable, accepted under that decision, instead of failing (decision D-G5-9). Do not edit the runner to hide it. If you do keep a mirror: a free Supabase project pauses after a week of low activity and can be restored for 90 days after that, so check it does not lapse. [N-35, N-51, N-36]
How to add market-based Scope 2
The inventory reports location-based Scope 2 only (decision D-G1-10), because there was no data on contractual instruments (renewable energy certificates, power purchase agreements, utility-specific emission rates) in the data room. To add market-based Scope 2: [N-37]
- Collect instrument data per site: the kWh covered by each instrument and its emission rate or certificate, from the company or its utility.
- Add the factors to the reference pack: the emission rate for any utility-specific rate, and a residual-mix factor for the electricity not covered by an instrument. Record the publisher, edition and download date in
reference-sources-v1.csvlike any other source. - Add a market-based stream to the calculation. This is a script change, not a settings change: the calculator today computes electricity only from the grid subregion factor. A new calculator version should compute market-based lines beside the location-based lines (never instead of them), tag them, and add them to the independent check. Under the GHG Protocol, report both figures. [N-38]
- Record the change as a decision and re-run the checks.
What does not carry over automatically
These were found during the build and left in place on purpose. Allow for them:
- Four hand-written input files, and the plausibility file the later steps read, are named in the code (steps 5 to 7).
- Some scripts write fixed output names, so old outputs need archiving first (step 5).
- The calculation line IDs use the template row number, so a new template with rows in a different order will not line up with the old IDs. [N-39]
- The ledger's fingerprint includes the run ID, so two identical runs give different file fingerprints. Compare figures, not files (step 2). [N-40]
- The run-log append has no file lock: do not run two scripts at the same moment. [N-41]
- The checker recognises cited text and rule labels by pattern. A new rule label may need the checker updated. [N-42]
- Exit multiples come from the data spine as Northwood's own assumption, and the browser check is done by hand. [N-43]
- The settings keys carry "2025" in their names. [N-44]
If something goes wrong
| What you see | What it means | What to do |
|---|---|---|
| "refusing to overwrite existing output" | A file with that name exists | Use a new --suffix, or archive the old file for a fixed-name script [N-45] |
| "audit-log chain is broken" | The run log was edited or truncated | Stop. Restore the log from your copy of the original and repeat from there [N-46] |
| The independent check disagrees with the calculator | A bug, or a judgment file the two read differently | Read the line it names; fix the input file; do not widen the tolerance |
| C-16 fails | A changed file is not named in a journal entry | Add a journal entry naming each file by its exact path |
| C-02 fails | A pinned input file changed since the last freeze | Revert it, or keep the new contents under a new file name and point the settings at it (step 4) |
| C-18 fails | No mirror comparison on record | See the mirror section above |
| The dashboard shows a number with no figure ID | An orphan number | Do not publish; the check C-19 should have caught it, so find which step was skipped |