Method and caveats

What was compared

Each test case is one DOE/PNNL prototype building generated by BEMGen's DOE generator at the prototype's own dimensions, with the skylights of its programs' atlas family laid out as the source does, the prototype's atlas programs (deterministic ruleset schedules; no daylighting control, no unoccupied lighting cap), the atlas 90.1 unified construction set of its vintage and climate zone, and the pinned TMY3 weather of the zone's representative city. The reference site is the prototype's full zoning (NoSimplification + Stack, IWHT and FHT on); the test sites are the zoning simplifications with geometry, programs, envelope and weather fixed. Ideal-loads sensible heating and cooling are the primary outputs; HVAC and service hot water are excluded.

The reference of a cell is the prototype's full zoning as BEMGen writes it: No simplification + Stack, with interior-wall heat transfer (IWHT) and floor heat transfer (FHT) on. The test sites combine a zoning simplifier with a floor stacker:

Simplifier	What it does
Conditioned merge	conditioned rooms of one floor merge; unconditioned rooms stay
No simplification	every room of the prototype is a zone (the reference)
Perimeter core	each floor is cut into perimeter zones of the given depth and a core
Semantic merge	rooms of one program on one floor merge
Single zone per floor	each floor is one zone

Perimeter core is run at 3 m and 4.57 m depth.

Stacker	What it does
Single zone building	the building is one zone
Single zone per floor type	each floor type (ground, middle, top) is one zone
Stack	every floor is written
Stacked floor zone multiplier	identical middle floors are one floor with a multiplier

The main matrix has 5 to 23 generated zonings per cell; the cells of the Hospital, Large hotel, Outpatient have fewer because a stacker or a perimeter depth does not apply to that prototype (multiplied rooms, or a wing narrower than twice the perimeter depth): the generator refuses those zonings, and each cell lists them as not generated. Orientation is a cell axis: the two orientations of one prototype, climate zone and vintage are two rows, and the headline over the pair is the robust rule below.

Metrics and threshold

Every value shown is the batch runner's own paired metric of a test site against its reference, as the report writer stored it; nothing is recomputed on this site. The annual error is the relative difference of annual ideal sensible heating or cooling; CV(RMSE) and NMBE are the hourly coefficient of variation of the root-mean-square error and the normalised mean bias error of ASHRAE Guideline 14; the peak error is the relative difference of the annual maximum hourly load; the area change is the change of conditioned floor area against the reference. The JSON files carry further metrics (P99 peaks, load-duration-curve errors, Kolmogorov–Smirnov distances, conditioned-temperature errors and zone-count ratios) that the pages do not show; the serial costs are the cost objects, shown as the cost columns.

Basis: ASHRAE Guideline 14 hourly limits (CV(RMSE) 30 %, NMBE 10 %), annual error 10 %, conditioned area 1 %; proposed, the owner decides
Loads judged: Sensible heating, Sensible cooling
Annual error: |error| ≤ 10.0 %
Hourly CV(RMSE): ≤ 30.0 %
Hourly NMBE: |NMBE| ≤ 10.0 %
Conditioned floor area: |change| ≤ 1.0 %
Loads not judged: a load whose reference is below 1.0 kWh/m² a year
Owner choice: yes: the default of the report writer; a later drop may carry another

A zoning meets the threshold when every judged load passes all three limits and the area limit holds; a missing metric of a judged load fails. The recommended zonings of a cell are those meeting it with the fewest modelled zones; ties are all listed.

Deduplication

Two test sites of one cell are duplicates when they have the same zone-geometry signature (each zone's multiplier and its floor polygons, vertices rounded to a millimetre, names left out) and the same annual ideal sensible heating and cooling within a relative tolerance of 10⁻⁹. The site kept is the first by the writer's order, heat-transfer setting closest to the reference's (IWHT and FHT both on; IWHT on, FHT off; IWHT off, FHT on; both off), then simplifier by name, join pieces (unset, off, on), perimeter depth as a number, stacker by name; a reference is never dropped (D-221). In the main matrix 3,179 of 5,016 generated test sites are duplicates of a listed zoning; each cell names them under its table.

Robust across orientations

The rule (ADR 0050 section 6, an atlas rule over the writer's file). For each prototype, climate zone and vintage, the robust zoning is the coarsest (fewest modelled zones) that meets the threshold at every orientation of the cell pair, by the writer's own verdict on each zoning at each orientation, with at most the reference's zone count, as the writer bounds a recommendation (D-220); a duplicate never meets, as the writer lists it. Ties at the fewest zones go to a zoning both orientations recommend, else to the first in the study's zoning order (simplifier, join pieces, perimeter depth, stacker, IWHT, FHT); the tied zonings are named. The errors reported are the worse of the orientations' values for each metric, so the row is a bound, not a mean; the cost ratios are the medians over the timed sites of that zoning in the pair. Where both orientations recommend the same zoning it is the robust one, since any coarser zoning meeting the threshold at both would have been recommended at each. Where no zoning meets the threshold at both orientations the pair says so and names each orientation's own recommendation, which stays the closest reading for that orientation; a prototype, climate zone and vintage with one orientation in the file has no pair and no robust pick.

Orientations: 0° and 45°
Cell pairs: 132: robust in 106, none meeting at both orientations in 26
Recommended at both orientations: 97 of 106 robust pairs; in the other 9 the robust zoning is one orientation's recommendation or finer than both
Ties at the fewest zones: 51 pairs, named in the row

What the data allows. The file lists every kept zoning of every cell with its metrics and the writer's verdict, so the rule compares the zonings themselves at both orientations, not only the two recommendations. It takes the writer's meets_threshold as the verdict and recomputes nothing: a zoning that is a duplicate at one orientation carries the writer's never recommended verdict there and does not qualify, which keeps the robust pick a kept test site at both orientations. The errors in a robust row are the worse of the two orientations' stored values for each metric (the larger magnitude, with its sign), so a row bounds both orientations and is not an average; the orientation each value came from is in the prototype's JSON (robust.pairs[].worst).

Limits. Robust means robust over the two orientations the study ran, 0° and 45°, not over every orientation; a later drop with more orientations is judged over all of them by the same rule. The robust zoning is at least as fine as each orientation's own recommendation, never coarser. The cost ratios of a robust row are the median over the timed sites of that zoning in the pair, which in this drop is one site at most, since only 0° cells were timed. The rule is computed at site build by scripts/lod_site.py and written into each prototype's JSON as a robust block labelled as such; the report writer's files are unchanged.

Cost protocol

Energy results do not depend on how many EnergyPlus processes run at once, so the stages ran 16 at a time and their times are not results (the files carry no processor-time ratio of those runs, BEMGen D-266). Cost was measured in a separate serial timing pass (R5, BEMGen 35e8a132): serial timing pass: --parallel 1 --timing-repeats 3 --timing-test-cases 12 (the first 12 test cases by the SHA-256 of their IDs); Intel Core i7-14700F (8 performance and 12 efficiency cores, 28 logical processors), otherwise idle during the timing pass. Per site the runner records the processor time (user plus kernel), the wall clock and the peak committed private memory; a site's cost is the median over its repeats, its cost ratio that median over its reference's median; a prototype's cost is the median of its timed test sites' ratios without the duplicate twins, and a zoning's and the overall cost are the medians of the prototype medians, each timed prototype one vote (D-202, D-222).

Timed: 244 sites of 12 cells: Full-service restaurant 5A 90.1-2019 0°, Strip mall 3C 90.1-2019 0°, Stand-alone retail 5A 90.1-2019 0°, Supermarket 3C 90.1-2019 0°, Supermarket 5A 90.1-2019 0°, Warehouse 5A 90.1-2019 0°, Primary school 3C 90.1-2019 0°, Small hotel 3C 90.1-2019 0°, Small hotel 5A 90.1-2019 0°, Hospital 3C 90.1-2019 0°, Hospital 5A 90.1-2019 0°, Outpatient 3C 90.1-2019 0°
Not timed: Large hotel, Medium office (not among the 12 timed test cases, so their costs are not measured)
Repeats: 3
Timed IDFs: 244 of 244 byte-identical to R3's
Session: 2026-11-02, the timing pass from about 04:20 to 05:11 local time: 732 of 732 runs, no interruption, no resume; the machine otherwise idle: total processor load 5.6 % on average and at most 9.5 % in 30-s samples, one sample with a foreign process (a system service, 2.8 %)
Drift over the repeats: each repeat's median over the sites' medians is 1.000, and the third repeat of the 40 sites of at least 4 s a median 0.996 of the mean of the first two (at most 1.024): the late-repeat slowdown of the earlier timed drop did not recur (D-265)
Gate: processor-time CV per site below 5.0 %: 242 of 244 sites pass; median 1.7 %, P90 3.3 %, largest 7.0 %; 2 misses are short runs (0.89 s, where the 15.6 ms count of processor time is about 2 % of the value) and none of the 40 sites of at least 4 s misses; not met
Treatment: D-202: costs are medians of site ratios; no per-site precision below the stated CV distribution is claimed
Repeat check: not run for this drop: a check between sessions needs a second session on an otherwise idle machine (D-265); the repeat check of drop 3 timed other models and is not carried over

Within the session the measurement is stable (median processor-time CV 1.7 %); no check between sessions was run for this drop, so how far a per-site ratio moves between sessions is not measured on these models. The costs shown are therefore medians: the per-cell ratios on the prototype pages are each site's median over its repeats, and the medians by zoning and by prototype on the overview are the figures to quote. No per-site precision below the stated CV distribution is claimed (D-202).

Reading the recommendations

A recommendation with IWHT or FHT off is a tie with a zoning with both on. In the heat-transfer sub-study, where a zoning has no surface between zones of the kind a switch pairs (one zone, or one storey for FHT), its on and off settings write the same model; deduplication keeps the twin whose setting is closest to the reference's (both on first, D-221), so the off twin is listed as a duplicate. An off zoning left in a recommendation is a model of its own that meets the threshold with as few zones as a zoning with both on (9 of 18 sub-study cells with a recommendation). The duplicate list under each cell names the twins, and the report's table by setting shows the effect itself: floors and ceilings between zones matter little, interior walls more.
A recommendation has at most the reference's zone count. Coarseness is the number of EnergyPlus zones before multipliers, so a Perimeter core split of a prototype with few rooms can have more zones than the reference; such a zoning is not recommended, since the study asks how coarse a zoning can be (D-220). A cell where only finer zonings meet the threshold has no recommendation and is flagged finer than the reference: Full-service restaurant 4A 90.1-2007 0°, Full-service restaurant 4A 90.1-2007 45°, Full-service restaurant 4A 90.1-2019 0°, Full-service restaurant 4A 90.1-2019 45° (4 cells). The robust rule takes the same bound.

Limits and caveats

Ideal loads. Heating and cooling are ideal-loads sensible demands; HVAC systems, service hot water and latent loads are outside the comparison. The errors say how a zoning changes the building's thermal demand, not its energy use.
Deterministic inputs. The prototypes' own geometry, glazing, atlas programs with ruleset schedules, the atlas 90.1 unified construction set and one TMY3 weather file per climate zone; no stochastic schedules, no holidays, no infiltration axis.
Coverage. The vintages are 90.1-2007 and 90.1-2019 (the atlas releases have no 90.1-2004 folders); climate zone 7 is registered as 7A, the atlas label of International Falls. IECC sets are out of scope.
Inputs (release 2.4.0). The medium office takes the programs of the detailed folder medium-office-detailed, one per room space type, which release 2.4.0 leaves unchanged, so release 2.4.0's PNNL office lighting power (ADR 0060) does not reach this study; taking the whole-building program instead would change the study's design (BEMGen D-262, for the owner). The 90.1-2007 small hotel takes one building-scope construction set, the residential row of the source's pair; the prototype gives the guest rooms the residential and the other spaces the nonresidential rows (BEMGen D-261, for the owner). Daylighting control and the unoccupied lighting cap are off in every model, although 13 of the study's 90.1-2019 program keys carry PNNL's cap rule in release 2.4.0 (BEMGen D-261, D-262).
Zonings that are not generated. Perimeter core and the single-zone stackers do not support multiplied rooms, and a perimeter depth needs a wing at least twice as deep, so the generator refuses those zonings: 1,056 in the main matrix (a wing is narrower than twice the perimeter depth: Outpatient; the zoning does not support the prototype's multiplied rooms: Hospital, Large hotel), each listed under its cell as not generated, not as a failed run.
Main matrix runs. 0 of 5,016 generated test sites did not succeed.
Loads not judged and the threshold's sensitivity. A load whose reference is below 1.0 kWh/m² is not judged: sensible heating in 108 of 264 cells; sensible cooling in 4 of 264 cells. With the three error limits scaled (area limit and judging floor unchanged), the cells with a recommendation are 182 at ×0.5, 204 at ×0.8, 213 at ×1, 215 at ×1.2, 223 at ×2 (D-223).
Cost precision and scope. The timing gate was not met (2 of 244 sites miss it), so costs are medians (the cost protocol above); the prototypes Large hotel and Medium office were not timed; costs are measured in 90.1-2019, climate zones 3C, 5A, at 0° only, in one session on one machine, and no repeat check was run for this drop.
Threshold. The threshold is the report writer's default and an owner choice, recorded in every file; the R6 report kept it. A later drop with another threshold carries the new one in every file.
Not a population statement. The cells are the prototypes at their own dimensions; nothing here weights them by stock frequency.

Provenance

Study: How Much Thermal Zoning Detail Is Enough? (lod-study-v1)
Run plan: docs/plans/2026-10-29-lod-study-run-plan.md in BEMGen, decisions D-193, D-200, D-201, D-202, D-203, D-211, D-219, D-220, D-221, D-222, D-223, D-224, D-246, D-249, D-252, D-261, D-262, D-263, D-264, D-265, D-266
Report: research/lod-study/report/2026-11-02-lod-study-report.md (R6: the results and findings of every stage)
Methodology: research/lod-study/report/2026-11-02-lod-study-methodology.md (R6: the method, every number with its source)
BEMGen: https://github.com/EnvironmentalSystemsLab/BEMGen: report writer scripts/lod-study/report.py and schema 1.2.0 at 99c0b0adcd44; the R3 and R4 runs with the IDF writer at 35e8a132; the R5 timing pass at 35e8a132; EnergyPlus 26.2.0 (4bd7a1f26f)
Atlas inputs: release 2.4.0 at 7c543f5df904, filled programs, unified constructions
Schema: schemas/lod-recommendations-v1.schema.json, a copy of BEMGen's scripts/lod-study/schemas/lod-recommendations-v1.schema.json; the main matrix is version 1.2.0, the sub-study 1.2.0
Previous drop: drop 4 (not timed, BEMGen dbbd87257ef1, this repository at 6d202b89): The drop this site carried before: the same R3 and R4 results, not timed. Its files differ from this drop's only by the cost and timing members R5 adds and the cpu_time_ratio member of the parallel runs that D-266 removes; in this repository at commit 6d202b89.
Archived drop: drop 3 (atlas release 2.0.0 inputs, BEMGen 8d2e3bda3fb9), archived in BEMGen at research/lod-study/archive/2026-10-29-atlas-2-0-0/
Repository files: data/lod-recommendations/v1/; drop 5
Licence: Original data CC BY 4.0 (ADR 0048, ADR 0050); attribution: Archetype Atlas, Energy-Atlas, LOD recommendations drop 5

File	Stage	Suite	Run	Cells	SHA-256
lod-recommendations.json	R3 main matrix with the R5 serial costs	lod-main-20261101-225850	run_20261101-230342	264	`65b237176843`
lod-recommendations-ht.json	R4 heat-transfer sub-study	lod-ht-20261102-001857	run_20261102-002039	22	`50dd9bb981e6`

Stage	Suite	Run	BEMGen	Gate
R3 main matrix	lod-main-20261101-225850	run_20261101-230342	`1.10.0+35e8a132f785a5329f4fd6015fce82a655e634ab`	complete: 5280 of 5280 simulated sites succeeded; 1056 refused by the generator by documented limits (listed with their reason)
R4 heat-transfer sub-study (IWHT/FHT)	lod-ht-20261102-001857	run_20261102-002039	`1.10.0+35e8a132f785a5329f4fd6015fce82a655e634ab`	complete: 1760 of 1760 simulated sites succeeded; 352 refused by the generator by documented limits (listed with their reason)
R5 serial timing pass	lod-timing-20261102-034653	run_20261102-034718	`1.10.0+35e8a132f785a5329f4fd6015fce82a655e634ab`	CPU-time CV < 5 %: 242 of 244 sites, not met; reported as medians (D-202)
R6 study report, methodology and consolidated results	–	–	`1.10.0+99c0b0adcd4449bedbe0f7a1b86b0db75e2d3ce5`	written by scripts/lod-study/study_tables.py --write from report.py of D-266 and compared with the archived drop; tables verified by --check (D-264, D-265, D-266)

A later consolidated drop replaces the files in place with the same layout; the pages are regenerated from them at the next site build (ADR 0050).