Tested against land people built on.

Every test we ran

In July 2026 we tested the Score against real build outcomes in Austin: 76 completed new-construction permits and 369 never-built parcels. The v1 rows are a live run over the same federal sources every report uses; the v2 and v3 rows re-score the same parcels.

We also ran the test designed to catch ourselves: remodel permits, which say nothing about whether land is buildable. How close they come to new builds shows how much of the raw score is location rather than parcel-level skill.

AUC

Area under the ROC curve: how often a built lot outscores a never-built one. 0.5 is a coin flip; 1.0 is perfect separation.

95% CI

The range the AUC plausibly sits in, given 76 built lots. Overlapping intervals mean this test cannot show one version ranks better than another.

σ

The spread of the published number across the cohort. A wider spread separates parcels within one market.

Win rate

In the paired test: for each built lot, the share of its never-built neighbours it outscores, averaged across built lots.

The current engine, v3, adds lot-scale terrain and geometry factors, grades zoning standards instead of gating on them, and never applies optimistic defaults. Its rows re-score stored per-factor sub-scores, not a fresh live run, and no weight was re-tuned. The neighbour-matched and placebo tests have not been re-run for v3.

SourceAustin building permits (Socrata)·Travis CAD parcels·City of Austin Land Use Inventory·all public

The paired and placebo tests, in detail

Pairing each built lot with never-built neighbours removes location. Under v1 the win rate does not fall toward 0.5 as the radius tightens, so the skill is parcel-level. v2 scored lower than v1 at every radius, and at 0.75 km its interval barely clears chance. Additions and remodels, scored against the same controls, run the predicted way.

Win rates are averaged per built lot, with bootstrap intervals over built lots. 45 cases and 91 pairs at 0.75 km is thin, hence the wide interval.

SourceAustin building permits (Socrata)·Travis CAD parcels·City of Austin Land Use Inventory·all public

Every factor, and what it reads

Each factor is sourced independently, because the datasets carry different uncertainty: a FEMA flood zone is a direct map lookup, while utility proximity is an inference from mapped infrastructure.

Seismic and radon are read, scored and shown as regional context, but not averaged into the parcel number. A county radon zone or a regional seismic grid cannot separate neighbouring lots; on their own the two rank at AUC 0.524, a coin flip.

• A factor with no answer drops out. The remaining factors re-normalize, so a property is never penalized for a data gap.
• The report says so. It shows which factors were present and which were omitted.
• Estimates are labelled. A regional estimate, such as a valuation, is marked Estimated.

Weights are the v3 national defaults. State overrides: flood weighs 25% in Florida, utilities 24% in Texas, and in California terrain 25% and zoning 22%.

SourceUSGS 3DEP·FEMA NFHL·USDA SSURGO·USFS·OpenStreetMap·Regrid·All data sources

Sources and freshness

Federal sources, Regrid and RentCast are queried directly whenever a fresh report runs; Census demographics are cached quarterly. The last column is how current each provider keeps its own data.

SourceAll data sources

Known limits

• One metro, one build type. Every validation number is Austin, TX, and one question: new single-family construction on vacant land.
• The range is compressed within a market. Under v1 the spread was about 1–2 points, because regional hazards barely vary inside one city. Do not over-read a 2-point gap between two parcels. Later versions added lot-scale terrain and geometry to widen it: on the same cohort σ rose from 5.58 (v2) to 6.62 (v3).
• Some questions are untested here. Zoning and utilities were absent from every parcel in the Austin cohort, and seismic was constant (one value in 445 of 445), so Austin cannot test the zoning or utilities weights, nor the seismic question for places like California where it varies within a metro.
• Re-scored, not re-run. The v2 and v3 rows re-score stored sub-scores; the neighbour-matched and placebo tests have not been re-run for v3. v3 is engineering-calibrated, not outcome-fitted.
• Zoning code lag. County GIS databases can lag actual zoning changes by 30–90 days. After a recent rezoning, the report may show the prior designation until the county updates.
• Comparable sales in thin markets. Market data depends on MLS and public-record completeness, which varies by jurisdiction. Markets with fewer than 5 comparable sales within 1 mile have wider estimation error.

SourceMethodology v3·what it does not claim

Questions