$0.00 is indistinguishable from a free runcost telemetryPer-run cost is read back from the run's own raw/costs/ rows. When there is nothing to report the system returns records=0 rather than a synthetic $0.00 — because a fake zero and a genuinely free run render identically, and the fake one silently understates spend on the exact axis the product is sold on.
Lesson: a metric with no producer must render "not measured", never a default value. tests/test_analytics_value_vocabulary.py now asserts that a KPI bound to an always-zero column can't masquerade as working.
SymbiYosys in BMC mode prints no property count, so a harness holding zero assertions reported byte-for-byte like a proof over eighteen. The fix: a property census counted in Yosys after prep, so a measured zero demotes a PASS to inconclusive — while an unreadable census stays None, because "unmeasured" and "zero" are different claims.
Lesson: a vacuous pass is the most dangerous kind of pass. Measure the thing that makes the verdict meaningful, not just the verdict.
A model that misreads the spec the same way twice writes RTL and a known-answer vector that agree — six of six checks pass, and the sign-off report describes a wrong design accurately. This is why AES-128 (verified against FIPS-197 vectors, an oracle outside the model) is materially stronger evidence than any design graded by vectors the model invented.
Lesson: "verified" without an independent oracle means "self-consistent". Every run should stamp which kind of evidence it had.
The 3-D builder wrote scene.json with triangles: 987992 beside an empty .glb. The manifest's numbers came from pre-write intent, never re-read from the written asset. Fix: derive the manifest's counts from the asset, and fail the build when the asset has no meshes rather than emitting a manifest for it.
Lesson: a manifest that describes what you meant to write, not what you wrote, is a lie waiting to be believed.
The primary account sat three weeks behind. Redeploying it surfaced two defects a green 3,993-test suite could not see — including GateSim failing every design, calc4 included, with 64 of 64 checks passing. Both appeared only on a real run, on the day of a demo.
Lesson: the gap between merge and deploy is where integration defects hide. Deploy often, and treat a long-stale environment as unproven.
Base image build is ~18 min (OpenROAD compiles from source); the app layer is ~90s. The base tag is a hash of Dockerfile.base + requirements.txt + the ORFS pin, so an app build against a stale base fails on a missing tag rather than producing a silently-wrong image. Three separate defects were each found only after paying a full build-and-run cycle, and each was decidable locally in seconds — which is why make preflight exists.
Lesson: make the expensive path fail loudly on a mismatch, and build a cheap local gauntlet that catches what a build-and-run would.
A synthetic /tmp workspace fails an upstream guard (workspace-prefix / missing PDK) before reaching the code under test, and proves nothing. Pull the recorded error JSON from S3 and reproduce with the actual toolchain (yosys / OpenSTA / OpenROAD / iverilog) locally, then commit.
Lesson: prefer local reproduction over deploy-and-observe. A fix "validated" against a repro that never exercised the real code path is not validated.
Every S3-deployed Lambda uses a versioned build-src/<version>/… key, so an updated zip is a real CloudFormation change. A constant key means CFN sees no change and the function keeps its original code — an update that appears to succeed and does nothing. Separately, a template referencing a key with no object behind it fails the whole stack with NoSuchKey — and a changeset preview cannot catch it, because CFN validates a key's syntax, never that the object exists.
Lesson: version your deployment keys, and package-then-upload in the deploy script — never trust a Default constant.
After inserting a statement into a role's Policies, an off-by-one edit can strip an adjacent statement's Resource. The error points at the role; the culprit is the sibling. It took three failed deploys to learn that the real fix was parsing the rendered template and confirming every Sid still has a Resource.
Lesson: don't re-edit the statement you just added when a neighbour is the one that broke. Parse the rendered template.
A retired Bedrock model id survived every deploy because no script passed the parameter explicitly, and a Default only applies at CREATE. The spec-intake stage silently fell back to the deterministic record — a feature that looked inert while appearing to work.
Lesson: pass every parameter explicitly on update, and monitor whether a model is invocable, not just whether it was once listed.
E3006 "does not exist in us-east-1" for AWS::BedrockAgentCore::* and XRay TransactionSearchConfig are stale-spec false positives — those resources deploy fine and live stacks reach UPDATE_COMPLETE. Judge by whether new E-levels appear on the changed templates, not by the raw E-total.
Lesson: know your linter's blind spots, or you'll block yourself on phantoms.
The zero-latches grader failed every design in a Tier-2 A/B — diagnosed as three real harness bugs, not model quality: yosys -q suppresses all output (empty log → cell-count regex matched nothing → false "zero cells"); the cell-count regex matched only the legacy format while modern Yosys prints 719 cells; and the latch counter matched bare DLATCH anywhere, false-positiving on the pass name PROC_DLATCH.
Lesson: reproduce the exact tool invocation before trusting a verdict. A harness bug wearing a model-failure costume wastes days.
A P&R parser was tested against a hand-written log in the format the parser expected — so it read UNAVAILABLE on every real run while the numbers sat in the log in a different shape. The rule now: capture real, byte-for-byte multi-pass tool logs into tests/fixtures/ before writing a parser, verify each by SHA-256, and parse that.
Lesson: a parser that agrees with an authored approximation of its input has proven nothing.
Real-EDA integration tests skip silently when a binary is absent — and a skip reads as green in the pytest summary. A tool you didn't install is a test you're not running. make eda-tools installs them; make preflight reports CLEAR TO BUILD only when the tiers actually pass.
Lesson: a green suite with silent skips is a false sense of coverage. Make the skip visible.
git merge lands on whatever branch is checked outmerge targetingA bare git -C <repo> merge <feature> merges into the currently-checked-out branch — usually not the intended deploy branch. The safe pattern that worked repeatedly: do the merge in a worktree checked out on the target branch, or use git branch -f for a fast-forward. (This very site was built in an isolated worktree for exactly this reason.)
Lesson: never assume which branch a merge targets. Check out the target, or work in its worktree.
git switch moves the whole checkout — including for a session mid-edit in the same directory — and carries uncommitted changes onto the new branch. For genuinely concurrent work, each session takes its own git worktree outside the repo. Before merging many commits, check the tree for uncommitted changes you didn't make.
Lesson: the working tree, not the branch, is the unit of isolation.
A dirty tree yields a sha-<commit>-dirty image tag that cannot be reproduced from any commit — and has already shipped code later discarded. Likewise, never switch branches while a deploy runs: the scripts read templates off the working tree stack by stack, and a mid-run checkout once shipped a template three commits stale.
Lesson: the artifact's provenance is only as good as the tree it was built from.
Step Functions owns the DAG; every pass/fail is a pure-Python Lambda parsing tool output; no LLM output ever feeds a Choice. This eliminates the entire class of "LLM mis-routes the pipeline" failures and makes a run auditable by a sign-off reviewer. It is a deliberate constraint, not a missing feature.
Lesson: put the model in the generative path and keep it out of the decision path.
13 agents with 13 scoped roles instead of a few multi-purpose ones: the verification agent can't submit Slurm jobs; the design agent can't tear down infrastructure. No role holds InvokeAgentRuntime, so agents cannot call each other — everything they exchange goes through state-machine JSON. A vulnerability in one agent can only reach that agent's narrow scope.
Lesson: the cost of many small roles is real; the cost of one compromised broad role is worse.
When the formal timeout needed to adapt, the answer was explicitly not to let the model propose one — a model-proposed retry budget is model output shaping control flow. The bounded fix loop, its budgets, and any timeout ladder stay operator-decided and deterministic.
Lesson: convenience is not worth letting a model touch the control plane.