LHDS runbook stubs¶
Stubs. Each lists only steps that exist in code. None has been executed against a live stamp. Gaps are called out inline.
Dispatch deploy-health.yml¶
- Actions,
deploy-health, Run workflow. Chooseboundary(Commercial or GCC-High) andrun_mode=whatif-onlyfirst. - For
full: leavecontrol_plane_audiencesempty. It is resolved for you: the input if typed, else repo variableLOOM_HEALTH_CP_APP_CLIENT_ID, else the Entra apploom-health-cpby display name (must be exactly one), giving<appId>,api://<appId>. A full run that cannot resolve it fails; it never deploys without the control plane. After the deploy, the run setsLOOM_HEALTH_CP_URL,LOOM_HEALTH_CP_AUDIENCEandLOOM_HEALTH_DEFAULT_STAMP_IDonloom-consoleand reads them back. A laterdeploy-fiab-commercialthat does not carry them resets them; re-run this lane to re-bind (#5346). Leaveallow_existing_hubandoverride_estate_pausefalse unless intended. - The post-deploy bootstrap (
bootstrap-health-cp-entra.mjs) declares GraphUser.ReadBasic.All+GroupMember.Read.All(application) onloom-health-cp-asand creates its service principal, so a Global Administrator can grant consent. It does not grant it. - A stood-down or what-if run is titled DRY RUN and is not a deployed SHA.
- IL5 and DoD are not offered. IL5 is
health-params/il5-health.bicepparam, supported-in-code only.
Tenant create / destroy¶
- Create:
POST /api/health/tenantsregisters the tenant; jobtenant.create(empty input) deploystenant.bicepand reads back resource ids. - Destroy: job
tenant.destroyrequiresconfirmTenantIdequal to the job subject. Crypto-shred is evidenced by key state. - Not wired:
main.tsdoes not register these handlers or an ARM adapter, so submitting these kinds viaPOST /api/health/jobsis not expected to work today. Key Vault data-plane disable/delete also needs a Crypto Officer grant (W1-2 execution state).
PDP bundle publish¶
- The control plane serves
GET /api/health/pdp/stamps/<id>/bundle.tar.gz(ETag, 304) from the directory inpdpBundleDir; the stamp's OPA fetches it. - Not implemented: a pipeline that builds, signs and places the bundle, and the per-stamp
data.lhds.config/stampdocument. A served bundle denies everything until then. #5140 covers the endpoint's missing per-stamp scoping. - Routing map:
POST /api/health/routing-map/publish(jobrouting.publish).
Pause / resume¶
- Estate pause is the Loom Estate Power lane; the
health-stampguard inlib/estate/pause-guards.tsrefuses a stamp with running jobs or active feeds. - PostgreSQL Flexible Server auto-restarts after 7 days stopped; the control plane has a job and scheduler that restarts a
pausedstamp's server once per 6-day window. - Not active: bicep emits no
loom-health-stamptag, nothing passes stamp activity to the orchestrator, andmain.tsdoes not run the restart scheduler.
Edge probe (tenant 403 and quota 429 + Retry-After)¶
Until this has been run against a deployed gateway, the cross-tenant 403 and the per-tenant 429 are modelled only (routing-edge.test.ts). The quota is enforced in APIM (rate-limit-by-key keyed on the URL tenant); a WAF rate-limit rule cannot key on a token claim, so the WAF rule is only a per-client-address volumetric backstop.
LHDS_EDGE_BASE_URL=https://<gateway> LHDS_TENANT_A=<slug> LHDS_TENANT_B=<slug> \
LHDS_QUOTA_RPM=<tenant A effective requests per minute> \
LHDS_TOKEN_A=<token with tenant claim A> LHDS_TOKEN_B=<token with tenant claim B> \
node scripts/measure/health/edge-probe.mjs
Exit 0 = every check passed, 1 = the gateway answered wrongly (defect), 2 = unmeasured (bad config, no response, or the positive control failed; this is not a claim the edge is broken). Run it per cloud: Commercial and GCC-High each need their own receipt. The WAF per-address rule is untested in Azure Government; if the rule type is rejected there, set healthConfig.wafClientRateLimitPerMinute to 0 in the boundary's health-params/*-health.bicepparam (health.bicep passes it to the stamp ingress; 0 drops the rule). The default is 6000 on every boundary, so Gov is not given a lesser product by default; the 0 is an escape taken only on a recorded rejection. Status: the 429 + Retry-After and the Gov WAF rule are compiled and structurally tested (scripts/ci/__tests__/lhds-edge-policy.test.mjs) but never exercised.