● LIVEEnterprise monitoring · current engagement

The dashboard that watches a 152-cluster hybrid cloud.

A full-stack TypeScript & SQL Server platform for Azure Local (formerly Azure Stack HCI). It pulls operational truth from planes that don't agree — health reports, live WMI faults, forwarded Windows events, Azure alerts and APIs — across five walled-off environments, into one operator surface that never pretends a number came from somewhere it didn't.

TypeScriptReact 19Node 24SQL Server 2025PowerShell 7.6Prisma 7AzureTailwind v4
TanStack StartTanStack QueryEffectEffect SchemaWindows Event ForwardingMotion
5
environments (prod, UAT, QA, dev, sandbox), one AD domain each
152
Azure Local clusters under watch
1,100+
VMs in the latest production report
132
SQL views, plus 235 stored procedures
1,265
test files, plus 35 CI quality gates
azure-local-ops
// The problem

The data was scattered across planes that don't agree.

Monitoring Azure Local isn't one feed — it's many, and they have different truth models. Some data comes from scheduled health reports. Some is a live WMI fault that fires and clears in seconds. Some is a firehose of forwarded Windows events. Some is an Azure API you poll, with its own clock and throttle. And it all lives in five Active Directory domains, where non-production can't reach production services directly.

The lazy version flattens everything into one table and fakes the joins. This one doesn't. Report-backed data is tied to a real report; live data stays live and reportless. That single decision is what makes the dashboard trustworthy instead of just busy.

// Architecture

PowerShell gathers evidence. SQL Server does the thinking.

Data shaping lives at the database and repository boundary — never as ad-hoc reads from a component.

01 COLLECT
PowerShell 7.6 collectors & listeners, Windows Event Forwarding, and Azure APIs gather raw JSON, WMI faults, event XML, and API evidence inside each domain.
02 LAND
SQL Server 2025 lands raw input first, then publishes current-state tables — one row per VM and per cluster — in one transaction with a completion marker.
03 SHAPE
132 views, 235 stored procedures, 43 functions & 15 triggers, checked in and drift-checked against what's deployed, compute dashboard contracts and the alert lifecycle in the database, not the UI.
04 CONTRACT
Prisma 7 + Effect repositories back 193 API route files, with schemas generated from Prisma and SQL procedure metadata, never hand-duplicated. Zod survives in 42 files, mid-migration.
05 RENDER
TanStack Start + React 19 + TanStack Query on Node 24, behind IIS and Windows auth, drive a desktop-first UI. Every value shows its source and clock; missing data reads "unavailable", never zero.
// How the data is captured

Eight sources, each with its own honesty.

Report-backed sources carry a real report ID. Live sources are environment-scoped and intentionally reportless.

HCI Fleet Health

REPORT-BACKED

PowerShell remoting for what no API reports: uptime, reboots, OS display version, update history, OS Config. Microsoft's own health check is launched, not awaited.

runbook-scheduled · ~29 min per prod sweep

AVHDX orphan scan

REPORT-BACKED

Finds orphaned differencing-disk files across cluster storage paths; SQL derives age, staleness, and per-server totals.

daily · per environment

Azure Local APIs

LIVE

Resource Graph, ARM, and metrics reads, each run an immutable manifest. Since September: CPU, memory, Arc connectivity, pending updates, OS build, hardware.

hourly · inventory, patches & security every 3 h

HCI Health Fault Listener

LIVE

Health Service WMI faults as they're created, modified, and resolved. Lower environments relay through production; stale faults are classified, never closed by silence.

continuous node-local listener

Forwarded Windows Events

LIVE

Eight per-domain query lists feed a watcher with catch-up and live phases, hash-first SHA-256 dedupe, and bulk writes. Retention: 36 days, by time.

continuous · flush every 10s

Azure Monitor & Resource Health

LIVE

Polled with tenant-bound OAuth2 into stage tables and MERGE. An alert resolves on the provider's word, never because the API went quiet.

every 5 minutes · staggered per environment

Forge jobs & runners

LIVE

Every automation job, heartbeat, event stream, and artifact is a queryable, environment-scoped record — automation is evidence too.

continuous · per job

Pipeline & app telemetry

LIVE

The platform watches itself: materialization steps, errors, slow queries, and Effect request traces land in SQL, so a stale view points at the failing step.

continuous / runtime
// The interesting part

Evidence-first: keep the two truth models apart.

The whole design hinges on never letting a live signal pretend it came from a report.

Report-backed

  • Has a real ReportDocument + ReportId
  • You can ask: which report produced this row?
  • What raw collector JSON backs the evidence?
  • HCI Fleet Health & AVHDX live here, published atomically

Live / reportless

  • Environment-scoped, never report-scoped
  • No synthetic ReportId is ever invented
  • WMI faults, Windows events, Azure alerts, API evidence
  • Prevents "this fault came from the latest report" lies
alert lifecycle — computed in SQL, not the UI (simplified)
-- A canonical key defines alert identity; an issue key defines an occurrence.
MERGE dbo.FleetIssue AS tgt
USING staged_alerts AS src
  ON tgt.IssueKey = src.IssueKey
WHEN MATCHED AND src.MonitorCondition = 'Resolved'
  THEN UPDATE SET tgt.Status = 'Resolved', tgt.ResolvedAtUtc = src.SeenUtc
WHEN NOT MATCHED
  THEN INSERT (CanonicalAlertKey, IssueKey, EnvironmentCode, Status)
       VALUES (src.CanonicalAlertKey, src.IssueKey, src.EnvironmentCode, 'Active');
-- FleetEvent then records every fired / changed / resolved transition.
// What was hard

Four problems that fought back.

Windows, SQL Server, Azure, and five AD domains all had opinions. Here's how each argument ended.

Windows Event Forwarding has limits it doesn't print on the box. In April the single subscription failed with error 50 (ERROR_NOT_SUPPORTED): too many queries. Two lists of 37 and 52 queries came first; eight lists, one per domain, is what held. Then the record IDs. WEF keeps each source's own EventRecordID, so IDs from hundreds of hosts interleave and can't be a watermark. The watcher hashes first — SHA-256 in UTF-16LE, byte-matching SQL's HASHBYTES — and renders XML only for events SQL hasn't seen.

Five domains, and none of them trust each other. Non-production can't reach production, and Get-Cluster has no -Credential parameter, so the health collector is a credentialed wrapper around a perform script. Lower-environment WMI faults are relayed through production with per-file replay cursors. Azure credentials are bound to a tenant per environment group, and a missing one fails instead of falling back to production. Environment codes are stamped at ingestion; an unknown one throws.

A query that looked indexed wasn't. The live timeline's header counts took about 105 seconds — a full index scan over roughly 17 million rows, even for a 60-minute window. Assigning results to variables (SELECT @Count = …) switches off OPTION (RECOMPILE) parameter embedding, so catch-all @X IS NULL OR col = @X filters never became seeks; a temp-table INSERT … SELECT fixed it. Query Store also caught one-second routes reading 150,000+ pages per probe. Now a probe reads 5, and nothing counts as indexed without logical-read and plan evidence.

Changing the engine in flight. In one September week, most node data moved from PowerShell remoting to Resource Graph, ARM, and metrics — the first big step of an API-first switch that's still under way — with operator decisions and alert history intact. The same three months brought Bun to Node, SQL Server 2025, and most of Effect 4: about 2,600 commits, one developer plus AI coding agents, kept safe by 35 CI gates, drift checks against the live database, and deploys that wait for a drain receipt.

// Inside the platform

Event Intelligence, The Forge, Ask ALDHI — built in.

  • Event Intelligence — turns the forwarded-event stream into ranked findings with researched meanings and replayable runs. See the full breakdown →
  • The Forge — governed automation inside the dashboard: allowlisted, hash-pinned PowerShell commands on a SQL-backed queue, one runner per environment, artifacts, and a full audit trail. No arbitrary shell from a browser. It also inherited the redeployment console's post-deploy checks. See the full breakdown →
  • Ask ALDHI — the platform's own AI assistant, on TanStack AI + OpenRouter: 38 typed tools over the same repositories the UI reads. No model-written SQL, no secrets in context, capped reads, and no email without a preview and an explicit send. See the full breakdown →
// Operator surfaces

Four views where the evidence becomes operational.

No screenshots — the views, in words.

cluster detail

One cluster, every plane it owns

Seven tabs plus node and VM drill-ins: health, Azure connectivity, patches, storage, and networks, every value with its source and time.

live alerts

One queue across five environments

WMI faults beside Azure Monitor and Resource Health alerts, badged by source. Each opens to exact evidence and a Correlated Context card that checks eight live sources, naming only the ones it actually checked.

windows events

The raw stream, made navigable

Forwarded events from every environment, with severity and node filters, saved noise-reduction views, and preset chips with real counts. Copy the full message and XML.

morning review

A daily sweep with a sign-off

Seven stations, from overnight cleared alerts to cluster balance and Event Intelligence. The copyable summary unlocks only once every station is reviewed.