One cluster, every plane it owns
Seven tabs plus node and VM drill-ins: health, Azure connectivity, patches, storage, and networks, every value with its source and time.
A full-stack TypeScript & SQL Server platform for Azure Local (formerly Azure Stack HCI). It pulls operational truth from planes that don't agree — health reports, live WMI faults, forwarded Windows events, Azure alerts and APIs — across five walled-off environments, into one operator surface that never pretends a number came from somewhere it didn't.
Monitoring Azure Local isn't one feed — it's many, and they have different truth models. Some data comes from scheduled health reports. Some is a live WMI fault that fires and clears in seconds. Some is a firehose of forwarded Windows events. Some is an Azure API you poll, with its own clock and throttle. And it all lives in five Active Directory domains, where non-production can't reach production services directly.
The lazy version flattens everything into one table and fakes the joins. This one doesn't. Report-backed data is tied to a real report; live data stays live and reportless. That single decision is what makes the dashboard trustworthy instead of just busy.
Data shaping lives at the database and repository boundary — never as ad-hoc reads from a component.
Report-backed sources carry a real report ID. Live sources are environment-scoped and intentionally reportless.
PowerShell remoting for what no API reports: uptime, reboots, OS display version, update history, OS Config. Microsoft's own health check is launched, not awaited.
Finds orphaned differencing-disk files across cluster storage paths; SQL derives age, staleness, and per-server totals.
Resource Graph, ARM, and metrics reads, each run an immutable manifest. Since September: CPU, memory, Arc connectivity, pending updates, OS build, hardware.
Health Service WMI faults as they're created, modified, and resolved. Lower environments relay through production; stale faults are classified, never closed by silence.
Eight per-domain query lists feed a watcher with catch-up and live phases, hash-first SHA-256 dedupe, and bulk writes. Retention: 36 days, by time.
Polled with tenant-bound OAuth2 into stage tables and MERGE. An alert resolves on the provider's word, never because the API went quiet.
Every automation job, heartbeat, event stream, and artifact is a queryable, environment-scoped record — automation is evidence too.
The platform watches itself: materialization steps, errors, slow queries, and Effect request traces land in SQL, so a stale view points at the failing step.
The whole design hinges on never letting a live signal pretend it came from a report.
-- A canonical key defines alert identity; an issue key defines an occurrence. MERGE dbo.FleetIssue AS tgt USING staged_alerts AS src ON tgt.IssueKey = src.IssueKey WHEN MATCHED AND src.MonitorCondition = 'Resolved' THEN UPDATE SET tgt.Status = 'Resolved', tgt.ResolvedAtUtc = src.SeenUtc WHEN NOT MATCHED THEN INSERT (CanonicalAlertKey, IssueKey, EnvironmentCode, Status) VALUES (src.CanonicalAlertKey, src.IssueKey, src.EnvironmentCode, 'Active'); -- FleetEvent then records every fired / changed / resolved transition.
Windows, SQL Server, Azure, and five AD domains all had opinions. Here's how each argument ended.
Windows Event Forwarding has limits it doesn't print on the box. In April the single subscription failed with error 50 (ERROR_NOT_SUPPORTED): too many queries. Two lists of 37 and 52 queries came first; eight lists, one per domain, is what held. Then the record IDs. WEF keeps each source's own EventRecordID, so IDs from hundreds of hosts interleave and can't be a watermark. The watcher hashes first — SHA-256 in UTF-16LE, byte-matching SQL's HASHBYTES — and renders XML only for events SQL hasn't seen.
Five domains, and none of them trust each other. Non-production can't reach production, and Get-Cluster has no -Credential parameter, so the health collector is a credentialed wrapper around a perform script. Lower-environment WMI faults are relayed through production with per-file replay cursors. Azure credentials are bound to a tenant per environment group, and a missing one fails instead of falling back to production. Environment codes are stamped at ingestion; an unknown one throws.
A query that looked indexed wasn't. The live timeline's header counts took about 105 seconds — a full index scan over roughly 17 million rows, even for a 60-minute window. Assigning results to variables (SELECT @Count = …) switches off OPTION (RECOMPILE) parameter embedding, so catch-all @X IS NULL OR col = @X filters never became seeks; a temp-table INSERT … SELECT fixed it. Query Store also caught one-second routes reading 150,000+ pages per probe. Now a probe reads 5, and nothing counts as indexed without logical-read and plan evidence.
Changing the engine in flight. In one September week, most node data moved from PowerShell remoting to Resource Graph, ARM, and metrics — the first big step of an API-first switch that's still under way — with operator decisions and alert history intact. The same three months brought Bun to Node, SQL Server 2025, and most of Effect 4: about 2,600 commits, one developer plus AI coding agents, kept safe by 35 CI gates, drift checks against the live database, and deploys that wait for a drain receipt.
No screenshots — the views, in words.
Seven tabs plus node and VM drill-ins: health, Azure connectivity, patches, storage, and networks, every value with its source and time.
WMI faults beside Azure Monitor and Resource Health alerts, badged by source. Each opens to exact evidence and a Correlated Context card that checks eight live sources, naming only the ones it actually checked.
Forwarded events from every environment, with severity and node filters, saved noise-reduction views, and preset chips with real counts. Copy the full message and XML.
Seven stations, from overnight cleared alerts to cluster balance and Event Intelligence. The copyable summary unlocks only once every station is reviewed.