Energy cost of compiled rule-based and language-model analyzer variants for instant health assessment
Authors
- Prozorov A.A., Director of Technology, RTLAB; Architect, Sbertech; ap@rtlab.ru;
Abstract
Objective: This paper compares three implementation variants of an instant health assessment analyzer by electricity use per unit of work, normalized to joules per patient-hour (J/patient-hour): a large Mixture-of-Experts (MoE) language-model analyzer, a small analyzer on a highly tuned language model, and a compiled C++ rule-based analyzer. The unit of work is one hour of real-time analysis for one patient over a continuous 12-lead electrocardiogram (ECG) stream sampled at 500 Hz per lead plus named electrolyte terms that update no more often than once per 30 minutes.
Method: The paper is a methodological modeling study with a transparent energy-accounting formula. It defines a functional comparability gate, a shared signal-preparation pipeline charged equally to every variant, a decomposition of the compiled-analyzer runtime into shared-preparation power and rule-evaluation increment, a mandatory deterministic validator stage charged to the language-model variants, and a recurring revalidation stage that captures model and prompt drift. External sources ground inference-energy ranges, lifecycle training energy, benchmark and power-measurement discipline, accelerator power envelopes, and data-center overhead [1-13].
Results: The ranking is stable across the modeled range: for the required traceable, evidence-producing, auditable workload, the compiled C++ rule-based variant has the lowest runtime energy by one to two orders of magnitude. In the gate-complete base scenario, runtime energy is about 3,600 J/patient-hour for the compiled analyzer, 59,400 J/patient-hour for the small language-model variant, and 545,400 J/patient-hour for the large MoE variant. After amortizing preparation, adaptation or authoring, release-readiness, recurring revalidation, and audit over 1,000,000 patient-hours, the base estimates become about 7,200 J/patient-hour, 117,000 J/patient-hour, and 768,600 J/patient-hour respectively, or roughly 1.0x, 16x, and 107x relative to the compiled analyzer. These are modeling estimates, not measured product claims.
Conclusion: For this specific bedside workload the hot path is narrow, continuous, evidence-bound, and deterministic, and that shape favors compiled rules and a deterministic Compute core. Language-model variants are better placed adjacent to the hot path, for rule-authoring support, offline clustering, reviewer-facing explanation of observations, and candidate rules for governed knowledge-graph refinement.
Keywords: energy efficiency; compiled rule-based analyzer; deterministic Compute core; language-model inference energy; mixture-of-experts; lifecycle energy amortization; instant health assessment; 12-lead ECG; auditable evidence; joules per patient-hour
Abbreviations And Terminology
Analyzer: A software module or service that consumes a bounded analysis workload and emits bounded findings with evidence. In this paper the term covers all three compared variants, regardless of internal mechanism.
Compiled C++ rule-based analyzer: The project-native variant in which authorized semantics are compiled into a released package executed by a deterministic Compute core over bounded windows, producing runtime evidence and durable audit. Rule-based means behavior is driven by explicit rules or compiled executable knowledge rather than by an opaque model judgement.
Large MoE language-model analyzer: A variant that uses a large sparse Mixture-of-Experts model for window-level interpretation over compact feature summaries and current named terms, requiring a full serving stack.
Small analyzer on a highly tuned language model: A variant that uses a compact domain-tuned model or constrained classifier/generator over structured features, capable of running on lower-power accelerators or optimized CPU paths.
Compute core: The deterministic execution component that runs one normalized request against a validated in-memory catalog and returns deterministic response and evidence.
Validator layer: The deterministic component that maps any analyzer output into authorized term identity, bounded cardio flag states, and published evidence. The compiled analyzer is itself this layer; a language-model variant must add it as a separate mandatory pass to satisfy the comparability gate.
Patient-hour: The normalization unit, one hour of continuous real-time analysis for one patient.
Window and stride: The rolling analysis window length and the advance between successive windows. The base case uses a 10 second window with 1 second stride, giving 3,600 analyzer invocations per patient-hour.
Runtime energy: The electricity used to serve one patient-hour once the variant is ready, including shared signal preparation, variant-specific computation, the validator layer, and evidence publication.
Lifecycle-amortized energy: Runtime energy plus preparation, training or authoring, compilation and release, readiness, recurring revalidation, and audit energy, divided by the deployment scale in patient-hours.
Functional comparability gate: The set of behavioral obligations a variant must meet for its energy number to be comparable: same input scope, same bounded output class, traceability, evidence publication, auditability, and clinical non-claim.
Shared signal-preparation pipeline: The common layer of protocol intake, lead validation and normalization, signal-quality checks, feature extraction, temporal alignment, and evidence packaging charged equally to every variant.
PUE (Power Usage Effectiveness): The data-center multiplier that converts information-technology energy into facility energy including cooling, uninterruptible power, and networking.
1. Problem Definition
The analyzer task is the instant health assessment scenario: real-time interpretation of mixed named terms and ECG oscillograms. The concrete input is a continuous 12-lead ECG stream with each lead sampled at 500 Hz; named electrolyte terms, specifically sodium, calcium, and magnesium, with the internal ECG narrative also discussing potassium, magnesium, and calcium as rhythm and morphology context; term-value updates that occur no more often than once per 30 minutes; and output as bounded cardio flags, exclusions, unsupported states, unresolved states, runtime evidence, audit references, and diagnostic-contour handoff.
The companion ECG interpretation paper defines named terms as governed clinical, signal, laboratory, or interpretation terms with stable identity and current values, defines ECG oscillograms as continuous real-time 12-lead signal material, and defines cardio flags as bounded interpretive outputs rather than autonomous diagnoses. The companion stream-analyzer paper defines rolling windows, event-time ordering, watermarks, late-data policy, and stream evidence projection. The companion stateless-analyzer paper defines the request-local service host, decoder, named-feature preparation, Compute core invocation, and publication surfaces for bounded 12-lead ECG windows. These manuscripts are the internal grounding for the task model and are listed in the Authoring Basis.
2. Functional Comparability Gate
Energy numbers are meaningful only if the variants perform comparable work. A variant is comparable only if it passes all of the following gates.
| Gate | Required behavior |
|---|---|
| Same input scope | The analyzer consumes the same patient-hour workload: 12-lead ECG stream, current named electrolyte terms, and declared windowing policy. |
| Same output class | The analyzer emits bounded findings: cardio flag, exclusion, absent, unsupported, or unresolved. |
| Traceability | Each output carries named term identity, runtime term reference where applicable, term-value context, rule or model release identity, and signal evidence reference. |
| Evidence publication | The analyzer produces result, audit, debug or explanation, failure, metric, and runtime evidence surfaces appropriate to its architecture. |
| Auditability | Durable audit is not reconstructed from debug traces; evidence can be replayed or reviewed against the accepted release basis. |
| Clinical non-claim | The result is not treated as autonomous diagnosis, regulatory validation, or independent clinical adjudication. |
If a language-model variant produces plausible prose but cannot attach stable named term references, release identity, evidence references, and auditable decision states, it fails the comparability gate. Its energy may still be interesting, but it is not comparable to the governed analyzer workload. The gate has a direct energy consequence developed in Section 6: a language-model variant must run a deterministic validator pass to convert free output into governed states and evidence, and that pass is not optional.
3. Source Reliability
The report uses four evidence levels. Level A is internal project narratives and specifications that define the task, authority boundaries, evidence obligations, and the C++ runtime architecture. Level B is peer-reviewed or standards-like external sources that ground lifecycle energy and inference-energy ranges. Level C is industry benchmarks and vendor specifications that ground benchmark methodology, throughput context, and hardware power envelopes. Level D is recent preprints and modeling assumptions used only for sensitivity framing and always labeled as such.
Important external sources include task-specific inference-energy measurement showing that text generation is much more energy-intensive than text classification [2]; lifecycle training-energy estimation for a large public model [3]; standardized inference benchmarking and a concrete power-measurement discipline using external power analyzers [4][5][6]; accelerator power envelopes for context only [7][8]; data-center overhead and PUE context [9][10]; a vision framing of “intelligence per joule” [1]; and recent preprints on energy-to-token production, context-length routing, and inference-energy diagnosis [11][12][13], which are not treated as settled measured facts.
4. Input Workload Model
4.1 Raw Stream Size
The ECG stream size is:
12 leads x 500 samples/s = 6,000 samples/s
6,000 samples/s x 3,600 s/hour = 21,600,000 samples/hour
21,600,000 samples/hour x 2 bytes/sample = 43,200,000 bytes/hour
The base raw voltage stream is therefore about 43.2 MB/patient-hour before metadata, quality annotations, derived features, compression, transport framing, and evidence references.
4.2 Windowing
The base windowing mode uses a 10 second window length, 1 second stride, and 3,600 invocations per patient-hour. Sensitivity cases vary the stride.
| Mode | Window | Stride | Invocations per patient-hour |
|---|---|---|---|
| Low-frequency review | 10 s | 10 s | 360 |
| Base real-time review | 10 s | 1 s | 3,600 |
| High-frequency review | 10 s | 0.5 s | 7,200 |
Electrolyte named terms are sparse compared with ECG. If values update at most once per 30 minutes, each electrolyte has at most two updates per patient-hour. Their energy impact is dominated by evidence alignment and audit linkage, not by numerical input volume.
5. Shared Signal-preparation Pipeline
A fair comparison does not ask a language model to read raw ECG as text in the main scenario. All three variants share a signal-preparation layer: protocol intake and source or session identity; lead validation, ordering, unit normalization, and sample-rate policy; signal-quality checks and artifact flags; feature extraction for rhythm, intervals, morphology, and channel summaries; temporal alignment of ECG windows with current named electrolyte terms; and evidence packaging with references, hashes, and release context.
This shared layer prevents a false comparison in which the compiled analyzer receives structured features while a language model receives an arbitrarily expensive raw signal encoding. It also has an important accounting consequence used throughout this paper: because the shared preparation power is charged equally to every variant, it is a floor under all three numbers and a relatively large fraction of the compiled-analyzer total, but a negligible fraction of the language-model totals. The compiled analyzer does not look cheap because it skips preparation; it looks cheap because its variant-specific increment above shared preparation is small. A separate anti-pattern analysis for raw ECG tokenization appears in Section 11.4.
6. Energy Accounting Formula
The report uses a runtime term plus an amortized fixed term:
E_total_per_patient_hour =
E_runtime_per_patient_hour
+ (E_prepare
+ E_train_or_author
+ E_compile_release
+ E_readiness
+ E_revalidation_total
+ E_audit) / N_patient_hours
E_runtime_per_patient_hour =
E_shared_prep
+ E_variant_compute
+ E_validator
+ E_evidence_publication
Where E_shared_prep is the shared signal-preparation energy charged to every variant; E_variant_compute is rule evaluation for the compiled analyzer or model inference for the language-model variants; E_validator is the deterministic pass that maps output into authorized terms, bounded states, and published evidence, which is intrinsic to the compiled analyzer and an added mandatory pass for language-model variants; E_evidence_publication is result, audit, and runtime-evidence writing; E_prepare includes dataset curation, test fixtures, replay corpora, and feature-preparation material; E_train_or_author is model training or fine-tuning for language models or rule formalization and scenario authoring for the compiled analyzer; E_compile_release covers compilation, packaging, release evidence, and deployment readiness; E_readiness covers regression suites, replay tests, acceptance harnesses, and benchmark runs; E_revalidation_total covers recurring re-acceptance after model, prompt, feature-schema, or rule changes over the deployment lifetime; E_audit covers compute, storage, indexing, replay, and report-generation energy for the audit contour; and N_patient_hours is the commercial operation scale over which fixed energy is amortized.
Human reviewer time is not converted into joules. Only electricity used by compute, storage, network, accelerator, and infrastructure is counted.
7. Coefficients Used
7.1 Literature and Hardware Anchors
| Quantity | Value used | Source status |
|---|---|---|
| Text classification energy | about 0.002 kWh / 1,000 inferences, or 7.2 J/inference |
Reported by task-specific inference-energy measurement [2]. |
| Text generation energy | about 0.047 kWh / 1,000 inferences, or 169.2 J/inference |
Reported by task-specific inference-energy measurement [2]. |
| Large-model training reference | 433,196 kWh dynamic electricity for one public 176B-parameter training run |
Reported lifecycle analysis [3]; idle and embodied contributions are additional. |
| H100 SXM TDP | up to 700 W configurable |
Vendor product specification [7]. |
| L4 TDP | 72 W |
Vendor product specification [8]. |
| Industry-average PUE | 1.56 base, 1.2 efficient hyperscale sensitivity |
Data-center survey and agency context [9][10]; applied only to cloud or data-center variants. |
7.2 Modeling Assumptions
The following coefficients are modeling assumptions for this report, chosen to be conservative and transparent rather than falsely precise. They are Level D.
| Quantity | Low | Base | High | Interpretation |
|---|---|---|---|---|
| Shared signal-preparation power (all variants) | 0.25 W (900 J/hour) |
0.5 W (1,800 J/hour) |
1.5 W (5,400 J/hour) |
Continuous filtering, beat detection, morphology, and feature extraction at 6,000 samples/s on an embedded or server core. |
| Compiled-analyzer rule-evaluation increment | 0.07 W (250 J/hour) |
0.5 W (1,800 J/hour) |
3.5 W (12,600 J/hour) |
Bounded rule and exclusion evaluation plus evidence and audit writing above shared preparation. |
| Small language-model inference | 2 J/window |
15 J/window |
80 J/window |
Optimized constrained classification through compact structured generation to less efficient serving. |
| Large MoE inference | 20 J/window |
150 J/window |
1,000 J/window |
Efficient MoE serving through generation-like serving to long-context or low-utilization serving. |
| Validator pass (language-model variants only) | 0.5 J/window |
1 J/window |
2 J/window |
Deterministic mapping of model output into authorized terms, bounded states, and published evidence. |
The compiled-analyzer runtime is the sum of shared preparation and the rule-evaluation increment: at base, 1,800 + 1,800 = 3,600 J/hour, equal to 1 W average, of which only half is the analyzer-specific increment. The large MoE base coefficient is near the text-generation energy anchor [2]; the choice is deliberate and is discussed as a proxy in Section 9. The small language-model base coefficient is about twice the text-classification anchor [2], leaving room for feature packaging and structured output. The validator base of 1 J/window is set deliberately at or above the entire compiled-analyzer rule-evaluation increment of 0.5 J/window, so that the comparison does not understate the language-model obligation; see Section 8.1.
7.3 Lifecycle Fixed-energy Assumptions
| Variant | Low | Base | High | Included work |
|---|---|---|---|---|
| Large MoE adoption | 5,000 kWh |
30,000 kWh |
150,000 kWh |
Dataset preparation, domain adaptation, validation, release-readiness, evidence and audit workflows. |
| Large MoE recurring revalidation | 8,000 kWh |
32,000 kWh |
120,000 kWh |
Re-acceptance after model, prompt, or feature-schema changes over the deployment lifetime (Section 8.2). |
| Small language-model adoption | 500 kWh |
8,000 kWh |
40,000 kWh |
Domain data preparation, fine-tuning, quantization and evaluation, release-readiness, audit workflows. |
| Small language-model recurring revalidation | 2,000 kWh |
8,000 kWh |
30,000 kWh |
Re-acceptance after tuning, prompt, or feature-schema changes over the deployment lifetime. |
| Compiled C++ rule-based | 100 kWh |
1,000 kWh |
5,000 kWh |
Rule authoring support, compilation, regression, deterministic replay, release evidence, audit workflows; deterministic re-acceptance on rule change is bounded and already included here. |
| Large model full-pretraining stress add-on | 433,196 kWh |
433,196 kWh |
2,000,000 kWh |
Separate ownership scenario, not the main adoption case. The low and base value is dynamic training electricity only [3]; broader infrastructure and idle energy would increase it. |
The main language-model scenario is use of a pretrained model plus domain adaptation. Full pretraining is reported separately because it is a different business and engineering boundary.
8. Two Hardening Terms
This section makes explicit two energy obligations that earlier informal comparisons tend to omit, both of which widen rather than narrow the compiled-analyzer advantage.
8.1 The Mandatory Validator Pass
The comparability gate (Section 2) requires governed output: stable term identity, bounded cardio flag states, and published evidence. The compiled analyzer produces these directly; they are its native output. A language-model variant produces tokens, and those tokens must be parsed, constrained, mapped to authorized terms, checked against bounded states, and turned into evidence by a deterministic validator. That validator is itself a small rule engine. Charging it at a base of 1 J/window is conservative, because it is set at or above the entire compiled-analyzer rule-evaluation increment of 0.5 J/window. The implication is sharp: even before counting any inference, a gate-passing language-model variant pays more per window for post-hoc validation than the compiled analyzer pays for its whole analytic increment. The validator cannot be removed without failing the gate, so it belongs in runtime energy for the language-model variants.
8.2 Recurring Revalidation
Deterministic compiled rules change only when authorized semantics change, and the change is a deterministic recompile with bounded regression and deterministic replay, already counted in the compiled-analyzer fixed energy. Language-model behavior is sensitive to tuning, prompt, context, and feature-schema changes, and can silently degrade when any of these drift from training assumptions. Each such change triggers a re-acceptance cycle: regression suites, replay corpora, and benchmark runs must be re-executed to re-establish the release basis. Modeling eight revalidation cycles over a deployment that produces 1,000,000 patient-hours, at half of the base adoption energy per cycle, adds 32,000 kWh for the large MoE variant and 8,000 kWh for the small variant. The compiled analyzer carries no equivalent recurring penalty because its re-acceptance is deterministic and bounded.
9. The MoE Base Coefficient Is a Generation Proxy
The large MoE base of 150 J/window is intentionally set near the measured text-generation energy anchor [2] rather than derived from a measured MoE serving trace. This is a proxy choice and is stated as such. The reason is twofold. First, public measured MoE serving energy for this exact workload does not exist, and the most defensible public anchor for window-level interpretive output is the generation task. Second, the routing topology preprint [12] gives an analytical MoE tokens-per-watt example but explicitly excludes MoE dispatch overhead, so it cannot serve as a measured base. Using the generation anchor is therefore conservative in form and honest in provenance: it neither claims a measured MoE number nor leans on a preprint that omits dispatch cost. Sensitivity to this choice is bounded by the low and high coefficients in Section 7.2.
10. Results
10.1 Runtime Energy, Gate-complete Base
Runtime energy excludes fixed preparation. The gate-complete model adds shared preparation to every variant and the validator pass to the language-model variants.
| Variant | Composition (base) | Runtime base | Relative to compiled |
|---|---|---|---|
| Compiled C++ rule-based | 1,800 prep + 1,800 rule eval |
3,600 J (1.0 Wh) |
1.0x |
| Small language-model | 1,800 prep + 54,000 inference + 3,600 validator |
59,400 J (16.5 Wh) |
16.5x |
| Large MoE | 1,800 prep + 540,000 inference + 3,600 validator |
545,400 J (151.5 Wh) |
151.5x |
For comparison with informal inference-only framings, the inference-only ratios are about 15x and 150x; the gate-complete ratios are slightly higher because the validator pass is mandatory. The low and high runtime ranges follow the coefficients in Section 7.2.
| Variant | Low | Base | High |
|---|---|---|---|
| Compiled C++ rule-based | 1,150 J (0.32 Wh) |
3,600 J (1.0 Wh) |
18,000 J (5.0 Wh) |
| Small language-model | 9,900 J (2.75 Wh) |
59,400 J (16.5 Wh) |
300,600 J (83.5 Wh) |
| Large MoE | 75,600 J (21.0 Wh) |
545,400 J (151.5 Wh) |
3,610,800 J (1,003.0 Wh) |
10.2 Lifecycle-amortized Energy at 1,000,000 Patient-hours
Amortized fixed energy per patient-hour is fixed energy in kilowatt-hours multiplied by 3.6 J at this scale. Language-model fixed energy now includes recurring revalidation (Section 8.2).
| Variant | Runtime base | Fixed (adoption + revalidation) | Amortized fixed | Lifecycle base | Relative |
|---|---|---|---|---|---|
| Compiled C++ rule-based | 3,600 J |
1,000 kWh |
3,600 J |
7,200 J (2.0 Wh) |
1.0x |
| Small language-model | 59,400 J |
16,000 kWh |
57,600 J |
117,000 J (32.5 Wh) |
16.3x |
| Large MoE | 545,400 J |
62,000 kWh |
223,200 J |
768,600 J (213.5 Wh) |
106.8x |
The hardening terms widen the advantage relative to an inference-only treatment: the compiled-analyzer base is unchanged at 7,200 J/patient-hour, while the small and large language-model bases rise to about 16x and 107x because the validator pass and recurring revalidation are now counted.
10.3 Scale Sensitivity
| Deployment scale | Compiled C++ rule-based | Small language-model | Large MoE | Large MoE plus pretraining stress |
|---|---|---|---|---|
10,000 patient-hours |
363,600 J (101.0 Wh) |
5,819,400 J (1,616.5 Wh) |
22,865,400 J (6,351.5 Wh) |
178,000,000 J (49,444 Wh) |
1,000,000 patient-hours |
7,200 J (2.0 Wh) |
117,000 J (32.5 Wh) |
768,600 J (213.5 Wh) |
2,328,600 J (646.8 Wh) |
100,000,000 patient-hours |
3,636 J (1.01 Wh) |
59,976 J (16.66 Wh) |
547,632 J (152.12 Wh) |
561,232 J (155.9 Wh) |
Small deployments are dominated by preparation, validation, and revalidation energy. Very large deployments are dominated by runtime energy. The ranking never reverses under the base assumptions.
10.4 Why 1,000,000 Patient-hours Is the Headline Scale
The headline scale is not arbitrary. One million patient-hours is about 114 patient-years of continuous analysis. A mid-size intensive care deployment of 20 monitored beds at roughly 70 percent occupancy produces on the order of 120,000 patient-hours per year, so one million patient-hours corresponds to roughly eight years of single-site operation or about one year across a small multi-site network of eight comparable units. This is the scale at which a governed clinical deployment is a going concern rather than a pilot, which is why it is used for the headline and why both a pilot scale and a very large scale are shown in Section 10.3 for contrast.
11. Sensitivity Analysis
11.1 Window Stride
Language-model runtime scales approximately with the number of invocations when each window causes a separate prompt and validator pass. Compiled runtime may scale sublinearly because feature extraction is streaming and rule evaluation reuses state, but the high case is kept broad enough to cover more frequent windows.
| Window mode | Invocations/hour | Small base (15 J + 1 J validator) |
Large MoE base (150 J + 1 J validator) |
|---|---|---|---|
| 10 s stride | 360 | 5,760 J (1.6 Wh) |
54,360 J (15.1 Wh) |
| 1 s stride | 3,600 | 57,600 J (16.0 Wh) |
543,600 J (151.0 Wh) |
| 0.5 s stride | 7,200 | 115,200 J (32.0 Wh) |
1,087,200 J (302.0 Wh) |
This is one of the strongest arguments against per-window large language-model reasoning: the workload is continuous, so small per-window inefficiencies and the mandatory validator pass are multiplied 3,600 times per hour in the base case.
11.2 Context Length and Utilization
The routing-topology preprint [12] argues that tokens per watt can vary dramatically with context length and that routing by context length can be more powerful than hardware upgrades, reporting analytically that tokens per watt can halve when context doubles. For this analyzer the consequences are direct: compact structured prompts are mandatory for any language-model variant; verbose per-window explanations should be off the hot path; long patient-history context should be retrieved only at audit or review time unless clinically required in the real-time gate; and a model that needs raw ECG history in context is energetically mismatched to continuous bedside analysis.
11.3 PUE and Deployment Location
Data-center variants must multiply information-technology energy by facility overhead, with servers around 60 percent of modern data-center electricity on average and cooling ranging from efficient hyperscale levels to much higher enterprise levels [9][10]. This report uses PUE = 1.56 as the base data-center multiplier, PUE = 1.2 as an efficient hyperscale sensitivity, and no data-center multiplier for bedside or embedded edge devices, where measured plug power should be used instead. This is itself an advantage axis for the compiled analyzer, developed in Section 12.
11.4 Raw ECG Tokenization Anti-pattern
The fair baseline gives all variants structured features. If raw ECG is tokenized into text, the language-model cases become noncompetitive. One patient-hour has 21.6M raw samples; even a compact textual representation such as lead=value,time would create millions to tens of millions of tokens per patient-hour before any model reasoning, worsening context length, key-value cache pressure, audit trace size, and protected-health-information exposure. Raw ECG belongs in signal processing and evidence storage; the language model, if used at all, should consume compact feature summaries and safe references.
12. Additional Structural Advantages of the Compiled Analyzer
Beyond the per-window arithmetic, several structural properties favor the compiled analyzer and are not captured by inference coefficients alone.
Resident-memory and idle power. A compiled rule bundle is kilobytes to megabytes resident, while a served language model holds gigabytes to terabytes of weights resident, with associated static memory power, high-bandwidth-memory refresh, and idle accelerator draw even between windows. For a continuous but low-rate workload, idle resident power can dominate language-model energy, while the compiled analyzer has almost no resident footprint.
No-accelerator edge deployment. The compiled analyzer runs on a bedside or embedded CPU with no accelerator and no data-center facility overhead, so no PUE multiplier and no network round-trip apply. A cloud-served language model adds network-transport energy for every window and a facility multiplier on every joule. For a continuous stream this network and overhead term is paid 3,600 times per hour.
Cooling and thermal scaling. Cooling energy scales with device power. A compiled analyzer at roughly 1 W average imposes negligible thermal load, while accelerators in the 72 to 700 W envelope [7][8] carry a cooling overhead that the PUE multiplier only partially captures at the bedside.
Determinism removes sampling multipliers. Language-model deployments that need higher reliability sometimes use sampling strategies such as self-consistency or ensembling that run inference several times per decision, multiplying energy. The compiled analyzer is deterministic: one evaluation yields one auditable result, with no sampling multiplier.
Bounded latency means bounded sustained power. The compiled analyzer meets real-time latency at a low, steady clock. A language-model variant must hit a token budget within the window deadline, which tends to require higher sustained power and leaves a heavier energy tail on longer outputs.
13. Variant-by-variant Critical Analysis
13.1 Large MoE Language-model
Strengths: it can integrate heterogeneous feature summaries and narrative context; it can produce human-readable explanations for audit or review; and MoE active-parameter routing can reduce decode compute relative to a dense model of similar total parameter count. Weaknesses: high runtime energy for continuous window-level invocation; context length and key-value-cache behavior can dominate energy; routing, expert placement, and memory movement reduce the apparent benefit of sparse active parameters; deterministic traceability and bounded output require the added validator layer; and full pretraining ownership is energetically large and must be reported separately. Best-fit role: not the primary real-time analyzer, but a possible offline reviewer assistant, evidence summarizer, diagnostic triage explainer, or authoring copilot behind a deterministic validator.
13.2 Small Analyzer on a Highly Tuned Language Model
Strengths: it can be domain-specialized; it may be efficient on compact inputs and constrained outputs; it can support triage or explanation where rules are incomplete; and it can run on lower-power accelerators than frontier serving stacks. Weaknesses: it is still more expensive than direct rule execution for high-frequency windows; it must prove that its output maps to authorized terms and cardio flag states through the validator layer; it may require frequent revalidation after tuning or prompt changes; and it can silently degrade if prompt, feature schema, or context differs from training assumptions. Best-fit role: constrained secondary analyzer, candidate generator for governed refinement, reviewer aid for ambiguous or unresolved windows, and fallback only when deterministic rules explicitly mark an unresolved state.
13.3 Compiled C++ Rule-based Analyzer
Strengths, stated at the same level of quantified detail as the critiques above: the work shape matches the task shape, so the analyzer-specific increment is about 0.5 W (1,800 J/patient-hour), half of its already-low 1 W total and the rest shared with every variant; runtime energy is one to two orders of magnitude below the language-model variants under the gate-complete model; deterministic replay and audit are native rather than reconstructed, so audit and revalidation energy are bounded; fail-closed behavior is compatible with governed clinical software; resident footprint, idle power, accelerator, network, and cooling terms are near zero at the bedside; and compile-time and runtime boundaries align with the internal project narratives. Weaknesses: rules and exclusions must be authored, reviewed, and maintained; clinical coverage grows only through governed semantic refinement; brittle or incomplete rule sets can emit unsupported or unresolved outcomes more often than a flexible model; and the energy advantage does not by itself prove clinical adequacy. Best-fit role: primary real-time analyzer for bounded cardio flags and evidence production, authoritative when paired with governed semantics, release evidence, a deterministic Compute core, and diagnostic-contour closure.
14. Interpretation
The report does not claim that compiled code is always better than a model. It says that for this specific workload the hot path is narrow, continuous, evidence-bound, and deterministic, and that shape favors compiled rules and a deterministic Compute core. The intelligence-per-joule framing [1] does not reverse the conclusion: the comparability gate fixes the unit of useful work as a governed, evidence-bound finding, and a language-model variant cannot deliver that unit without the deterministic validator pass, so capability per joule for this task is equal-or-better on the compiled path.
The strongest language-model use cases are adjacent to the hot path: authoring assistance for rule prose and test scenarios; offline clustering of unresolved evidence; reviewer-facing explanation over already published evidence; candidate generation for governed graph updates; and audit-report summarization where deterministic facts remain authoritative. The weakest language-model use case is per-window autonomous interpretation of the live stream without deterministic evidence grounding; such a system may consume more energy and still fail the comparability gate.
15. Practical Measurement Plan
The next project step should be an empirical measurement harness. It should define a replay corpus of synthetic and de-identified 12-lead ECG windows, named electrolyte term updates, expected cardio flag and exclusion surfaces, and unsupported and unresolved cases. It should implement a measurement wrapper that records wall-clock runtime, central-processing-unit package power where available, accelerator power where available, process memory, storage writes, network bytes, and evidence output size. It should measure four workloads: shared signal preparation only; the compiled rule-based runtime path; a small language model over compact features; and a large MoE model over compact features. It should run the three window modes of Section 4.2. It should report joules per window, joules per patient-hour, watt-hours per patient-hour, evidence bytes per patient-hour, audit-write energy where measurable, the validator-pass energy for language-model variants, and a pass or fail result for the functional comparability gate.
For data-center tests, the harness should follow a standardized power-measurement discipline [6]: measure system power with an external power analyzer or validated telemetry, record idle and active periods separately, and report utilization. For edge tests it should measure plug power or board-level rails rather than relying only on software estimates.
16. Limitations And Non-claims
The compiled-analyzer runtime numbers are modeling assumptions, not measured project results, and are intentionally marked as such; the shared-preparation and rule-evaluation decomposition is likewise a modeling assumption. The language-model runtime numbers combine literature anchors and modeling assumptions, and actual serving energy can vary substantially with hardware, batching, utilization, quantization, prompt length, and output length. The validator-pass and recurring-revalidation coefficients are modeling assumptions introduced to reflect gate obligations the literature anchors do not measure; they are deliberately conservative but remain Level D. External inference-energy studies often measure different tasks such as text classification, question answering, summarization, or image generation, and are useful anchors rather than direct measurements of ECG analysis. MoE serving energy is especially uncertain because active parameter count is not the only driver; routing, dispatch, memory traffic, and utilization matter, which is why the base is treated as a generation proxy (Section 9). The report measures electricity, not carbon footprint, water footprint, cost, embodied hardware emissions, or human labor. Clinical quality is outside the energy model: a low-energy analyzer that fails the functional gate or clinical validation is not acceptable.
17. Conclusion
For the specified instant health assessment task, the energy-optimal primary runtime is the compiled C++ rule-based analyzer backed by authorized semantics, a compiled package, deterministic Compute core execution, runtime evidence, and auditable diagnostic closure. The gate-complete base lifecycle-amortized estimate is about 7,200 J/patient-hour (2 Wh/patient-hour).
The small analyzer on a highly tuned language model is a plausible auxiliary technology, especially for triage, candidate generation, or constrained secondary interpretation, but its gate-complete base lifecycle-amortized estimate is about 117,000 J/patient-hour (32.5 Wh/patient-hour), roughly 16x the compiled base once the mandatory validator pass and recurring revalidation are counted. The large MoE model is the least energy-matched hot-path technology; even with compact inputs and without charging full pretraining to the main scenario, its gate-complete base lifecycle-amortized estimate is about 768,600 J/patient-hour (213.5 Wh/patient-hour), roughly 107x the compiled base.
The practical recommendation is therefore hybrid but asymmetric: keep the real-time analyzer deterministic and compiled, and use language models outside the hard real-time loop for authoring support, evidence summarization, reviewer assistance, and governed-refinement candidates.
Authoring Basis
The article draws on the internal documentation of the HealthOS project: the specifications, schemas, and registries that define authorized semantics and its compiled form, the runtime and tooling that produce execution evidence, and the companion manuscripts of the series, bound together by a shared terminology canon. In particular it builds on the companion ECG interpretation manuscript for the named term and cardio flag model, the companion stream-analyzer manuscript for windowing and stream evidence, the companion stateless rule-based analyzer manuscript for the request-local service host and Compute core invocation, and the companion compute-core and safety-kernel manuscript for deterministic execution and the separation of durable audit from debug trace. These are working sources that ground the engineering decisions and keep terminology consistent across the series. They remain internal authoring basis rather than public references and are not cited as repository links in the text.
Conflict of Interest
The manuscript describes the implementation of a specific aspect of the HealthOS platform, namely: the comparative runtime and lifecycle energy of compiled rule-based and language-model analyzer variants for instant health assessment.
Funding
The work was funded by RTLAB.
References
- Chen D. et al. AI+HW 2035: Shaping the Next Decade. arXiv:2603.05225, 2026. https://arxiv.org/abs/2603.05225
- Luccioni A.S., Jernite Y., Strubell E. Power Hungry Processing: Watts Driving the Cost of AI Deployment? ACM FAccT 2024. https://facctconference.org/static/papers24/facct24-6.pdf
- Luccioni A.S., Viguier S., Ligozat A.L. Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model. JMLR 24(253), 2023. https://www.jmlr.org/papers/v24/23-0069.html
- MLCommons. Llama 2 70B: An MLPerf Inference Benchmark for Large Language Models, 2024. https://mlcommons.org/2024/03/mlperf-llama2-70b/
- MLCommons. MLPerf Inference Results v5.0. https://docs.mlcommons.org/inference_results_v5.0/
- MLCommons. Power Measurement - MLPerf Inference Documentation. https://docs.mlcommons.org/inference/power/
- NVIDIA. H100 Tensor Core GPU. https://www.nvidia.com/en-us/data-center/h100/
- NVIDIA. L4 Tensor Core GPU. https://www.nvidia.com/en-us/data-center/l4/
- IEA. Energy demand from AI, 2025. https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai
- Uptime Institute. Global Data Center Survey 2024. https://intelligence.uptimeinstitute.com/resource/uptime-institute-global-data-center-survey-2024
- Liu X. et al. Position: LLM Inference Should Be Evaluated as Energy-to-Token Production. arXiv:2605.11733, 2026. Preprint. https://arxiv.org/abs/2605.11733
- Chen H. et al. The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency. arXiv:2603.17280, 2026. Preprint. https://arxiv.org/abs/2603.17280
- Chung J.W. et al. Where Do the Joules Go? Diagnosing Inference Energy Consumption. arXiv:2601.22076, 2026. Preprint. https://arxiv.org/abs/2601.22076