ACE Framework Theory

Causal inference at Pearl's second rung. Pearl's causal hierarchy [2] distinguishes three levels of reasoning: observation (Rung 1, “what is”), intervention (Rung 2, “what if I act”), and counterfactual (Rung 3, “what if it had been otherwise”). AI-assisted Modeling, Simulation, and Analysis (MS&A), mission planning, and battle management live squarely at Rung 2: the planner must answer “what happens to mission success if the force commits platform i to waypoint v or releases weapon w at time t,” not merely “how did that maneuver correlate with success in historical engagements.” This distinction is foundational. Observational data alone cannot identify the mechanism of a control parameter beyond a Markov equivalence class of plausible structures [3, 4]: a correlation between a maneuver X and mission outcome Y is equally consistent with X driving Y, with Y driving X, or or with a third factor Z, such as an adversary's posture, driving both. This common-cause ambiguity cannot be resolved by any amount of passive observation. Only by actively intervening on X (executing the maneuver in a representative battlespace and observing the response of Y) can its causal effect be separated from confounding [3]. The ACE Framework operates exclusively in this interventional regime, which is precisely why it can certify that a learned battle-management behavior is causally responsible for mission success rather than an artifact of correlated training conditions.

The active experimentation problem. Given a Structural Causality Graph (SCG) supplied by domain experts, the governing question is which interventions to run. The combinatorial space is often intractable: in a recent analysis of how a complex hybrid space communications architecture supported naval operations, we had to optimize 7 parameters with 10 settings each, full-grid enumeration is 10^7 simulations. Classical theory bounds the number of interventions sufficient to identify causal relations among N variables [8] and establishes that adaptive designs require strictly fewer interventions than non-adaptive ones [9], but offers little guidance on the sequential choice of which parameter to intervene on and at what value. Traditional methods are sharply limited. Random sampling and full Design-of-Experiments grids treat interventions as static schedules and cannot adapt to what has been learned. Greedy information-maximization strategies, including the Bayesian Optimal Experimental Design (OED) methods that represent current published state of the art [6, 7], choose each intervention by maximizing expected information gain under the current posterior; these are principled but myopic, evaluating each step in isolation and unable to learn a multi-step strategy from experience. Worse, the Bayesian-OED family must maintain a posterior over graph structures, which is NP-hard and combinatorial in the parameter count [14], capping published demonstrations near twenty variables [7]. ACE sidesteps posterior maintenance entirely: it treats experimental design as a sequential decision problem and learns the policy from the campaign itself, which is what lets it operate in the thirty-parameter regime where the prior art does not scale [1].

The policy as a pretrained forward model. Choosing a pretrained transformer as the experimental policy is a deliberate theoretical commitment, not an engineering convenience [1]. Rung-2 design requires a forward model: the policy must anticipate “what will happen if I intervene on parameter X with value v,” and the quality of any policy is bounded above by the quality of its forward-model prior. An MLP trained from scratch begins with a uniform prior and must construct the entire forward model from experience; a one-step Bayesian acquisition function carries a correct but shallow prior; a human battle manager brings deep prior knowledge of how tactical parameters propagate. A language model pretrained on scientific and technical text is the closest computational analogue of that human prior: its weights encode a soft distribution over how variables in real systems relate causally, and published benchmarks confirm this prior is genuinely informative for causal-direction and mechanism queries [11]. Because in-context conditioning behaves as implicit Bayesian inference under that prior, prompting the policy with the SCG, the per-parameter losses, and the recent intervention history yields an approximate posterior-predictive distribution over sensible next interventions, which Direct Preference Optimization then specializes to the system at hand. Two structural properties make this practical for battle management: the action space (parameter, value) is emitted natively as text, so larger missions simply produce longer prompts without any change to the network; and the policy is model-scale-agnostic, demonstrated here at the 1.5B-parameter Qwen2.5 scale [13] and trivially upgradable to 70B-class production models.

ACE architecture (Figure 1). At each step ACE assembles a context describing the current state of the campaign: the structural causal graph, the current per-parameter mechanism-estimation losses, and a summary of recent intervention history. It renders this context as a structured prompt to the transformer policy, which generates K=4 candidate interventions in textual form (e.g., “do(waypoint_2_lat = 34.7)” or “do(weapon_aimpoint = ...)”). Each candidate is evaluated via lookahead: a cloned copy of the MLP student learner receives the data that executing the candidate would produce against the environment, and the resulting per-parameter loss reduction estimates the candidate's information value. The highest-scoring candidate is executed against the mission environment (the StarCraft II battlespace of SOW Task 4); the MLP student is updated with the new evidence; and the highest- and lowest-scoring candidates form a preference pair used to fine-tune the transformer policy via Direct Preference Optimization [5]. The composite reward that ranks candidates combines three components: information gain (the loss reduction from lookahead), node importance (parameters whose current mechanism estimates are poor receive higher weight), and a diversity term (under-sampled parameters and intervention values receive higher weight). The published component ablation establishes that the diversity term is non-optional: removing it causes the policy to collapse onto a single parameter and degrade below static baselines [1].

Figure 1

The transformer policy proposes K=4 candidate interventions from the current campaign state; lookahead scores each on a cloned student learner; the best candidate is executed on the mission environment and the (best, worst) pair updates the policy by scale-invariant DPO. Lookahead is a training-time signal only and is not used at deployment.

Why preference learning rather than value-based reinforcement learning (Figure 2) for Mission Planning. Active experimentation has a structural property that breaks standard value-based RL formulations: information gain is inherently non-stationary. Early interventions on an uninformed model produce large loss reductions; late interventions on a near-converged model produce negligible ones. The published ACE work measures roughly a 500x decay in expected information gain over a single training run [1]. Formally, the per-step reward decomposes as r_t(a) = f(t) g(a), where f(t) is a monotonically diminishing scale factor (the vanishing information gain) and g(a) is the approximately stationary relative quality of action a. A value-based RL critic [12] is therefore asked to predict a target whose magnitude shifts by orders of magnitude across the run, producing chronic value-prediction error and policy plateau (the published PPO baseline, given ACE's identical reward signal, stalls at 2.06 versus ACE's 0.61). Direct Preference Optimization [5] resolves this by consuming the candidate ranking rather than the candidate values: under the decomposition above, the ordering of any two candidates depends only on g(a) and is provably invariant to f(t), so the preference pairs fed to DPO remain well-defined throughout training even as absolute reward magnitudes collapse. Empirically the ranking is stable (Spearman correlation greater than 0.85) across the full training arc even when expected information gain drops two orders of magnitude (Figure 4) [1].

The use case demonstrates ACE’s tangible advantage for high-dimensional mission planning. The principal independent variables include Blue LEO antenna power, LEO altitude, LEO satellite count, MEO antenna power, MEO altitude, MEO satellite count, cruise missile velocity, and Red mission/target-specific jamming. These variables interact nonlinearly and produce thresholded outcomes: small architecture changes may have little operational effect when the system is far below the communications-support threshold, large effect near the mission-performance “elbow,” and diminishing returns after adequate link margin and coverage are achieved. The trade space assessed in this example consists of 564,480 discrete possibilities for unique variable combinations, which is demonstrable of a trade space where grid search, random sampling, and traditional Design of Experiments may substantially over-budget on uninformative regions.

ACE avoids this inefficiency by algorithmically selecting interventions that are expected to most densely inform mission-specific mechanisms. At each step, the ACE policy proposes candidate interventions, evaluates their expected information value, executes the most informative intervention, and updates the student mechanism learner with the resulting evidence. The highest- and lowest-value candidates are converted into preference pairs that update the LLM policy through Direct Preference Optimization, improving subsequent intervention selection so later experiments are increasingly aligned with the causal structure of the mission. ACE therefore does not merely sample the trade space; it learns how to experiment on the trade space.

In the satellite communications SCM evaluation, ACE evaluated the mission mechanism and converged on causal relationships using only 15 total interventions (Figure 5). In fact, after just two ACE interventions approximately 80 percent of the eventual MSE reduction had been achieved, showing that ACE rapidly identified the dominant causal levers rather than requiring broad enumeration. ACE reduced predictive mechanism error to a converged MSE of 0.0381, while Random Sampling and Grid Search remained materially higher over the same intervention budget. The result demonstrates that ACE’s advantage is not simply more sampling; it is better experimental selection. ACE spends each run on high-information regions of the trade space, while static sampling methods continue allocating runs to low-sensitivity or already-understood regions. This result is operationally significant because mission planning does not require every possible design point to be simulated; it requires rapid identification of which parameters matter, where they matter, and which combinations move the mission from failure to success. In practice this is what we often rely on mission commanders for, but ACE can yield superior insights in novel or time-constrained situations.