Deterministic Token Scoring Versus Autoregressive Deliberation: Mitigating Decision Fragility in Automated Real-Time Settlement Policy Screening

Abstract

Modern wholesale and retail payments have converged upon instant gross settlement rails, such as FedNow, The Clearing House Real-Time Payments (RTP), Target Instant Payment Settlement (TIPS), and Brazil’s PIX, where transactions clear with irrevocable legal finality within sub-second latency envelopes. Concurrently, escalating regulatory obligations spanning sanctions enforcement, antimoney laundering (AML), and intraday liquidity quotas demand rigorous automated pre-settlement policy screening. Recent industry initiatives have explored deploying generative foundation models and multi-step Chain-of-Thought (CoT) reasoning to interpret complex compliance rules over rich ISO 20022 payment messages. In this paper, we conduct an extensive theoretical and empirical examination of unconstrained autoregressive language model deliberation within instant settlement architectures. Evaluating frontier models across 12,500 real-world and synthetic institutional payment flows, we uncover severe decision fragility: multi-step autoregressive rationalization induces cumulative semantic drift, causing models to invent non-existent regulatory safe harbors, exhibit acute option-order sensitivity (OOS reaching 34.2%), and breach sub-second operational service-level agreements by accumulating latency in excess of 2,400 milliseconds. To eliminate these vulnerabilities, we design and validate the Deterministic Token Scoring (DTS) framework. DTS abandons free-form token generation, replacing it with an architecture that pairs deterministic Abstract Syntax Tree (AST) parsing of ISO 20022 messages with constrained single-token logit cloze scoring over an authorized policy decision vocabulary subspace Vpolicy. Evaluating candidate settlement outcomes directly through unnormalized projection logits guarantees mathematical order invariance (OOS ≡ 0.0%), eliminates intermediate semantic drift, reduces per-transaction compute latency to 42 milliseconds, and guarantees 100% syntactic compliance with ISO 20022 status reporting standards. Our findings demonstrate that in safety-critical, legally irrevocable financial clearing, deterministic constrained token extraction decisively outperforms open-ended generative deliberation.