Planning under uncertainty

AI-guided design of efficient clinical trials with BATON

Naim Rashid

Department of Biostatistics · Lineberger Comprehensive Cancer Center · UNC Chapel Hill

Computational Medicine Seminar · September 4, 2026

UNC, Lineberger, and TBCRC logos.

Overview

Section 1

The problem

Complex adaptive design becomes a search problem.

Section 2

BATON

A search over a trial simulator.

Section 3

Does it work, and what did it change?

Section 1

The problem

Why isn’t simulation alone enough?

A clinical trial is an experiment built for a decision

The question
Does the treatment beat a benchmark or a control?

How patients are used
Single-arm (everyone gets the treatment) or randomized (treatment versus control).

Two errors
Advance a treatment that fails, or drop one that works.

Two choices dominate: how many patients, and which rule decides.

Bayesian adaptive trials decide as they go. Each rule adds settings.

Two schematic timelines. A traditional trial enrolls all patients then analyzes once. An adaptive trial enrolls in blocks separated by interim looks that can stop the trial early for futility, ending in a final go-or-stop analysis.

Maximum N, look timing, futility and efficacy thresholds, priors, margins: 8 to 13 interacting choices, and every one changes how the trial behaves.

Say aloud: “The design is Bayesian because those decisions rest on posterior probabilities under a model and a prior.” Gloss “futility” (the treatment looks unlikely to work) and “interim” (a look before the end).

Pull-from-if-needed: The precise contrast, per the paper: unlike classical group-sequential designs (Jennison and Turnbull), complex Bayesian adaptive trials generally lack closed-form operating characteristics (Berry 2010; Thall and Simon 1994). The flexibility comes from posterior-probability monitoring; the cost is that neither formulas nor enumeration reach the behavior.

Q&A pocket: Bayesian rule, frequentist error rates? The monitoring rule is Bayesian (posterior probabilities drive stopping); regulators require frequentist behavior demonstrated by simulation. Calibration is the job of making one satisfy the other.

Feasible designs meet prespecified error-rate and power targets

Type I error (false go)
Share of simulated trials declaring efficacy under the null. Cap: 0.10.

Power (true go)
Share declaring efficacy when the treatment works as hypothesized. Floor: 0.80.

	Typical trial	EVOLVE
Designs needed	One	Many subtrials
Design parameters	Few	Many, interacting
Power and type I error	Often closed-form	Simulation only

Simulation tells us how one candidate behaves. It does not tell us which candidate to try next.

One candidate design simulated ten thousand times under two truths. When the alternative is true, 8,630 of 10,000 simulated trials declare efficacy, so power is 0.86. When the null is true, only 13 declare efficacy, so type I error is 13 in 10,000, with average enrollment under the null of 11.7 patients.

EVOLVE: one master protocol, 10 to 15 cohorts opened over time, each needing its own design

Schematic of the EVOLVE platform with cohorts tagged single-arm, randomized, and seamless: a master protocol spine spawning a sequence of biomarker-defined subtrials over time, each carrying its own design and calibration task.

Not one calibration problem. Ten to fifteen of them, each with its own design, opened on a short clock.

One subtrial became a two-week search

2 weeks

manual tuning of one subtrial’s design
8 interacting settings

power stalled at 0.78 with 39.2 expected patients when the treatment is inactive

Several more subtrials were waiting. And even if you find a feasible design, is it efficient under the objective you actually care about?

This is not just EVOLVE: simulation-calibration recurs across adaptive trial designs

Phase I dose-finding
escalation and safety rules tuned by simulation

Phase II Bayesian designs
posterior thresholds and interim looks tuned by simulation

Platform trials
every added feature adds settings that interact

Same shape every time: many interacting settings, operating characteristics evaluated by simulation, and calibration that still requires search.

Delivery: First person. “I hit exactly this designing phase I and phase II trials with the standard tools, for the same reasons. The more ambitious the design, the worse it gets.”

Pull-from-if-needed: The paper’s own generality claim (Discussion): the calibration challenge “arises across simulation-calibrated Bayesian adaptive designs, including basket, umbrella, response-adaptive, and enrichment designs” (Berry et al. 2010; Woodcock and LaVange 2017 master protocols; Golchi 2022); thresholds cannot be tuned in isolation (Granholm et al. 2025). Existing Bayesian phase II calibrators (BOP2 and successors) search posterior-probability thresholds by grid or particle swarm and return one design under one criterion. The phase I claim on the slide is my own experience, not a citation from the paper.

Grid search scales poorly; random search wastes evaluations in our benchmark

Heatmap of a fictional two-dimensional design-space slice, darker meaning better, covered by a uniform lattice of one hundred evaluation dots, most sitting in light low-quality regions.

The same heatmap with one hundred evaluation dots scattered uniformly at random; most again land in light low-quality regions.

In two dimensions a grid is easy. As dimensions grow, the number of combinations explodes. Random search spent 93% of its 2,000 evaluations on designs that fail. (Darker = better design.)

Confirmed 2026-09-04 (scripts/run_random_search_analysis.R, results/validation/random_search_summary.csv): the random-search benchmark drew 2,000 configurations uniformly over the binary two-stage (Simon-type) design space, the same parameter family as the Simon benchmark (efficacy and futility thresholds, stage-1 size, total size) over its own search box, 2,000 simulations each; not commensurable with the ten-seed reproducibility run; 136 feasible (6.8%); best random design expected null N 17.75 in 9.3 minutes. If asked which dimension: the Simon-type parameters, not the 8- or 13-parameter EVOLVE designs.

Q&A pocket: Didn’t random search find a better value than BATON somewhere? Different design class and an unvalidated selection minimum over 2,000 noisy estimates; the commensurable internal comparison is on the benchmark slide later.

Delivery: If the meme slide follows, this is its setup; one beat only.

Section 2

BATON

What should we simulate next?

This is a familiar computational problem

Machine learning

hyperparameters
↓
expensive training run
↓
validation performance

Trial design

design parameters
↓
expensive Monte Carlo simulation
↓
power, type I error, enrollment

The same computational shape as hyperparameter tuning, except feasibility is defined by hard statistical constraints.

BATON needs a simulator, not a formula

BATON: Bayesian Adaptive Trial OptimizatioN

Constrained Bayesian optimization: surrogates learn power, type I error, and enrollment; they choose the next candidate.

You specify: a power floor, a type I error cap, what to minimize.

Your simulator: settings and scenarios in, operating characteristics out.

Four-box cycle: propose a design, simulate trials via the trial simulator, learn from everything seen via a surrogate model, choose the next design worth trying, and repeat.

A closed loop: simulate, learn, choose, repeat.

A surrogate predicts designs we have not paid to simulate

Scatter of nine simulated designs along one design setting, each with a Monte Carlo error bar on its estimated power.

The same nine points with a smooth fitted curve and an uncertainty band through them, labeled as the model's prediction for designs never simulated.

Design settings in, trial behavior out, and the model says how sure it is.

The next simulation goes where it has the greatest expected value for improving the feasible design

Schematic two-dimensional design space with a probably-infeasible region, a probably-feasible region containing a low-predicted-enrollment area, an uncertainty band along the feasibility boundary, and a single next-simulation candidate marked on that boundary near the low-enrollment area.

Balance the potential improvement in the objective against the probability that the design satisfies the constraints.

The seed sketches the map. Then BATON proposes each batch.

Heatmap of a fictional design-space slice, darker meaning better, with fifteen scattered space-filling evaluation dots and no lattice.

The same surface with thirty model-guided evaluations clustering on the dark high-quality basin; repeated cells overlap. No connecting lines.

Initial points learn broadly; later points concentrate where the answer may improve.

Spend precision where it matters

Cheap screening for the many, expensive re-checks for the few, multi-seed verification for the one design you report.

What the code actually does (verified against the driver 2026-09-03): Stage 1 searches at 3,000 replications; the top 3 feasible designs, then 2 more, are re-simulated at 10,000. If at least 30% of them survive, stages 2 and 3 are skipped and the best validated design goes to verification. Only if validation fails does the search escalate with narrowed bounds at 5,000 and then 10,000. In the production runs behind the paper’s tables, validation succeeded after Stage 1 in essentially every run, so the escalation stage is drawn dashed. Do not describe it as “three rounds of increasing fidelity” out loud.

Pull-from-if-needed: Name the failure mode if asked: selection-induced optimism, the winner of a noisy search looks better than it is precisely because it won. The verification stage exists to remove it: independent multi-seed re-simulation at high fidelity, reporting means and an upper confidence bound on type I error (worst seamless mean 0.084, bound near 0.087 against 0.10; 100% strict feasibility across seeds). Reported numbers come from verification, never from the search.

Section 3

The evidence

Feasibility, efficiency, and richer designs

On a problem with a known answer, BATON recovers the optimum or a near-neighbor

Simon’s two-stage design: enroll n1, stop if too few responses, otherwise enroll to n. Every candidate in this design class can be enumerated, so the exact constrained optimum is known.

Best feasible expected sample size against cumulative design evaluations for ten seeded runs: the space-filling phase plateaus near 19.7, then the model-guided phase drops onto Simon's exact optimum at 17.74, where eight of ten runs terminate.

10 random starts: 8 return Simon’s exact optimal design; 2 land 0.15 patients away.

Across 12 benchmarks: feasible in all 12, median objective gap 2.5%

One run per scenario; ratio = BATON’s expected null enrollment / Simon’s exact optimum.

Claim	Value
Feasible design returned	12 / 12
Exact Simon design returned	4 / 12
Median ratio, BATON / Simon	1.025
Worst-case ratio	1.135
Best-case ratio	1.000

Feasibility is verified directly. Objective optimization remains approximate.

In EVOLVE, BATON found a feasible design where manual tuning had stalled

MANUAL
2 weeks
power 0.78
39.2 expected patients if inactive
infeasible

BATON
~4-hour workflow
power 0.86
11.7 expected patients if inactive
feasible

Illustrative comparison, not a controlled benchmark across analysts. Same simulator, same constraints.

Same requirements, different trials

Many designs meet power ≥ 0.80 and type I error ≤ 0.10. Which should we optimize for?

EVOLVE’s TNBC cohort, different objectives:

	Design A	Design B
Maximum patients	47	30
Expected patients, inactive treatment	11.7	22.8
Power	0.86	0.94
Type I error	0.001	0.021

One quits losers quickly; one caps the worst case. Neither is better.

The objective should reflect the trial you want

H0-Optimal (quit losers quickly)

H0: no better than the benchmark

Minimize expected N under the null; larger maximum N.

Minimax (cap the worst case)

Minimize maximum N; more patients under the null.

Admissible (the compromise)

A weighted blend, explicit.

Two-stage designs made these tradeoffs explicit for decades; BATON does so at scale.

Spoken (moved off the slide): “A feasibility criterion tells us whether a design is acceptable. It does not encode which acceptable design we prefer.” Full caption: “Frequentist two-stage designs made these tradeoffs explicit for decades. BATON makes the same tradeoffs explicit when exhaustive enumeration is impractical.”

Say once, here: “From here on, when I say efficient, I mean efficient in patients, not faster computation.”

Delivery, spoken transition into this slide: “So far I have pretended there is one objective. There is not. Power and type I error tell us whether a design is acceptable; they do not tell us which acceptable trial we would rather run.” Clinician anchor: “this is really a choice between how many patients we expose to an inactive treatment and how much worst-case capacity the network must reserve.”

Pull-from-if-needed: Simon (1989); Jung’s admissible designs. The claim is transport, not “first to notice tradeoffs.” BATON takes the weighting as input: explicit, on the record, before any data. And because we model the whole response surface, we can begin to see which design parameters drive these tradeoffs.

Q&A pocket: When is this locked? At design time, in the protocol, before data are seen, like a Simon optimal-versus-minimax choice.

Q&A pocket: Why not multi-objective BO for the whole front? Fair alternative; named scalarizations keep constraint handling direct and give a steering committee a legible, pre-registrable objective.

EVOLVE’s TNBC cohort: three objectives, three designs

Trade-off plane for the TNBC single-arm cohort: H0-Optimal at maximum 47 and expected 11.7 under the null; Admissible at 31 and 11.9; Minimax at 30 and 22.8; a dashed frontier connects them.

The platform’s TNBC cohort

H0-Optimal: 47 max, 11.7 if the treatment is inactive
Admissible: 31 max, 11.9
Minimax: 30 max, 22.8

For one additional patient of maximum capacity, Admissible saves about eleven patients on average when the treatment is inactive. That made the compromise especially attractive.

Search makes a 13-parameter seamless design practical to calibrate

$Three trial architectures with rising parameter counts, ending in a seamless design that begins single-arm and converts to a randomized comparison at an interim gate: thirteen parameters.$

Screen first with a single-arm stage, then demand randomized evidence from the cohorts that earn it. Thirteen interacting parameters, calibrated by automated search.

Two philosophies converged on the same 13-parameter design

Seamless trade-off plane with three philosophies: H0-Optimal at maximum 150, and Minimax and Admissible landing on one identical point at maximum 108 and expected 87.6, circled.

Minimax and Admissible calibrated to the identical design. That is not a coincidence, and the next slide shows why.

The gate rarely opens when the treatment is inactive, so the two objectives collapse

When the treatment is inactive, only ~4% of trials reach stage 2. Expected enrollment therefore changes very little with stage-2 size, so the maximum-enrollment component drives the Admissible solution toward Minimax. H0-Optimal can tolerate a larger stage 2 because its objective never penalizes maximum N.

BATON informed EVOLVE’s operational decisions

Capacity: maximum N locked 3 months before first enrollment, from the frontier.

Program planning: enrollment and timeline projections for 10 to 15 subtrials.

Interim calendar: look spacing set the safety board’s schedule.

Turnaround: a feasible design in under 45 minutes; workflow about 4 hours.

We generally chose Admissible designs: efficient enrollment, early stopping, bounded capacity.

BATON is designed as a general framework, not an EVOLVE-specific tool

Cropped title block of the BATON manuscript: Constrained Bayesian Optimization for Calibration of Bayesian Adaptive Clinical Trials, under review.

Official BATON package hex sticker.

Manuscript under review at JASA
Open-source R package
Bring your own trial simulator

Problems often emerge late

54%
of 640 therapeutics reaching phase 3 or pivotal trials failed in late-stage development; 57% of failures for inadequate efficacy

Locally reasonable decisions can leave problems that only become visible downstream.

Hwang et al., JAMA Internal Medicine 2016

Early decisions become downstream commitments

Sotorasib (KRAS G12C)

960 mg: highest phase I dose tested
↓ carried forward
selected, developed, approved
↓ after approval
FDA required a 240 vs 960 mg comparison
↓
median PFS was similar at 240 mg and 960 mg

The optimal-dose question remained unresolved until after approval.

Source: CodeBreaK 100 dose comparison; Ann Oncol 2023, Eur J Cancer 2024

Confirmed 2026-09-04 (CodeBreaK 100 randomized dose comparison, 960 vs 240 mg, n = 209; Ann Oncol 2023 VP4, Eur J Cancer 2024): median PFS 5.4 vs 5.6 months (HR 0.95, 95% CI 0.67 to 1.35); ORR 32.7% vs 24.8%; median OS 13.0 vs 11.7 months (HR 0.75, 95% CI 0.53 to 1.07). FDA considered the postmarketing requirement fulfilled in December 2023 and 960 mg remains the labeled dose. Do not say “similar efficacy”: ORR and OS trended toward 960 mg; PFS is the endpoint that was similar. Gloss PFS aloud: “time until the cancer worsens or the patient dies.”

Pull-from-if-needed: FDA Oncologic Drugs Advisory Committee review of sotorasib dosing and CodeBreaK 200, October 2023; the design lesson (the highest tolerated dose was not the best one) is drawn in our RAS-era trial-design commentary. Be fair to the program: the early trial did its local job and the drug is approved and effective; the point is that a fundamental early decision stayed unresolved enough to require post-approval revisiting.

Delivery: Human-scale information debt. Walk the chain, land the sentence, and say this distinction aloud: “Nothing here means the Phase I trial was incorrectly analyzed. It means the question it was optimized to answer was narrower than the decision the rest of the development program ultimately needed.” Then move to the extensions.

We optimize the trial. The program is what succeeds.

Phase I
choose dose

choice carries forward
→

Phase II
choose population, signal

uncertainty carries forward
→

Confirmatory trial
must resolve remaining uncertainty

What if we reverse the logic?
final therapeutic claim + its evidentiary requirements
↓
simulate alternative development architectures
↓
phased · seamless · potentially phaseless

Phase structure becomes a design variable.

Q&A pocket: Isn’t this seamless designs again? Seamless merges adjacent phases operationally. This changes what each stage is optimized for, and ultimately whether the partitions are assumptions at all.

Delivery: Narrate the build: “We behave as if these are separate optimization problems. But Phase I chooses a dose that Phase II inherits; Phase II chooses a population and signal the confirmatory trial inherits. Errors do not disappear at the phase boundary.” Reveal the reversal: “Start from the evidence the final claim requires, then derive the fastest architecture that produces it reliably. Sometimes that still looks like Phase I, II, III. Sometimes seamless. Sometimes fewer boundaries.” Then say the clarification, aloud and explicitly: “Phaseless does not mean confirmation-less. It means phase boundaries are design choices rather than assumptions.” “Phaseless does not mean evidence-less or confirmation-less. It means the Phase I, II, III boundaries stop being assumptions and become design choices. Some programs may still need all three; others may support the same reliable claim with fewer separations or no distinct selection-confirmation boundary.” Then the payoff: “The goal is not to weaken evidence by removing phases. It is to stop paying automatically for boundaries that may not be necessary.” Slowest delivery in the talk. Deeper point if pressed: some information lost through earlier selection is expensive or impossible to recover downstream, which is exactly why the program, not the trial, is the right unit of design.

Pull-from-if-needed, the economic chain: wrong early choice, larger downstream trial, failed confirmatory study, repeat development: years, patients, cost. Versus: optimize the program, better upstream choices, reuse accumulating evidence, fewer redundant transitions: faster, smaller development.

Extending the same search logic across development

Across possible futures

BATON-C · in progress, with Amber Young

Optimize over a prespecified family of plausible truths, not one.

Earlier in development

BATON-D · in progress

Optimize dose-finding rules for safety and for what later trials inherit.

Across trials

Phaseless / whole-program design · long-term goal

Optimize dose, population, interim, and confirmatory decisions as one program.

What I hope you remember

For complex adaptive trials, simulation tells us how a candidate behaves; design is the search for what to try next.
BATON searches the simulator for feasible, efficient designs and makes the patient-use tradeoffs explicit.
Tractable calibration lets us consider richer adaptive trials; the same search logic extends to other decisions across development.

Acknowledgments

EVOLVE / ADAPT

MPIs: Lisa Carey (UNC), Ian Krop (Yale), Chuck Perou (UNC), Eric Winer (Yale), Antonio Wolff (Hopkins)
EVOLVE steering committee and site investigators across 15 TBCRC sites
ARPA-H ADAPT program leadership

Statistics / BATON

Amber Young (UNC), who led BATON's design and validation, and co-leads BATON-C
Didong Li (UNC) · Dinelka Nanayakkara (ARPA-H statistics group)
Statistical working group: Ruizhe Chen (Hopkins), Sue Hilsenbeck (Baylor), Nabihah Tayob (DFCI)
Rashid Lab

FUNDING · ARPA-H 140D042590009 · NCI P50-CA257911 (UNC SToP SPORE) · DOD HT9425241103100121 · NCI U01-CA274298

Thank you

Questions

naim@unc.edu · rashidlab.github.io
BATON R package: github.com/naimurashid/BATON · manuscript under review at JASA

Q&A routing index (v3 numbering): - Why 0.10 / 0.80, conventions -> slide 6 notes. - Bayesian vs frequentist hybrid -> slide 5 notes. - Random search “beat” you -> slide 11 notes; slides 20 to 21. - GP, acquisition, correlated constraints, CRN -> backup B1 (code-verified answers). - Simon seed accounting, worst case -> slide 21 notes; backup B2 (scatter), B7 (ten seeds). - Collapse an artifact? -> slide 27 notes; slide 28. - Unspent alpha at 0.001, search-box edges -> slide 25 notes; backup B3. - Multi-objective BO; when is the philosophy locked -> slide 24 notes. - Two-weeks story detail -> slide 22 (manual vs BATON). - Limits, failure modes -> backup B5. Bidirectional stopping -> B6. Randomized cohort -> B8. - ARPA-H requirements, the program -> backups B9, B10. - Company / translation -> disclose relationship verbally; keep to one sentence.

Backup

Reference slides

Backup: the surrogate and the acquisition

Two-panel schematic. Left: a Gaussian process posterior over one design dimension with wider credible bands in regions of higher Monte Carlo noise, illustrating the heteroskedastic variance model. Right: the acquisition surface formed by multiplying expected improvement by the modeled probability of feasibility, peaking near the constraint boundary.

One GP surrogate per behavior metric (power, type I, expected enrollments).
Heteroskedastic noise model: Monte Carlo variance differs across the space (early-stopping designs give noisier estimates per replication), and hetGP models it rather than assuming it constant.
Acquisition: expected constrained improvement, expected objective gain multiplied by the modeled probability that every constraint is satisfied.
Batched proposals; constraint estimates carry their Monte Carlo standard errors into the model.

All three answers below are verified against the BATON source (v0.4.0, the release behind the paper’s results; unchanged in v0.8.0 unless noted).

Q&A pocket: How is the incumbent defined under noise? Best posterior mean, not best observed value: take the evaluated designs whose observed metrics satisfy the constraints, predict the objective surrogate’s posterior mean at each, and use the minimum. Feasibility classification is a plug-in on the simulated metrics; the objective value is de-noised through the surrogate. This is the plug-in form of Picheny (2013), and the manuscript states it that way. A best-observed option exists in the code but is not the default and was not used.

Q&A pocket: Constrained EI multiplies marginal feasibility probabilities; power and type I come from the same evaluation. Independence? Independent surrogates and a product of marginals, deliberately. The noise justification: power and type I are simulated under disjoint scenarios from non-overlapping segments of the RNG stream, so their Monte Carlo errors are independent even though they share an evaluation call. The latent surfaces can still be correlated as functions of the design, the paper says so, and the approximation affects only the search: every reported design is verified by direct simulation, never through the product. (Wording caveat: the supplement says “distinct random seeds”; the code uses consecutive segments of one stream. Independent either way.)

Q&A pocket: Common random numbers? Not during the search: every evaluation gets its own seed (base seed plus evaluation index), so candidate comparisons carry full independent Monte Carlo noise, and higher-fidelity re-evaluations are independent replicates, which is what the heteroskedastic surrogate’s replicate structure wants. The five-seed verification stage does use a shared seed set, so those cross-design comparisons are paired. If pressed on why no CRN in the loop: it was never implemented, not evaluated and rejected; variance control comes from the fidelity ladder and the noise model instead.

Q&A pocket: Isn’t plug-in feasibility for the incumbent inconsistent with a smoothed objective? A real asymmetry, concede it: feasibility of the incumbent set uses observed metrics while its value uses the surrogate mean. Mitigations: the search applies a buffered (tightened) constraint threshold, and no design is reported without high-fidelity multi-seed verification.

Backup: the 12-benchmark scatter

Per-scenario ratio of BATON to the Simon optimum plotted against effect size; most scenarios sit inside a plus-or-minus five percent band around one, the worst near 1.14.

Always feasible: 12 of 12. Exact optimum in 4 of 12; median within 2.5%; worst case 13.5%.

Primary target: verified feasibility. Objective optimization is approximate in an expensive, noisy, discrete space.

Backup: what the constraints and parameters actually are

Calibrated (searched) parameters, single-arm:

maximum enrollment; interim spacing; minimum-events gate before decisions count
posterior futility threshold; posterior efficacy threshold
decision margin over the historical benchmark
analysis-prior concentration; hazard-model resolution (piecewise intervals)

Seamless adds conversion-gate thresholds and second-stage sample-size and stopping parameters (13 total).

Constraints (headline scenarios):

type I error at most the ceiling (0.10 in the scenarios shown)
power at least the floor (0.80)
seamless designs add gate-behavior constraints (null conversion rate bounded, alternative conversion rate floored)

Objectives, by philosophy: expected null enrollment (H0-Optimal), maximum enrollment (Minimax), weighted blends (Admissible family), expected alternative enrollment (H1-Optimal).

Scenario A truths: null median 3.8 months, alternative 9 (hazard ratio 0.42), Weibull shape 1.3, accrual 2 per month; maximum enrollment searched in [30, 120]. Two Simon scenarios use a 0.05 ceiling with 0.90 power.

Backup: related work and what BATON adds

Existing approaches

BOP2 / GBOP2 (Zhou; Zhao): grid or particle-swarm search over posterior-probability stopping thresholds; return one design under one criterion.
Operating-characteristic acceleration (Golchi 2022): speeds the estimation, does not optimize the design.
Single-objective BO for trial design (Richter 2022): one objective, without multi-constraint feasibility.
gsDesign / rpact / BATSS: group-sequential and frequentist tooling.

What BATON adds, jointly in one framework

finds feasible designs under multiple noisy behavior constraints;
models heteroskedastic Monte Carlo error;
optimizes under philosophy-specific sample-size objectives.

It exposes the trade-off surface, and reveals when philosophies coincide and why.

Backup: known failure modes and limits

Budget exhaustion: 3 of 10 reproducibility runs used the full 300-evaluation budget rather than stopping early; quote evaluations used, not “convergence.”
Approximate optimization: worst benchmark gap 13.5% from the enumerated optimum (single seed); the guarantee is high-fidelity feasibility, not exact optimality.
Correlated constraints: the acquisition multiplies marginal feasibility probabilities; latent surfaces for power and type I error can be correlated. Affects the search only; reported designs are verified directly.
No common random numbers in the loop: candidate comparisons carry full independent Monte Carlo noise; verification uses a shared seed set.
One assumed truth per calibration: every current result conditions on an assumed clinical scenario; robustness across scenarios is BATON-C, in progress.

Backup: bidirectional stopping variants

The single-arm and between-arm headline scenarios use futility-only interim stopping (interims can stop for futility; efficacy is decided at the end). The seamless design is calibrated with bidirectional stopping throughout, because its conversion gate is efficacy-driven. The paper’s supplement repeats the single-arm and between-arm calibrations with bidirectional stopping, where interims may also stop early for efficacy.

The frontier structure persists, but the philosophies compress: under bidirectional stopping in Scenario A, H0-Optimal and Admissible calibrate to the same maximum enrollment (31), separating only in thresholds.
Expected enrollment under the alternative drops substantially (early efficacy stopping pays off exactly when the drug works).
The choice between stopping structures is itself a design decision upstream of the philosophy choice.

Backup: ten seeds, one design

The whole pipeline, rerun end to end at ten independent random seeds:

Runs returning a feasible design	10 / 10
Runs returning Simon’s exact optimal design	8 / 10
Expected null enrollment, mean (spread)	17.77 (0.06)
Evaluations used, median	250 of a 300 budget

The other two seeds return a near-neighbor design (capacity 29 instead of 25) whose expected enrollment differs by 0.15 patients. What reproduces is the design’s behavior, and usually the design itself.

For a method meant to sit inside regulatory submissions, this is not a nicety. That is the bar for a design going into a regulatory submission.

Pull-from-if-needed: Expected null enrollment takes two values across the ten seeds: 17.74 in eight runs and 17.89 in two. Simon’s exact optimum for this scenario (response rates 0.20 versus 0.40) is 17.74, so the modal BATON design IS the Simon optimum, a 0.2% simulated gap. Three of ten runs used the full 300-evaluation budget; quote evaluations used, never “convergence.” Continuous thresholds vary across seeds (futility 0.09 to 0.26) while mapping onto the same integer stopping boundaries: the behavior surface is piecewise constant in the thresholds, which is why behavior reproduces exactly.

Q&A pocket: Do all ten seeds return the identical parameter vector? No, and say so plainly: eight of ten return the identical vector, which is Simon’s optimum; two return capacity 29 instead of 25. The design vector is the contract with sites and budget, so the honest claim is “one of two designs, the modal one exactly optimal,” not “one design, always.”

Delivery: Fast slide. The 8-of-10-exactly-Simon line is the only number to say aloud.

Backup: the randomized-cohort frontier

Trade-off plane for the ER-positive doublet-versus-triplet cohort: maximum 124 at expected 65, 104 at 73, and 99 at 82, steepening toward the Minimax end.

Cutting the worst-case cap from 124 to 104 costs about 8 extra patients on average when the treatment does not help; the next 5 cost about 9 more.

Backup: the ARPA-H ask and the N-of-1 addition

(ii) Clinical trial characteristics:

Clinical trial for metastatic cancer patients.
Clinical trial that includes multiple lines of therapy under a single protocol.
Flexible clinical trial protocol that enables changes in design (e.g., randomized, adaptive, etc.).
Strong capability to collect tumor and blood-based measurements.
Possess a treatment line with a Progression Free Survival (PFS) rate of <9 months average.
Mechanisms for biomarker evaluation.
Modular clinical trial design enabling rapid dissemination and protocol uptake by different sites.

(iii) General characteristics: key clinical needs; evolutionary design examples and rapid-enrollment plans; treatment lines; biomarker testing and integration plans.

New ask (program-manager email): integrate an "N-of-1" trial component.

excerpts from the ARPA-H solicitation and program-manager correspondence

The ask: design an adaptive multi-line trial to discover, evaluate, and implement novel biomarkers, with an additional N-of-1 subcomponent

Animated this-is-fine cartoon from the original slide.

Backup: ADAPT technical areas and the EVOLVE team

Official Meet the Team slide: MPI photographs and roles beside tables for the steering committee, fifteen participating TBCRC sites, and the working groups.