Context Compaction Evolution

源码版本rust-v0.145.0

codex treats conversation history as a finite resource. Once the accumulated tokens approach the model context window, older history has to be folded into a summary to free space for the next turn. This compaction path has been rewritten repeatedly in core/src/compact*.rs: locally, the model itself generates the summary; later a server-side remote compaction was added; and then v2 evolved. All three implementations share the same hook and analytics lifecycle, but where the "summary" comes from is completely different.

Responsibilities

1. Decide when to trigger compaction: three sources — auto (hits the token threshold), manual (user types /compact), or CompactionReason (model downgrade, comp hash change, etc.) — dispatched by run_inline_auto_compact_task and run_compact_task.
2. Run a summarization turn: the local version uses the Responses API itself to run one model call for SUMMARIZATION_PROMPT; the remote version has the server emit ResponseItem::Compaction directly; v2 further requires the server to return exactly one compaction item.
3. Build the replacement history: combine the summary + the last few user messages into the new history; the old history is discarded wholesale. build_compacted_history_with_limit trims user messages against a 20k-token cap.
4. Maintain the context window: each compaction opens a new window; advance_auto_compact_window / start_new_context_window update window_id / previous_window_id for subsequent token counting and rollout trace.
5. Expose hooks and analytics: PreCompactHookOutcome / PostCompactHookOutcome let plugins intercept compaction; CompactionAnalyticsAttempt reports the trigger, reason, implementation, and phase of each attempt.

Design motivation

The earliest local compaction (compact.rs) is "the client sends another request to the model to summarize". Its problem is that the summary itself costs tokens, and summary quality depends on how well the model follows SUMMARIZATION_PROMPT. Once model providers began natively supporting server-side compaction, compact_remote.rs appeared: the server returns a folded Compaction item directly in one Responses request, and the client does not participate in summary generation.

But v1 remote sends the entire history to the server and gets back only a summary; the client has to re-trim function-call output itself. v2 (compact_remote_v2.rs) tightens the flow: the server does not just return a summary, it also decides which messages are retained, and the client only collects the single Compaction output, failing fatally on error. v2 also introduces RETAINED_MESSAGE_TOKEN_BUDGET = 64_000, preventing retained messages from blowing up the context on their own.

compact_token_budget.rs is another branch: it skips summary generation and just opens a new window. This path is reserved for "don't spend another model call on a summary" scenarios — for example when the user explicitly says "clear context". It still goes through the full hook + ContextCompaction turn-item lifecycle; it just does not call the model.

Key files

codex-rs/core/src/compact.rs:150-219 — run_compact_task_inner, the local compaction main entry, with pre/post hook and analytics wrapping.codex-rs/core/src/compact.rs:221-378 — run_compact_task_inner_impl, runs the model stream, retries, and finally calls replace_compacted_history.codex-rs/core/src/compact.rs:602-663 — build_compacted_history_with_limit, pushing user messages from the tail forward until the 20k-token cap.codex-rs/core/src/compact_remote.rs:76-107 — run_remote_compact_task, the remote v1 entry, tagged ResponsesCompact.codex-rs/core/src/compact_remote_v2.rs:85-115 — the v2 run_remote_compact_task, tagged ResponsesCompactionV2.codex-rs/core/src/compact_remote_v2.rs:385-443 — collect_compaction_output, constraining the server to one compaction item, fatal otherwise.codex-rs/core/src/compact_remote_v2_attempt.rs:32-142 — run_remote_compact_v2_attempt, which trim_function_call_history_to_fit_context_windows before sending the request.codex-rs/core/src/compact_model_fallback.rs:8-19 — should_retry_with_current_model, defining which errors are worth a model-switch retry.codex-rs/core/src/compact_token_budget.rs:64-90 — token-budget compaction, skipping the summary and directly opening a new window.

InitialContextInjection is a seemingly trivial but critical enum: it decides whether the new history after compaction should inject initial context. Pre-turn / manual compaction uses DoNotInject, letting the next normal turn re-inject on its own; mid-turn compaction must use BeforeLastUserMessage, because the model is trained to treat the summary as "followed by the last user message", so the context must be spliced before the last real user message.

// compact.rs:56-69 — mid-turn 与 pre-turn 压缩的上下文注入策略
#[derive(Debug)]
pub(crate) enum InitialContextInjection {
    BeforeLastUserMessage(Arc<WorldState>),
    DoNotInject,
}

The core of local compaction is to send the whole history + SUMMARIZATION_PROMPT to the model, and after the stream completes, take the last assistant message as the summary. Note it does not use the model output directly as the new history; it wraps a prefix with format!("{SUMMARY_PREFIX}\n{summary_suffix}") so subsequent is_summary_message can recognize it.

// compact.rs:323-336 — 从模型 stream 结果里取出摘要并打标
let history_snapshot = sess.clone_history().await;
let history_items = history_snapshot.raw_items();
let summary_suffix = get_last_assistant_message_from_turn(history_items).unwrap_or_default();
let summary_text = format!("{SUMMARY_PREFIX}\n{summary_suffix}");
let user_messages = collect_user_messages(history_items);

let mut new_history = build_compacted_history(Vec::new(), &user_messages, &summary_text);

The key constraint for v2 remote is that the server must return exactly one ResponseItem::Compaction, otherwise it fatals:

// compact_remote_v2.rs:419-434 — server 返回多条 compaction 视为协议错误
if !saw_completed {
    return Err(CodexErr::Stream(
        "remote compaction v2 stream closed before response.completed".to_string(),
        None,
    ));
}
if compaction_count != 1 {
    return Err(CodexErr::Fatal(format!(
        "remote compaction v2 expected exactly one compaction output item, got {compaction_count} from {output_item_count} output items"
    )));
}

Data flow

Boundaries and failures

• ContextWindowExceeded self-heals: if the local compaction's own prompt blows the window, it loops history.remove_first_item() to drop the oldest entry and retries, only erroring out when one item remains and it still blows up (codex-rs/core/src/compact.rs:285-300).
• Tighter stream retry cap: v2 remote clamps retries to MAX_REMOTE_COMPACTION_V2_STREAM_RETRIES = 2, because compaction itself is slow; reusing the generic stream retry budget would stall one compaction for too long (codex-rs/core/src/compact_remote_v2.rs:54-57).
• Model fallback: errors like InvalidRequest, ContextWindowExceeded, ServerOverloaded trigger should_retry_with_current_model, retrying with the current main model to avoid a compaction being stuck on a provider quirk (codex-rs/core/src/compact_model_fallback.rs:8-19).
• Mid-turn injection position: in the post-compaction new history, initial context cannot simply be pushed to the end; it must be spliced before the last real user message. If there is no real user message, it is inserted before the summary, so the summary is always last (codex-rs/core/src/compact.rs:542-587).
• Token-budget compaction has no summary: it goes through start_new_context_window to swap the window directly without calling the model, but still emits a ContextCompaction turn item so that hooks and UI do not miss events due to a different path (codex-rs/core/src/compact_token_budget.rs:76-82).

Summary

The complexity of compaction mostly comes from "three implementations coexisting": local stream, remote v1, remote v2 each have their own entry and inner_impl, but share the CompactionAnalyticsAttempt, hook, and InitialContextInjection skeleton. When adding a feature, first confirm which path the provider takes — should_use_remote_compact_task is the branch point — then read the corresponding file. To see how the next model call goes out, continue with Client and Responses API; to see how the compaction result is stored into history, see session thread and message history.