Building blocks
Provide the complete functional anatomy of production agentic systems.
Before you begin
By the end
The reader can locate every model call, context item, plan, state transition, memory operation, retrieval step, tool action, identity, execution boundary, human control, trace, evaluation, agent message, adaptation step, operational mechanism, and artifact.
Chapters
Section titled “Chapters”Models and routing
- 01 Model roles and selection Explains model roles, capability profiles, selection dimensions, provider adapters, and cost-latency-quality trade-offs in production agentic systems.
- 02 Routing, cascades, and fallbacks Explores dynamic model routing, progressive escalation cascades, circuit breaker patterns, and multi-provider fallbacks for high-availability agent architectures.
- 03 Capability, cost, latency, and reliability Analyzes the multi-dimensional trade-offs between model capability, token economics, latency profiles, and operational reliability in production agent workflows. Deep dive
- 04 Routing evaluation Explains quantitative evaluation frameworks, cost-quality Pareto benchmarks, and judge calibration techniques for model routers and cascades.
Context construction
- 05 Context sources and precedence Explains the runtime discipline of context engineering, classifying dynamic prompt sources, trust boundaries, and precedence hierarchies in agent loops.
- 06 Context budgets, selection, and ordering Explains quantitative token budgeting, knapsack chunk selection algorithms, and attention-anchored positional ordering in agent context windows.
- 07 History, summaries, and compression Explains conversation history management, lossy and lossless token compaction, tool trace pruning, and recursive summarization for long-running agents. Deep dive
- 08 Provenance and context debugging Explains token-level lineage tracking, context debugging inspection consoles, OpenTelemetry telemetry standards, and boundary integrity auditing.
Planning and reasoning
- 09 Reactive and reason-act patterns Explains reactive action selection and interleaved ReAct (Thought-Action-Observation) loops, error recovery, and cycle bounds in autonomous systems.
- 10 Decomposition and plan-execute Explains multi-step goal decomposition, the two-tier Planner-Executor pattern, subtask lifecycle management, and dynamic replanning protocols.
- 11 Reflection, evaluation, and replanning Explains iterative reflection, the Evaluator-Optimizer architecture, episodic verbal reflection memory, and dynamic replanning protocols in autonomous agents. Deep dive
- 12 Search, budgets, and termination Explains deliberate reasoning via tree and graph search over candidate thoughts, multi-dimensional search budgets, heuristic pruning, and deterministic termination protocols. Deep dive
State and lifecycle
- 13 Run, thread, and event models Separate continuing conversations, task attempts, events and state, with explicit lifecycle conventions and runtime ownership.
- 14 Checkpoints, interrupts, and resumption Explains durable state checkpointing, human-in-the-loop breakpoint interrupts, state rehydration across infrastructure restarts, and state time-travel.
- 15 Retries, idempotency, and concurrency Explains resilient tool retries with exponential backoff and jitter, idempotency keys for duplicate side-effect prevention, and concurrency controls in multi-agent runtimes. Deep dive
- 16 Termination, cancellation, and cleanup Distinguish stopping from successful completion; explain cooperative cancellation, compensating actions and resource cleanup limits.
Memory
- 17 Memory versus context and state Explains the fundamental distinctions between ephemeral context windows, mutable execution state, and durable multi-tier memory systems across agent sessions.
- 18 Short-term and working memory Explores short-term and working memory mechanisms in AI agents, including in-context scratchpads, conversation buffers, token budgeting, and eviction strategies.
- 19 Persistent memory types and lifecycle Follow reusable memory records through writing, ranked recall, updates and deletion, with explicit storage and access limitations.
- 20 Consolidation, forgetting, and evaluation Explores how autonomous agents synthesize episodic memories into higher-level semantic insights, prune obsolete or decaying records, and evaluate memory performance.
Retrieval and rag
- 21 RAG system and ingestion Explores the foundational architecture of Retrieval-Augmented Generation (RAG) systems, comparing offline document ingestion pipelines with online query and grounded generation flows.
- 22 Sparse, dense, and hybrid retrieval Examines lexical, vector, and hybrid search methods, demonstrating how combining BM25 exact-token matching with dense semantic embeddings via Reciprocal Rank Fusion overcomes vocabulary mismatch and semantic drift.
- 23 Chunking, ranking, and advanced retrieval Explores advanced retrieval architectures beyond flat passage matching, including contextual chunking, cross-encoder neural reranking, hierarchical tree indexing with RAPTOR, and community-summarized knowledge graphs with GraphRAG. Deep dive
- 24 Grounding, long context, and retrieval evaluation Compare broad context and retrieval on measured tasks, separate ranking from claim support, and explain evaluator limitations.
Tools and function calling
- 25 Tool capabilities, schemas, selection, and dispatch Follow a proposed tool call from a capability description through argument validation, dispatch and a correlated result.
- 26 Results, failures, side effects, and confirmation Explain tool outcomes, uncertain failures, side effects, confirmation and idempotent retries.
Identity authorization and secrets
- 27 Identity, authentication, authorization, and delegation Trace the represented subject, acting workload and authorization decision through a delegated tool action.
- 28 Workload identity, credentials, and revocation Follow workload credentials through issuance, use, renewal, expiry and revocation without exposing them as model context.
Execution environments
- 29 Execution boundaries, isolation, and sandboxes Compare execution locations and isolation mechanisms, then describe their resources through an executor contract.
- 30 Browsers, files, networks, resources, and cleanup Map the files, browser sessions, network destinations and budgets exposed to a job, then separate retained output from temporary execution state.
Human in the loop
- 31 Human control, approval, and escalation Place explicit human decisions in a run, bind approval to a concrete proposal and define escalation and resumption behavior.
- 32 Interrupts, steering, feedback, and operator experience Distinguish interruption requests from acknowledged pauses, then apply steering, feedback and takeover to a recorded run state.
Observability and tracing
- 33 Events, traces, metrics, and correlation Use structured events, span relationships and aggregate measurements to reconstruct and measure an agent run.
- 34 Lineage, replay, redaction, retention, and integrations Connect versioned evidence to observed runs, distinguish replay from live execution, and design selective telemetry with explicit retention. Deep dive
Evaluation and benchmarks
- 35 Evaluation levels, test design, and metrics Measure components, observed steps, actual outcomes and repeated-run reliability using explicit acceptance rules.
- 36 Benchmarks, regression, and release evaluation Use benchmark results as scoped evidence and test candidate releases against retained behaviors, new cases and operational budgets. Deep dive
Multi agent systems
- 37 Multi-agent roles, delegation, and coordination Decide who owns the reply, what specialists receive and return, and how team work stops before adding agents.
- 38 Shared state, failures, termination, and evaluation Track task ownership, reject stale results and separate accepted outcomes from the reasons a multi-agent run stops. Deep dive
Learning and self improvement
← Previous section
Agent architectures
Published path
You reached the current end
