Building blocks
Provide the complete functional anatomy of production agentic systems.
Follow the chapters in order. Each one combines a clear explanation, local illustrations, traceable sources, and runnable examples where code helps.
Module Overview & Outcomes
Section titled “Module Overview & Outcomes”The reader can locate every model call, context item, plan, state transition, memory operation, retrieval step, tool action, identity, execution boundary, human control, trace, evaluation, agent message, adaptation step, operational mechanism, and artifact.
Required Concepts & Scope
Section titled “Required Concepts & Scope”Follow the numbered child sections. Preserve distinctions among context, state, memory, and artifacts. Treat identity, execution, policy enforcement, and the engineering lifecycle as functional architecture before their security implications.
Published chapters in this section
Section titled “Published chapters in this section”Model roles and selection
Explains model roles, capability profiles, selection dimensions, provider adapters, and cost-latency-quality trade-offs in production agentic systems.
- Differentiate the core model roles in agent architectures: planner, router, worker, and evaluator.
- Evaluate models across capability dimensions including reasoning depth, latency, token pricing, context retention, and structured tool schema compliance.
- Implement provider adapters to decouple application logic from vendor-specific API formats.
Routing, cascades, and fallbacks
Explores dynamic model routing, progressive escalation cascades, circuit breaker patterns, and multi-provider fallbacks for high-availability agent architectures.
- Implement dynamic routing mechanisms including rule-based, embedding similarity, and learned threshold routers.
- Design progressive model cascades (FrugalGPT) that escalate from fast SLMs to frontier reasoning models upon confidence failure.
- Construct resilient circuit breaker gateways with automated provider failover, jittered retries, and graceful degradation.
Capability, cost, latency, and reliability
Analyzes the multi-dimensional trade-offs between model capability, token economics, latency profiles, and operational reliability in production agent workflows.
- Deconstruct inference latency into Time to First Token (TTFT), inter-token latency (TPOT), and queue delays across streaming and non-streaming modes.
- Calculate total agent loop economics including prompt token vs completion token pricing, context caching discounts, and reasoning token overheads.
- Evaluate model capability trade-offs across reasoning depth, tool schema adherence, and long-context needle degradation.
Routing evaluation
Explains quantitative evaluation frameworks, cost-quality Pareto benchmarks, and judge calibration techniques for model routers and cascades.
- Deconstruct router evaluation metrics including Call-Through Rate, Quality Recovery, and Cost Reduction on Pareto curves.
- Implement offline and online router evaluation benchmarks using ground truth and calibrated LLM judges.
- Mitigate router evaluation biases including position, length, and domain distribution shifts.
Context sources and precedence
Explains the runtime discipline of context engineering, classifying dynamic prompt sources, trust boundaries, and precedence hierarchies in agent loops.
- Distinguish dynamic context engineering from static prompt engineering across autonomous agent iterations.
- Classify runtime context sources across system instructions, developer policies, user queries, memory history, retrieved evidence, and tool outputs.
- Establish strict precedence rules and trust boundary tags to prevent lower-tier context from overriding higher-tier authority.
Context budgets, selection, and ordering
Explains quantitative token budgeting, knapsack chunk selection algorithms, and attention-anchored positional ordering in agent context windows.
- Allocate token budgets dynamically across system policies, RAG evidence, conversation history, and completion headroom.
- Implement relevance-based knapsack and greedy selection algorithms for bounded context packing.
- Structure context ordering using primacy and recency anchors to counteract Lost in the Middle attention degradation.
History, summaries, and compression
Explains conversation history management, lossy and lossless token compaction, tool trace pruning, and recursive summarization for long-running agents.
- Deconstruct conversation history growth curves and quantify token accumulation across multi-step agent loops.
- Implement lossy vs lossless context compression techniques including sliding window FIFO, tool output compaction, and recursive summarization.
- Formulate structured state extraction protocols that preserve active goals, confirmed variables, and error states across compaction cycles.
Provenance and context debugging
Explains token-level lineage tracking, context debugging inspection consoles, OpenTelemetry telemetry standards, and boundary integrity auditing.
- Track token-level provenance and data lineage across system policies, user inputs, RAG chunks, and tool outputs.
- Implement context debugging inspection consoles to audit token allocation, boundary integrity, and prompt transformations.
- Standardize context telemetry using OpenTelemetry Generative AI semantic conventions and W3C PROV-DM structures.
Reactive and reason-act patterns
Explains reactive action selection and interleaved ReAct (Thought-Action-Observation) loops, error recovery, and cycle bounds in autonomous systems.
- Contrast pure reactive tool calling against interleaved ReAct (Reason-Act-Observe) trajectories in autonomous agents.
- Implement the cyclical ReAct state machine across Thought, Action, Environment Execution, and Observation phases.
- Formulate deterministic loop bounding controls including max iteration limits, repetition traps, and error recovery policies.
Decomposition and plan-execute
Explains multi-step goal decomposition, the two-tier Planner-Executor pattern, subtask lifecycle management, and dynamic replanning protocols.
- Decompose complex high-level goals into directed subtask graphs with explicit dependency ordering.
- Decouple strategic planning (Planner Model) from tool execution (Executor Model) in two-tier agent architectures.
- Track subtask lifecycles across PENDING, IN_PROGRESS, COMPLETED, and FAILED states.
Security & Threat Model Connections (Pass 2 Preview)
Section titled “Security & Threat Model Connections (Pass 2 Preview)”Every child section must link to its matching area under component security.