First Edition · 2026

Building Agentic AI From Goals to Autonomous Systems

The theory and practice of autonomous AI: from goal-directed reasoning and planning to multi-agent coordination and real-world deployment.

Alexander (Sasha) Apartsin, Ph.D. & Yehudit Aperstein, Ph.D.

An agent is a system that perceives its environment, decides what to do, acts, and learns from the outcome. That loop runs whether the policy is a planning algorithm, a learned reward function, or a language model. Where Building Language AI asks how models work, this book asks what they do: pursue goals, use tools, collaborate with other agents, and operate autonomously over long horizons. The treatment is complete: decision theory and classical planning first (the mathematics every agent is built on), then reinforcement learning and deep RL (agents that improve from reward alone), then LLM-powered agents and agentic RAG, and finally multi-agent systems, embodied deployment, safety, and governance. Each idea is built from first principles and shown in production code.

4 parts 39 chapters 5 appendices & a capstone First Edition, 2026

The Four-Part Arc

Each part stands on the one before it; together they span the full spectrum from a single reactive loop to autonomous organizations.

Foundations of Agency

The conceptual and mathematical bedrock: what agents are, how they perceive and act, how they represent preferences as utility, how they plan toward goals, and how they reason with knowledge and uncertainty.

7 chapters II

Learning Agents

Agents that improve from experience: Markov decision processes, tabular and deep reinforcement learning, model-based agents, hierarchical control, memory systems, and lifelong and meta-learning.

9 chapters III

LLM-Powered Agents

Language models as policies: prompt engineering, reasoning via chain-of-thought and extended thinking, tool use and MCP, memory-augmented and agentic RAG, autonomous task agents, computer use, and self-improvement.

11 chapters IV

Multi-Agent and Deployed Systems

Agents working together and at scale: multi-agent systems, MARL, swarm intelligence, software engineering agents, research agents, observability, cost engineering, embodied agents, safety, and governance.

12 chapters

How This Book Teaches

Five habits, kept in every chapter from the first agent loop to the last autonomous system.

Worked Pipelines

Every chapter builds complete, runnable systems (a grid-world agent, a PPO policy, a multi-tool LLM agent end to end), never isolated snippets.

Library Shortcuts

After each from-scratch build, a shortcut callout shows the same task in a few lines of LangGraph, Stable-Baselines3, or the Anthropic SDK, and names exactly what the library handles for you.

A Callout System

Pitfalls, math asides, benchmark tables, and cross-references are typeset as distinct boxes, so you can read deep or skim fast and never miss a trap.

Exercises & Labs

Each chapter closes with hands-on exercises that extend its worked pipelines, from quick checks to small projects you can put in a portfolio.

Classical Ideas Return Learned

The utility function becomes the reward signal, the search tree becomes chain-of-thought, the knowledge base becomes agentic RAG. One story, told twice.

The Hands-On AI Science Series

Building Agentic AI is one of nine connected books, each a deep, build-it-yourself guide to a major field of AI.

Hands-On AI Science is a series of in-depth guides to the major fields of artificial intelligence. Every book goes deep into the theory, models, and internals, covering the classical foundations and the most recent ideas, then shows you how to build each one in Python with the modern libraries and tools that get the job done. The writing stays plain and light (illustrations, analogies, mental models, worked examples, and a little fun) without trading away rigor or coverage. Each volume is self-contained and complete enough to anchor a full course on its subject.