From Chatbot to Agent: Building a 50-Line Python AI Agent from Scratch
Prompt & Circuit Editorial
May 14, 2026 · 12 Min Read

Why "agent" is worth defining
The line between a chatbot and an agent sounds fuzzy until you try to build one. Frameworks such as LangChain, LangGraph and CrewAI are useful, but they hide the core mechanism behind abstractions. Writing the loop yourself, in plain Python, forces you to name every moving part.
This article builds a minimal agent in about 50 lines. It talks to OpenAI's API or to a local model served by Ollama, uses two simple tools, and stops itself after a fixed number of steps.
What makes something an agent
A practical checklist:
- It takes a goal, not just a question ("Plan a two-day trip to Seattle").
- It decides the next step itself: answer now, or gather more information first.
- It acts: calls a tool, reads a file, queries an API.
- It observes the result and loops, using what it learned to choose the next step.
A plain chatbot does step 1 and produces text. An agent runs steps 2 to 4 in a loop. That loop is the whole difference, and it is what we will build.
The code
Requirements: Python 3.10 or later and the OpenAI Python SDK (pip install openai). For local mode, install Ollama and pull a model (ollama pull gpt-oss:20b). Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1/, so the same client library works for both backends.
import os
from openai import OpenAI
SYSTEM = """You are an agent that completes tasks step by step.
Available tools:
search: <query> - search the web
read: <path> - read a local text file
To use a tool, reply with exactly one line: TOOL: <name>: <argument>
When you have enough information, reply with the final answer and no TOOL line."""
def search_web(query: str) -> str:
return f"(simulated) Top results for '{query}'" # swap in a real search API
def read_file(path: str) -> str:
with open(path, encoding="utf-8") as f:
return f.read()[:5000] # cap what goes back into the context
TOOLS = {"search": search_web, "read": read_file}
cloud = OpenAI() # reads OPENAI_API_KEY from the environment
local = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama") # key is ignored
def call_llm(messages: list, use_local: bool) -> str:
client, model = (local, "gpt-oss:20b") if use_local else (cloud, "gpt-6-luna")
resp = client.chat.completions.create(model=model, messages=messages)
return (resp.choices[0].message.content or "").strip()
def parse_tool(reply: str):
for line in reply.splitlines():
if line.startswith("TOOL:"):
parts = line.split(":", 2)
if len(parts) == 3:
return parts[1].strip(), parts[2].strip()
return None
def run_agent(task: str, use_local: bool = False, max_steps: int = 8) -> str:
messages = [{"role": "system", "content": SYSTEM},
{"role": "user", "content": task}]
for step in range(1, max_steps + 1):
reply = call_llm(messages, use_local)
messages.append({"role": "assistant", "content": reply})
call = parse_tool(reply)
if call is None:
return reply # no tool requested: this is the final answer
name, arg = call
try:
result = TOOLS[name](arg) if name in TOOLS else f"Unknown tool: {name}"
except Exception as exc:
result = f"Tool error: {exc}"
print(f"[step {step}] {name}({arg!r})")
messages.append({"role": "user", "content": f"Observation from {name}:\n{result}"})
return "Stopped: step limit reached."
if __name__ == "__main__":
task = "Plan a two-day Seattle itinerary with one coffee shop suggestion per day."
print(run_agent(task, use_local=os.getenv("USE_LOCAL") == "1"))
Walkthrough
1. The system prompt defines the protocol
The model can only use tools it knows about, so the system prompt lists them and specifies an exact output format: TOOL: <name>: <argument>. Anything without that line counts as the final answer. This is a deliberately simple text protocol so that every part of the loop is visible.
2. Tools are ordinary functions
search_web is a stub you can replace with a real search API. read_file reads a local file and truncates it, because whatever a tool returns goes back into the model's context and costs tokens. The TOOLS dictionary maps names to functions, so adding a tool is one function and one dictionary entry.
3. One client, two backends
Because Ollama implements the OpenAI Chat Completions API, the only differences between cloud and local mode are the base_url and the model name. Ollama requires an API key value but ignores it. Set USE_LOCAL=1 to run entirely on your own machine.
4. The loop
Each iteration does four things:
- Reason: send the conversation so far and get the model's reply.
- Detect: check whether the reply asks for a tool.
- Act: run the tool, catching errors so a failure becomes information instead of a crash.
- Observe: append the result to the conversation and go around again.
The tool result is sent back as a user message labelled "Observation". The chat API also has a dedicated tool role, but that belongs to native tool calling, where each result must reference the ID of the specific tool call the model made. Since this agent uses a text protocol, there is no call ID to reference, and a plain labelled message is the correct choice.
5. The step limit
max_steps is not optional decoration. Without it, a model that keeps asking for tools, or keeps misformatting its reply, will loop and spend money indefinitely. Every production agent needs a budget: steps, tokens, time, or all three.
Mixing cloud and local models
The design does not care which model answers each step, so routing is easy to add. For example, use the cloud model for the first step, where planning quality matters most, and the local model afterwards:
reply = call_llm(messages, use_local=(step > 1))
Whether that trade-off is worth it depends on how well your local model follows the tool protocol. Smaller models are more likely to misformat tool lines, so test both modes on your own tasks.
From toy to production
The text protocol is great for learning. For real systems, the main upgrades are:
- Native tool calling. OpenAI, Anthropic, Ollama and others let you declare tools as JSON schemas. The model returns structured tool calls instead of free text, which removes most parsing failures. On OpenAI's Responses API, each result is sent back as a
function_call_outputitem whosecall_idmatches the model's call. - Real tools. Replace the search stub with an actual search API, and validate every argument.
- Sandboxing.
read_fileas written will read any file the process can access. Restrict it to an allowed directory before exposing the agent to untrusted input, and treat tool output as untrusted text that may contain prompt injection. - Persistence. Save
messagesafter each step so a run can be resumed or audited. - Guardrails. Require confirmation before tools that spend money, send messages or change data.
- Observability. Log every step, tool call, result and token count.
Once you need several of these, a framework starts to earn its keep. Having built the loop by hand, you will know what the framework is doing for you.
Final thoughts
The defining feature of an agent is not a particular library or model. It is the loop: the model chooses an action, the program executes it, and the result feeds the next decision. Once you have written that loop yourself, frameworks stop being black boxes, and you can judge which of their features you actually need.
Sources
Responses (0)
Sign in to join the conversation and share your thoughts.

Prompt & Circuit Editorial
Contributing Editor
Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.
Continue Reading

Implementing Hybrid Semantic and Lexical Search for RAG with BM25 and Reciprocal Rank Fusion
Dense embeddings miss exact identifiers and BM25 misses paraphrases. This walk-through combines both in Python with rank_bm25, sentence-transformers and Reciprocal Rank Fusion, and explains the trade-offs before you move it to a search engine.

Demystifying LangChain Models: From LLMs to Embeddings in Plain English
A plain-English guide to LangChain's model layer: chat models, legacy text-completion LLMs, embedding models, invoke(), key parameters, and how embeddings power semantic search, with code for the current LangChain 1.x APIs.

Building Human-in-the-Loop Agentic Workflows with LangGraph
How to pause a LangGraph workflow for human approval with interrupt(), resume it with Command, and design review steps that hold up in production.