I Built a Coding Agent and Realized Calling the Model Is Just the Beginning

Before Building a Coding Agent, My Understanding of Agents Was Simple

Before building a coding agent, my understanding of agents was simple.

Give the LLM a goal, then add prompts and tool calling. The user says, "Help me fix this bug," the model analyzes the code, calls tools, modifies files, and runs tests. As long as this chain works, an agent seems done.

After actually building one, I realized that calling the model is easy, but taking a task from start to finish is hard.

The model knows how to solve the problem, yet the task still stalls midway. It might complete the first few steps but not know where it is. After a tool call fails, the state doesn't reconnect. As the context grows, earlier decisions get overwritten. The scope of changes expands, and the final code even goes beyond what the user originally wanted to change.

These problems cannot be automatically solved by switching to a stronger model. Model capability matters, of course, but it only handles one part. The rest has to be handled by the software system.

I Initially Focused on the Model

When I started building CodeLeveler, I spent a lot of time comparing models.

What model to use, how large the context is, how good the reasoning is, how well it supports tool calling.

At the time, I assumed that the stronger the model, the stronger the agent.

This judgment often holds for short tasks. The user asks a question, the model gives an answer, and the process ends quickly. With coding agents, tasks get longer, and model capability is no longer the only constraint.

An agent needs to do many things continuously. It has to understand the goal, make a plan, call tools, read results, modify code, and run tests. If any step goes wrong, subsequent actions may lack a basis.

The model knowing what to do next doesn't mean the system remembers what has already been done.

The Agent Starts to Resemble a Runtime

Later, when redesigning CodeLeveler, I gradually came to see it as a runtime.

When the user says, "Help me optimize this project," that sentence contains at least several types of information. What problem does the user actually want to solve, what is the current plan, which steps are completed, which steps haven't started, and what files and results have been produced along the way.

If this content only exists in the current context, every time the agent makes an error, it's like returning to the starting point. It re-reads the requirements, re-makes judgments, and may even repeatedly modify files it has already changed.

State management thus became the first thing that needed serious attention.

Tool calling is also not as simple as the model sending a function request and waiting for the result. The real process usually involves the system first checking permissions, then executing the tool, recording the result, compressing the context if necessary, and finally deciding whether to continue planning.

The model's call is just one action in the process.

Another easily overlooked part is observability.

When a traditional program has a problem, engineers look at logs. When an agent has a problem, we also want to know why it chose this plan, why it called this tool, why it stopped here, and whether it tried to recover after failure.

This information needs to be recorded. Event Log, Trace, Tool History, Task Timeline—these are not to make the system look more complex; they determine whether developers can find problems and whether users can understand what actually happened in a task.

At this point, the agent increasingly resembles a software system that needs to run for a long time.

Chat Is Just the Entry Point

This also changed my view of the product form of coding agents.

The chat interface is just the entry point; the execution process behind it is the key.

The user proposes a goal, the system breaks down the task, the agent enters the execution loop, tools handle operations, state updates continuously, and results are validated. After validation fails, it continues to correct or hands the problem back to the user.

The quality of a chat usually depends on a single response. The quality of a task depends on whether it can keep progressing, recover after failure, allow the user to take over midway, and finally deliver a validated result.

The difference between these two products only becomes apparent when you actually work on long tasks.

Why Demos Are Impressive but Hard to Use Long-Term

Many agent demos show the first success.

The user gives a goal, the model completes a call, and a beautiful result appears on the screen. The whole process ends in a few minutes and looks smooth.

Real work often lasts several hours, modifies dozens of files, handles various errors along the way, and needs to understand previous historical context. The user may insert new requirements at any time, tools may return exceptions, and a certain step's judgment may need to be rechecked.

At this point, stability amplifies the gap in intelligence.

An agent with only eighty percent capability but that can run continuously and know where it is, is usually more valuable than an agent with one hundred percent capability but that often forgets state, repeats operations, and loses control midway.

Because what users truly need is often not a one-off impressive answer, but a task that can be reliably completed.

I Now Ask a Different Set of Questions

Previously, when building software, I focused more on whether the functionality was completed.

Now, when building agents, I continue to ask several things.

Can it recover after a task fails? Can it continue after the user pauses? Are tool results correctly recorded? As the context grows, can the system still retain important information? Is the final result validated?

These questions seem unrelated to model parameters, but they determine whether users can use the product long-term.

After building CodeLeveler, my understanding of software also changed.

In the past, software usually required people to write out the operation steps in advance. People told the machine how to do things, and the machine executed according to the rules.

In the agent era, users are more often describing what goal they want to achieve. The planning, operations, checks, and corrections after the goal require another software system to complete.

What I care most about now is no longer how to make the model say one more beautiful sentence, but how to make it forget fewer steps in complex tasks, lose control less often, and finally get things done.