When an LLM agent does something wrong, you go straight to rewriting the prompt. You guess at what might fix it, redeploy, and hope for the best. There’s a faster way: ask the model why it made that choice. It usually knows.
Here’s what that looks like in practice. Recently, our voice agent at HOAi started hanging up on callers unexpectedly. A caller would give their address, the agent would look it up, and then ask “What can I help you with today?” When the caller said “Thank you”, the agent would reply “You’re welcome, have a great day” and end the call. The caller never got to ask their question.
Find the turn
Every agent session is a sequence of LLM calls. To debug this, start by finding the exact turn where the agent makes the wrong decision. If you’re logging your LLM calls (and you should be), you can pull the full conversation for a session:
select *
from llm_call_log
where chat_id = '3f2b8a1c-9d4e-4f6a-b7c8-0d1e2f3a4b5c'
order by created_at;I inspect the response for each turn and look for the one that doesn’t make sense. In this case, I’m looking for the turn where the agent calls the endCall tool. Once I find it, I copy the full request payload. That payload contains the system prompt, the conversation history up to that point, and the tool definitions. Everything the model saw when it made the decision.
Reproduce the flawed output
Before changing anything, I need to confirm that this input reliably produces the wrong output. I paste the request into OpenAI’s playground at platform.openai.com, set the model to gpt-5.4 and reasoning effort to none (matching the production config), and run it.
The output should match what the agent actually did. If it doesn’t, something about the environment is different and I need to investigate that first.
In this case, the playground output matched the behavior in production. The model called the ask tool with “You’re welcome, Dana. Thank you for calling Oakwood HOA Management. Have a great day.” and called endCall in parallel.
Ask the model why
I didn’t expect this to work the first time I tried it. You can just ask the model why it made that choice, right there in the playground. I asked “Why did you end the call?” and the model explained its reasoning clearly.
It said it interpreted “Thank you” as the caller being finished. The system instructions included a rule to end the call when the caller says thanks, so it triggered the closing sequence. It even recognized the mistake on its own: the caller’s original reason for calling hadn’t been resolved, and it should have asked a follow-up like “What can I help you with today?” instead of ending the call.
Then I pushed further: “What would you change in the prompt to prevent this?”
Fix the prompt
The model pointed to this sentence in the system prompt:
When the caller says goodbye, thanks you and has no more questions, or says “that’s all”, thank them for calling, wish them a good day, and call endCall to end the conversation.
The problem is the word “thanks.” The model read that as “if the caller says thanks, end the call.” Technically the prompt says “thanks you and has no more questions” as a compound condition, but the model wasn’t parsing it that carefully. It saw “thanks” and treated it as a signal to end the call.
The fix:
A caller saying “thank you,” “thanks,” or similar politeness by itself does NOT mean the call is over. Do NOT end the call unless the caller clearly indicates they are finished.
I ran the same request again with this change, and the agent responded differently. Instead of ending the call, it said: “You’re welcome. Before I let you go, I want to make sure we’ve fully taken care of your question. Is that resolved for you?”
That’s the right behavior. The agent stays on the line and checks whether the caller actually got what they needed.
The general pattern
Every agent debugging session I’ve done follows the same three steps. Find the turn that behaves unexpectedly. Reproduce the flawed output in an isolated environment. Iterate on the context until the output improves.
We run this process often enough that we turned it into a Claude Code skill, so anyone on the team can point it at a session and get the diagnosis without doing the steps manually. This works the same way for a voice agent hanging up too early, a coding agent choosing the wrong file to edit, or a support agent hallucinating a policy.
Most of the time, the model did exactly what the prompt told it to do. When the behavior is wrong, the prompt is wrong.
Counterintuitively, the fastest debugger is the model itself. It can read its own context and tell you which instruction it followed, why it interpreted it that way, and how to rewrite it. You don’t have to guess. Just ask.