Keeping LLM Answers Short Enough for the Phone: Rules and Tests

Keep a phone agent's replies to two or three short sentences with one question per turn, written as a rule in the global prompt and tightened or relaxed on single steps. Then test it on transcripts, because models keep rules differently. Dograh, the open-source AI voice agent platform, runs both, self-hosted or on Dograh Cloud.

Key Takeaways

• Put the length rule in the global prompt, with per-step exceptions.
• A token cap cuts replies mid-sentence, so the prompt sets length.
• Check reply length on transcripts for every model you try.

This post is part of our guide to how to build an AI voice agent, where the Global Node holds the rules every step shares. Below, we show what a long reply costs, write a spoken length rule, set limits per step and test that each model keeps it.

What a long answer costs on a call

A reply that reads fine on a screen takes a long time to say out loud, and the caller has to sit through all of it. In the example call our LLM API pricing guide uses, the agent says about 37 words a turn, which is about 15 seconds of speech. Twelve agent turns take 180 seconds of a 5-minute call.

Here is a typical reply, as an example: "Sure, I can help with that. I see your order from Tuesday, and it left our warehouse this morning. It should reach you by Friday afternoon. Would you like me to send the tracking link to your phone?" That is 38 words in four sentences.

The useful part is the delivery day, and the question the caller has to answer comes last. Callers often start talking once they have what they need, so the agent gets interrupted before its question, or the caller sits through two sentences they did not ask for. Long replies also cost more. Every extra word is voice time and output tokens, and it is sent back to the model again on every later turn.

Write the length rule into the prompt

The rule works best as a plain instruction in the prompt every step shares. In Dograh that is the Global Node, whose prompt is added to every node that has Add Global Prompt switched on. Dograh's own prompting guide, which the MCP builder follows when it drafts an agent, writes this rule in as "Keep responses short, 2-3 sentences max" and asks the agent to end almost every turn with a question or a nudge.

Give the model a count it can follow and the one exception you expect. The lines below set a sentence limit, put the question last, and allow longer read-backs where the caller needs every detail:

Speak in one or two short sentences. Ask one question per turn, at the end.
If the caller asks for detail, give the most useful part first, then ask if they want more.
When you read back a booking or an address, you may use up to three sentences.

What this does: It caps ordinary turns at two sentences and makes the question the last thing the caller hears. The third line stops the length rule from cutting a read-back short, which is the one place a longer turn helps the caller.

Watch for an instruction collision, two rules that pull against each other. Dograh's prompting guide gives an example: "Be conversational and empathize deeply" in one place and "Keep responses under 10 words" in another. A model cannot do both, so it picks a different side on different turns. Read the global prompt and every node prompt together, and keep one length rule.

Looser and tighter limits for single steps

One length does not fit every step, so set exceptions in the node prompt for that step. A step that confirms a booking may need three sentences to read back the date, time and place. A step that only asks for a postcode can be held to a few words. Write the exception in the node, and leave the global rule as it is.

Read-backs have their own rules on top of length, such as reading numbers in groups and spelling names back. Our post on reading back names, numbers and emails covers those.

Some lines should never vary at all. The Greeting Text on the Start Call node and a pathway's Transition Speech are fixed text, spoken word for word. A pre-recorded clip goes further, because it is the only line whose length and wording are guaranteed on every call. Put your opening, your disclosures and any line you say on every call there, and let the model write only the parts that change.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Why a token cap is the wrong control

A limit on output tokens looks like a direct way to keep replies short, but it stops the model where the count runs out. OpenAI's Chat Completions reference describes max_completion_tokens as an upper bound on the tokens a reply can generate. The model does not plan around that limit, so a long reply ends in the middle of a sentence, and the caller hears half a delivery date.

A softer control works with the model. The same reference lists a verbosity setting for GPT-5 models, with values low, medium and high, medium by default, where lower values give more concise replies. Dograh sends low verbosity and minimal reasoning to every GPT-5 model automatically.

For every other model in Dograh, reply length comes from the prompt. That keeps the rule in plain words you can read, test and change without touching the model settings. Shorter sentences also need to be easy to hear, with numbers, dates and prices written as words, which our guide on how to reduce robotic TTS covers.

Test that each model keeps the rule

A length rule is a request, and models honour requests differently, so check it on every model you try. On aiewf-eval, a public 30-turn voice benchmark, the share of turns where a model broke a prompt instruction ranges from 0.0% for Claude Sonnet 5.5 and Gemini 3.7 Flash at low reasoning to 18.8% for gpt-5-mini. Those checks cover instructions in general, not length, but they show how far models drift. Our comparison of GPT vs Claude vs Gemini for phone agents reads those results in full.

Run each step several times, not once. A language model can give a different reply to the same prompt and the same caller words, so one good test call proves little. Then place real phone calls, because callers interrupt and ramble in ways a typed test does not.

Let QA mark the long turns for you. The QA node reviews each finished call with a System Prompt you can edit, and it scores each step of the call separately. Add a tag of your own to that prompt, such as REPLY_TOO_LONG for any agent turn over your sentence limit, and the tag shows which step runs long.

For an exact number, export the transcripts and count the words in each agent turn. Track the average and the longest turn per step, and change the node prompt for the steps that run over.