Dograh

Groq vs Cerebras for Voice Agent LLMs: What the Numbers Show

Groq vs Cerebras for Voice Agent LLMs: What the Numbers Show
ComparisonNovember 17, 2026·9 min read

Groq vs Cerebras for Voice Agent LLMs: What the Numbers Show

Vemu Sandeep
Vemu Sandeep·GTM Engineer, Dograh AI

GTM engineer at Dograh working across voice AI and enterprise AI. IIT (BHU) and IIM Visakhapatnam alumnus.

On short, voice-sized prompts, Groq and Cerebras return gpt-oss-120b's first token in about the same time. Cerebras reaches the first answer token sooner and streams faster, and Groq charges less per token. For a platform your team owns, Dograh is the open-source AI voice agent platform to self-host or run on Dograh Cloud.

Key Takeaways

  • On 1k-token prompts, both hosts return gpt-oss-120b's first token in about 0.2 seconds.
  • At low reasoning, Cerebras delivered the first answer token in about half Groq's time.
  • Cerebras labels its self-serve Developer tier as not intended for production use.

Groq and Cerebras both serve open-weight models fast enough for live phone calls, and both host gpt-oss-120b. We compare first-token time, reasoning settings, tool calls, price, limits and production terms, show how to measure each on your own phone calls, then where open-source Dograh fits.

Groq and Cerebras in one table

Both hosts run the same open-weight model behind an OpenAI-compatible API, so the differences come from the hardware, the settings and the terms. Benchmark rows are Artificial Analysis's November 2026 measurements for gpt-oss-120b.

GroqCerebras
HardwareLPU inference chipsWafer-scale inference chips
Price per 1M tokens, gpt-oss-120b$0.15 input, $0.60 output$0.35 input, $0.75 output
First token, 1k prompt, high reasoning0.205 s0.208 s
Output speed, 1k prompt, high reasoning462 tokens a second1,844 tokens a second
First answer token, low reasoning, mixed prompt sizes1.26 s0.65 s
Paid self-serve limit on gpt-oss-120b1,000 requests and 250,000 tokens a minute1,000 requests and 500,000 uncached tokens a minute
Self-serve termsFree and Developer tiers, with Llama models now on EnterpriseDeveloper tier "not intended for production use", Enterprise for production

Why first-token time matters on a call

A voice agent does not wait for the whole reply. The voice engine starts speaking as soon as the first sentence streams in, so the caller's wait depends on how fast the first useful words arrive. Speed after that matters much less on a phone call than in a chat window. A spoken reply is often around 60 tokens, which streams in about 0.13 seconds at Groq's 462 tokens a second and about 0.03 seconds at Cerebras's 1,844.

That is why a Groq vs Cerebras post on DEV Community argues that either host makes the language model step fast enough for most voice apps. On first-token time at short prompts, the numbers back that up. The real gap shows up elsewhere, in how long a reasoning model thinks before it answers, how strictly each host checks tool calls and what each one allows in production.

First token on a voice-sized prompt

Artificial Analysis's gpt-oss-120b provider page measures every host on the same workloads, averaged over five days and re-run every six hours. Its 1k-input workload is the closest match to a voice turn, a system prompt plus a short conversation.

On that workload, in Artificial Analysis's November 2026 measurements at high reasoning, Groq returned the first token in 0.205 seconds and Cerebras in 0.208 seconds. That is a tie for any caller. On the multi-turn workload, which replays a growing conversation, Groq measured 0.304 seconds and Cerebras 0.276 seconds, again too close to hear.

These figures are measured from Artificial Analysis's servers, not from yours. The network trip from your platform to the host's region adds its own time on every turn, which is why the region you run in matters as much as the host you pick.

Reasoning settings change the result

gpt-oss-120b is a reasoning model. Before it answers, it can produce thinking tokens the caller never hears, so the first token out may be a thought, not a word. Artificial Analysis therefore also reports a first answer token, which includes the thinking time, and that is the figure a caller actually feels.

Here the hosts split. At high reasoning on Artificial Analysis's headline mix of 1k, 10k and 100k prompts, its November 2026 measurements put Groq's first answer token at 6.94 seconds and Cerebras's at 2.10 seconds, both far too slow for a live call. At low reasoning on the same mix, Groq measured 1.26 seconds and Cerebras 0.65 seconds. The headline mix includes prompts much longer than most voice turns, so treat these as a ranking, not as what your caller will hear.

The practical rule is to set the reasoning level yourself. Groq's API reference gives gpt-oss a default reasoning effort of medium, so an agent that never sets it may think longer than a phone call allows. For most call flows, low reasoning is the setting to test first.

Dograh

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Tool calls, errors and accuracy

Voice agents call tools to book, look up and transfer, so a host that rejects a tool call breaks the conversation. Artificial Analysis notes that on tool use, about 7% of tasks fail on Groq because its API rejects function calls whose arguments do not exactly match the tool's schema, and that retries do not help. Tight, simple tool schemas reduce that risk, and the only way to know is to run your own tools.

A caller asks the agent to book. The agent's tool call, drawn as a puzzle piece, does not match the gap cut for the tool's schema on a host's board, so it is thrown back and retries do not help. On a second host's board, a tight, simple schema takes the piece.
Tight, simple tool schemas reduce that risk, and the only way to know is to run your own tools. Model choice still matters more than host choice for following instructions.

On the same page, Groq's endpoint scored about 90% of the reference accuracy for gpt-oss-120b and Cerebras about 92%. Groq's error rate over the period was under 2%, and Cerebras's was zero. On Cerebras, gpt-oss-120b does not accept tools and a structured response format in the same request, which matters if your agent asks for JSON while also calling tools.

Model choice still matters more than host choice for following instructions. Our GPT vs Claude vs Gemini benchmark shows how hosted models compare on a public 30-turn phone script.

What one call costs on each

Groq's model list prices openai/gpt-oss-120b at $0.15 per million input tokens and $0.60 per million output tokens. On its Developer tier, Cerebras prices gpt-oss-120b at $0.35 and $0.75.

Take an example 5-minute call with about 10 language model turns, sending 20,000 input tokens and getting 600 output tokens back. On Groq that comes to about $0.0034 a call. On Cerebras it is about $0.0075. Over 1,000 calls, that is about $3.36 against $7.45.

Both figures are small next to the voice engine and the phone minutes on the same call. Our guide to LLM API pricing walks through the turn-by-turn method if your prompts are longer or your calls run past five minutes.

Join the Dograh Community

Dograh is an OSS alternative to Vapi. Join our Slack community for queries, releases, best practices & community interactions.

Rate limits and production terms

Limits decide how many calls you can run at once. Groq's rate limits for gpt-oss-120b allow 30 requests and 8,000 tokens a minute on the free tier, and 1,000 requests and 250,000 tokens a minute on Developer. Limits apply per organization, cached tokens do not count, and a 429 response comes with a retry-after header.

Cerebras's rate limits for gpt-oss-120b allow 5 requests and 30,000 uncached tokens a minute on the free trial, and 1,000 requests, 500,000 uncached tokens and 1.5 million total tokens a minute on Developer. The context window is 65,000 tokens on free and 131,000 on paid plans.

As a rough guide, a call that sends 2,000 tokens a turn and takes two turns a minute uses about 4,000 tokens a minute. That puts Groq's Developer limit at around 60 calls at once and Cerebras's at around 120, before prompt caching helps either one.

Terms differ too. Cerebras describes its Developer tier as "For development, evaluation, and experimentation; not intended for production use", with production workloads on Enterprise. Groq retired Llama 3.3 70B and Llama 3.1 8B on its free and developer tiers on 16 August 2026 and points users to openai/gpt-oss-120b or openai/gpt-oss-20b.

How to measure it on your own calls

Published numbers rank the hosts, but your prompt, your tools and your region decide what callers hear. In Dograh, every agent reply in a test call's Call Transcript shows a Reasoning Delay, the language model's time to first token for that turn. Run the same agent twice, switching only the host, and compare the delays turn by turn. Keep in mind that Reasoning Delay covers one part of the turn, so never set it against an end-to-end latency budget.

For the full picture, View Trace opens the call in Langfuse with the speech-to-text, AI and voice time on each turn. Our guide to reading call traces shows how to find the slow layer, and our post on server colocation covers the network time between your platform and each host.

Outside any platform, stream a fixed request to each OpenAI-compatible endpoint with your real system prompt and tools, set the reasoning effort you will run in production, and time the first content chunk rather than the first reasoning chunk. Then run your expected number of calls at once and watch the rate-limit headers and any 429 responses.

Which one fits your voice agent

Groq fits teams that want the smaller per-token rate of the two on gpt-oss-120b and room to grow on its paid self-serve tier, as long as their tool schemas pass its strict checks in testing. Cerebras fits teams that want the fastest first answer at low reasoning and the fastest streaming, and that will move to its Enterprise tier for production. Test both with your own prompt and tools, because a tie on paper can split on your workload.

Where Dograh fits

Dograh's part is orchestration, running the call flow, tools and telephony, while the speech and language models are a separate layer of your choosing. Each agent can use its own speech-to-text, LLM and voice provider, through your keys or open models you host yourself. For Groq, choose openai/gpt-oss-120b in the Groq provider. Cerebras comes in through the Hugging Face or OpenRouter provider, or an OpenAI-compatible base URL.

The OpenRouter provider takes a provider order, so you can pin a low-latency host instead of OpenRouter's default price-weighted routing. With your own keys, you can also turn on an LLM fallback, a backup model that takes over if the main one errors or gives no output within a set time, 1.5 seconds by default.

Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee. On a self-hosted install your team looks after the server, the database and the upgrades, and support comes from the community on the open-source edition or through custom SLAs on Enterprise.

Glossary

Time to first token
How long a model host takes to send back the first token of a reply. On a reasoning model that token may be a hidden thought, not a word.
First answer token
The first token of the reply the caller will actually hear, measured after any thinking. It is the better guide for a voice agent.
Reasoning effort
A setting that controls how long a reasoning model thinks before answering. Groq defaults gpt-oss to medium, and low is the usual starting point for calls.
Rate limit
The most requests and tokens a host accepts from your organization in a minute. Divide the token limit by tokens per call per minute to estimate calls at once.

Frequently Asked Questions

Get started with Dograh

Build, deploy, and scale AI agents with Dograh. Join the community of developers building the future.