When Customers' AI Agents Call Your Business: What to Allow

A customer's AI assistant calling your business is a third party acting for someone. Let it ask about hours, prices, stock and open slots. Keep account changes, refunds, payments and personal details for the verified customer. Dograh, the open-source AI voice agent platform, can route these calls, and you can self-host it or run it on Dograh Cloud.

Key Takeaways

• Voice no longer tells you who is calling. Ask who the assistant acts for.
• Public answers and bookings suit AI callers. Account changes need the real customer.
• A phone number finds a record. It never proves who is calling.

Your next caller may be software booking a table for someone who never picked up the phone. We cover who already places these calls, how to tell them from people, what to let them do and what to hold back, then how Dograh handles them in both directions.

Who already sends AI to make calls

AI assistants that phone businesses for their users are live products, and they reach ordinary front desks. With Let Google Call, a shopper searching for a toy or a charger nearby can ask Google to ring local stores and get back whether the item is in stock and what it costs, promotions included, by text or email. Google's page for business owners lists the other calls it places for customers, such as booking appointments and checking wait times, prices and availability. It also says these calls are monitored and recorded for quality assurance.

Google is not the only one doing this agentic calling. Gemini calls businesses and works through phone menus for its user, and Meta's Muse places calls to US businesses on a user's behalf. Pindrop's article on AI agents calling banks reports that about 1 in 80 calls in its bank call data came from AI agents or bots, and that 94% of the agent callers acted legitimately while 6% were malicious. Pindrop also saw Muse's weekly call volume grow eightfold among its customers within a few weeks.

You can turn Google's calls off in your Business Profile's advanced settings, or by asking on the call. Most businesses will keep them, though, because a booking made by an assistant is still a booking.

How to tell an AI caller from a person

The surest sign is the one the assistant gives you itself. Pindrop notes that Gemini opens each call by saying it is an AI assistant, and that its user can take over the call. You cannot count on that from every caller, because a good synthetic voice now sounds like anyone. In Pindrop's words, "Does it sound human?" is simply not a useful test anymore.

So ask, and then listen to what the caller does. Train your staff, and your own voice agent, to ask one question early: who are you calling for, and what have they asked you to do? A legitimate assistant answers plainly, with the customer's name and the task. Behaviour tells you more than the voice does. Software tends to repeat a request in the same words, answers with no pause at all, steps through a phone menu option by option and pushes back on identity questions instead of answering them.

None of these signs is proof, and none needs to be, because what the caller asks for decides how much it may do.

What to let an AI caller do

Most of what assistants ask for is safe to give anyone who rings. Opening hours, today's wait, whether an item is in stock, the price of a service and the next open slot are public or close to it. Pindrop's advice to banks draws the same line: a known agent making a routine read-only request, such as a balance check, can go to an automated channel, while requests that could change an account need more verification.

Bookings sit in the middle. An assistant booking a haircut or a table is doing exactly what its user asked, and turning it away sends that customer somewhere else. Take the booking, read the details back, and send the confirmation to the customer's own phone number or email, so the real person sees it.

Banks are drawing their own lines. JPMorgan Chase stops outside AI agents from logging in to retail accounts with customers' passwords, and lets them read data such as balances only through a connector like Plaid, with no money moved and no bills paid, as CNBC reported. That rule covers online logins rather than phone calls, but the split is the one a phone line needs too: reading is one risk, changing is another.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

What still needs the real customer

Anything that changes an account, moves money or shares personal details needs the customer, verified, on the line. The order we use for every voice agent holds here: greet, verify, then disclose. A phone number finds a record, but it never proves who is calling, and an assistant ringing from its user's number proves even less. Rodolfo Raimundo, a research scientist at Pindrop, put it plainly: "Checking a balance and changing a payout address carry very different risks."

The awkward case is an assistant that knows the answers. If a customer has given an app their date of birth and account number, the app can pass a knowledge check built for people. So for the risky requests, such as an address change, new payout details, a refund or health information, use a step-up check: ask for the customer to come on the line or call back, and treat a caller that refuses as a reason to stop. Our post on voice cloning fraud and AI agents covers the case where the voice itself may be fake.

Health providers have a rule that already fits. Under the Health Insurance Portability and Accountability Act (HIPAA), a provider may share health information with a family member, friend or anyone the patient names, limited to what matters to that person's part in the patient's care, and when the patient is present, only if they agree or do not object. Our reading is that an AI assistant calling for a patient is a third party under that rule, so it gets no more than the rule allows unless the patient confirms on the line. Our post on verifying a patient's identity before an agent shares health details shows that check.

Disclosure is a separate duty. The European Union's Artificial Intelligence Act (EU AI Act) requires whoever provides an AI system that talks directly with people to make sure they know it is an AI, unless that is obvious, so for an assistant calling you the duty generally sits with the assistant's maker. Our post on what a voice agent must tell callers under the EU AI Act covers when it applies, and you should confirm with counsel how both rules fit your calls before you write your policy.

Setting up your voice agent for AI callers

Your own voice agent can follow the same policy. In Dograh, add a pathway out of the opening step with a condition such as "the caller says it is an AI assistant acting for someone else", and point it to an agent step whose prompt allows only public answers and bookings. Choosing that pathway is the language model reading the condition you wrote, so treat it as the model's judgement and keep the account rules in every step's prompt as well.

With Enable Variable Extraction on that step, the agent notes the caller type and the task as the caller stated them, while the transcript and recording stay as the exact record. The QA node scores and tags each step after the call, and View Trace opens a call's full trace, so you can see how the agent handled each assistant. Assistants can also carry instructions aimed at your agent, as our post on prompt injection in voice agents explains.

When a risky request comes in, hand the call to a person. The Transfer Call tool works on Twilio, Telnyx, Plivo, Cloudonix and Asterisk ARI. Dograh dials your staff member while the caller hears hold music, and connects the call once that person answers. If the line rings out, is busy, fails or times out, the agent gets transfer_failed and the call goes on, so write the prompt to offer the real customer a callback in that case.

When your agent calls and an assistant answers

The same shift runs the other way, because phones now screen calls for their owners. Once you switch on "Enable voicemail and screening handling" in your agent's settings, which is off by default, Dograh sorts what answers an outbound call into voicemail, a phone menu or a call screener. For a screener, the Screening message states your name and reason for calling, then the agent waits silently for the person, 30 seconds by default. Leave the message blank and screened calls end there.

Keep that message to one sentence an assistant can repeat to its owner. If you tell people they're speaking with an automated assistant, by choice or because local law requires it, put that line in the Greeting Text next to the business's name. Then switch off Allow Greeting Interruption on the Start Call node, so an assistant that talks over the start still hears the whole notice.

Write your AI caller policy now

The easiest time to set the rules is before the volume arrives. Write one page that says what an assistant may do on its own and what needs the customer on the line, and name the person who takes the unclear calls. Pindrop also suggests telling customers in advance what agents can do by phone. Put the page into staff training and your agent's prompts, and read a sample of assistant calls each week.