AI Voice Agents for Post-Visit Follow-up in Telehealth: Ask, Then Hand Over

An AI voice agent for post-visit follow-up in telehealth calls the patient after a virtual visit to ask how they are doing and whether the next steps are clear. It records each answer as the patient said it, and a webhook sends the answers to the patient record. Any symptom or clinical question goes straight to a nurse or clinician.

Key Takeaways

• The care team writes the questions and the red-flag list.
• Record answers as the patient said them and send them by webhook.
• Any symptom goes to a nurse. The agent never interprets it.

This post is part of our guide to AI voice agents for hospitals and health systems. The guide's section on patient calls after a visit opens with this short check-in, the outpatient call that follows a virtual visit, kept apart from the higher-risk call after a hospital stay. This post builds it in Dograh, an open-source, self-hostable voice agent platform, where each answer becomes an extracted variable and lands in the patient record.

What the check-in is for, and what it is not

This call has two questions at its heart: how the patient is doing since the visit, and whether they know what happens next. Next steps might be a lab order, a referral, a prescription to collect or a follow-up booking. A patient who left a video visit unsure about any of them may call back confused, or may not call back at all.

It is an outpatient call. The call after a hospital discharge carries more risk, runs on a longer checklist and has its own post, AI voice agents for post-discharge follow-up. It is also not a survey. Asking a patient to rate the visit belongs in a separate call, covered in AI voice agents for patient surveys in telehealth.

Most virtual visits are behavioral health visits, as the post on AI voice agents for telehealth shows from national claims data. So the check-in has to expect a patient in distress, and it has to route that patient at once.

The care team writes the questions and the red-flag list

The agent asks, and the clinicians decide what it asks. The care team writes a short checklist for each visit type, such as whether the patient has collected the prescription and whether the referral is booked. They also write the red-flag list: the answers that mean a nurse should hear about this patient today. The agent never adds to either list, and it never decides that an answer is fine.

Timing is the clinic's choice too. Many teams call the day after the visit, and others wait two or three days for a new medication to settle in. Your telehealth platform or electronic health record (EHR) starts the call through Dograh's API Trigger once the visit is closed, passing the visit type and a patient reference so the agent uses the right checklist.

A check-in placed with an artificial voice to a mobile number generally has to fit the Federal Communications Commission (FCC) healthcare exemption or rest on the patient's prior consent, and the guide's consent section quotes the rule.

Record each answer as the patient said it

The answers are the point of the call, so they have to land where a clinician will see them. In Dograh, variable extraction records each answer as an extracted variable after the call, from whether the prescription was collected to any new symptom in the patient's own words. Extraction is an AI step and not a word-for-word copy, so the transcript and the recording remain the exact record. Extracted values also cannot feed a later prompt in the same call, so anything the agent must act on during the call belongs in the conversation flow itself.

When the agent is not sure what the patient said, it asks again or hands the call over. It never writes down a guessed symptom. Speech recognition can stumble on an accent or on a quiet, tired voice, and a misheard word about chest pain is not a risk worth taking.

After the call, a webhook node sends the extracted variables, the visit reference and links to the transcript and recording to the patient record. A temporary failure such as a timeout or a server error is retried automatically, up to five attempts. A wrong URL or credential is not, so test the endpoint before the first live call. The tutorial on webhook-driven voice agents covers the payload.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Any symptom goes to a nurse

The agent collects, reminds and routes, and on this call routing is the job that matters most. A new symptom, a side effect, a question about the medication or a worrying answer on the red-flag list goes to a nurse or clinician. The agent does not reassure the patient or interpret the answer. Questions about taking a medication go to a person every time, and the post on AI voice agents for medication adherence covers the calls built around them.

Dograh's Transfer Call hands the patient to the nurse line. It works on Twilio only and passes no summary, so the agent writes the answers to your system before it transfers, and the nurse sees them on screen. The transfer rings for 30 seconds by default, settable from 5 to 120. If nobody answers, the patient stays with the agent, which offers a callback and writes the request to your system through an HTTP API tool. There is no built-in callback task, so write that step into the prompt.

Emergencies skip the queue. If the patient describes chest pain or trouble breathing, the agent tells them to hang up and call 911, and crisis language gets the 988 Suicide and Crisis Lifeline. Both lines are fixed wording once the emergency pathway is chosen, played by a pathway's Transition Speech as fixed text or a recorded clip. The model chooses the pathway by reading its written condition, so keep that phrase list short and explicit, and test it on real calls. Text Transition Speech does not play on speech-to-speech models, so use a standard agent or a recorded clip.

Greet, verify, then name the visit

The greeting comes before verification, so it says who is calling and nothing about the visit. Write the Greeting Text to name the practice and say the patient is speaking with an automated assistant; Dograh plays exactly what you write, word for word at the start. Variables need a fallback, such as {{practice_name | your care team}}, or a missing value is read aloud as a raw placeholder.

Then the agent verifies: date of birth plus one reference, such as the visit date or part of the medical record number. A phone number finds the record. It does not prove who answered. The Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule at 45 CFR 164.514(h) requires a provider to verify the identity of a person requesting protected health information (PHI) when that person is not known to it. Only after that does the agent mention the visit or the plan. If someone other than the patient answers, the agent shares no detail and asks when to call back.

Keep the answers on your own servers

A check-in after a virtual visit holds PHI in every answer, often in a behavioral health context. Self-hosting Dograh runs the platform on your own servers, so the recordings and transcripts stay next to your EHR. No business associate agreement (BAA) with Dograh is needed then, because Dograh never receives the PHI. Each model provider you use through an API key still hears the audio, so each needs its own BAA. The phone carrier may need one too, depending on what it stores. A BAA delegates duties and never transfers your liability. The audio stays inside your infrastructure only when the speech and language models run on your servers as well.

There is no retention setting and no automatic deletion, so your own policy sets how long recordings and transcripts are kept. On a self-hosted server, recording can be switched off for the whole server, and transcripts are still saved. Self-hosted Dograh has no per-minute platform fee, though servers, any paid model keys and phone minutes still cost money. Dograh Cloud has a 1¢-a-minute platform fee.

Start with one visit type and a short checklist. Read the first hundred transcripts with the nurse who owns the red-flag list before you add a second visit type.