June 18, 2026 | 5 min read

Vivek Muppalla,
VP of Engineering,
Hippocratic AI
in

Sathvik Perkari,
Software Engineer,
Hippocratic AI
in

Dishank Jhaveri,
Software Engineer,
Hippocratic AI
in
In Part 1, we walked through what makes healthcare voice AI hard: the latency budget, turn-taking, bandwidth asymmetry, and the safety stakes that change everything. This post is about how we engineer against them. Polaris is our voice AI system for healthcare, built on what we call a constellation architecture — a primary conversational agent backed by specialized supervisor models, all coordinated inside the real-time budget. To date, Polaris has handled over 180 million patient interactions at 8.95/10 patient satisfaction. The rest of this post walks through the architecture, where we’re pushing the frontier, and why we’re hiring
Early on, single-model prototypes plateaued around ~80% accuracy on clinical questions: impressive for general AI, unacceptable for healthcare. The failure mode wasn’t lack of knowledge. It was inconsistency. Missing a contraindication on one turn, failing to escalate risk on another, reasoning that wouldn’t hold steady across a multi-turn conversation. Bigger model, same problem.
What worked was treating safety as a system, not a single model. A primary conversational agent runs in the live loop, optimized for real-time, coherent dialogue. Specialized supervisor models run in parallel, each tuned narrowly to one job: medication safety, escalation logic, policy constraints, privacy, clinical consistency. We call this the constellation pattern, and at Polaris 5.0 scale it reaches 99.89% clinical accuracy, tested by over 7,500 clinicians on over 700K calls.
The design choice that matters most is what sits in the critical path and what doesn’t. Some supervisors are hard gates that block unsafe outputs synchronously, like incorrect medication guidance. Others monitor asynchronously and intervene on subsequent turns. That two-tier structure is how you keep adding safety coverage without paying for it in latency.
Coordination is the hard part. Multiple models evaluating the same interaction under tight time budgets, reconciling disagreements, converging on a single response without introducing lag or instability. Orchestration under those constraints is its own technical problem. It’s also why the frontier engineering challenges below matter.
Where We’re Pushing the Frontier
Running trillion-parameter-scale systems at conversational latency demands serious engineering: FP8/quantization, KV-cache optimization, continuous batching, paged attention, tensor parallelism, kernel-level tuning, and aggressive caching. Staying at the frontier here is itself a competitive advantage and it’s work we invest in deeply.
But the infrastructure is table stakes. The harder problems sit on top:
Efficient reasoning under a real-time budget.
In voice, you can’t afford verbose “thinking tokens.” You need models that reason well without thinking slowly. That’s why there’s growing research on training models to reason more efficiently e.g., compressing or restructuring reasoning to reduce latency.
Semantic turn-taking at scale.
Our production turn-taking, described earlier, already moves past pure VAD. The frontier is doing this perfectly across accents, emotional states, and clinical contexts at conversational speed. “When to speak” remains one of the hardest parts of natural dialogue.
Memory architectures for voice.
In Polaris, memory is hierarchical by design. Voice is low-bandwidth and ephemeral, so the system must continuously decide what to retain, what to compress, and what to discard. Get this wrong and you fail in one of two ways: forget a clinically critical detail, or drown the model in irrelevant history. In healthcare, that’s not a UX bug. It’s a safety risk. The frontier is compressing the right context without losing the details that change clinical decisions. The right memory hierarchy is the work: tiered KV-cache, aggressive summarization, selective retention.
Speculative execution for dialogue.
If you wait until the user finishes speaking to start thinking, you lose. We speculate throughout the pipeline: drafting likely continuations during user speech, verifying once intent locks in. The frontier is going more aggressive on speculation without sacrificing correctness.
Multi-agent coordination at real-time latency.
The constellation pattern is described above. The frontier is pushing it further: more supervisors, deeper checks, richer cross-model coordination, all without giving up the latency budget. Every supervisor added is one more model to coordinate inside the conversational window.
Adversarial multi-turn safety.
Healthcare isn’t just “be accurate.” It’s “be robust over many turns,” including cases where intent is unclear or concealed. Multi-turn interactions create new failure modes: prompt injection, gradual jailbreaks, safety degradation over time. Our work like RED QUEEN highlights how multi-turn, intent-concealing interaction can expose risks that single-turn safety checks miss. We test multi-turn adversarial behavior continuously, not just at release.
Multilingual care at real scale.
Real deployment means accents, code-switching, non-English preferences, cultural variation, and equity constraints. This isn’t a nice-to-have; it’s part of clinical quality. Our agents ship across several languages today, and the frontier is delivering clinical-grade accuracy across accents, code-switching, and cultural register at production scale – and working to bring more languages online at clinical level.
Audio-native reasoning.
Most systems still do speech → text → reasoning → text → speech. That’s a pipeline, not a protocol. The frontier is models that can reason with audio itself: prosody, pacing, hesitation, stress signals, without flattening everything into transcription. Benchmarks are starting to evaluate this, but the capability is still early.
And even that is incomplete. The real challenge is operational: shipping a probabilistic system into complicated real-world healthcare, measuring safety continuously, catching regressions fast, and preserving a “do no harm” posture day after day, not just in benchmarks but in production. This is the Hippocratic AI difference – addressing this problem in real time.
Healthcare is bottlenecked by conversation and time with patients. There’s a nursing shortage, physician burnout, and millions of patients who can’t get timely access to care. Chronic conditions go unmanaged not because we lack treatments, but because the system doesn’t have bandwidth for the conversations that keep people on track.
We think about the next wave of AI in three steps: co-pilots (≈1.1× productivity), autopilots (≈10× automation), and infinite-pilots, systems that unlock work we never attempted because the staffing math and economics made it impossible. That’s healthcare abundance in practice: infinite surge capacity, infinite patience, and relentless follow-up, applied safely to patient outcomes, not hype. Our CEO Munjal Shah (2026) has written more on this framework.
Voice AI won’t replace healthcare professionals. But it can extend their reach: handle routine conversations that consume hours, catch details that slip through when a nurse has 30 patients instead of 5, and be available at 3 AM when symptoms get scary and the on-call line has a 45-minute hold time.
We’re seeing what happens when human-quality conversation becomes scalable. Zero harm incidents to date. In Polaris 5.0 deployments, average call duration grew from 5.5 minutes to 9.5 minutes; patients choose to stay engaged when the experience is safe, coherent, and empathic.
In Polaris 5.0 deployments, average call duration increased from 5.5 minutes to 9.5 minutes. Patients choose to stay engaged when the experience is safe, coherent, and empathic.
But technology alone isn’t enough. The best voice AI is useless if clinicians don’t trust it or don’t know how to orchestrate it. That’s why we invest in workforce readiness alongside model development. We partnered with Chamberlain and Walden on a micro-learning AI education program: 478 learners and faculty completed 612 courses, demonstrated significant knowledge gains, and 99% of faculty committed to apply what they learned (Chamberlain University & Walden University, 2024).
At 180 million patient interactions, we’ve built something that works at scale. We’re also honest about what’s ahead. “Five nines” reliability remains aspirational. Regulations continue to evolve. EHR integration is real, ongoing work. The discipline is operating a probabilistic system in messy real-world healthcare: measuring safety continuously, catching regressions fast, preserving “do no harm” day after day.
If you’ve read this far, you probably see what we see: a problem space that’s technically fascinating, clinically meaningful, and where the work is far from done.
We’re not building another chatbot. We’re building infrastructure for a new kind of healthcare interaction: one that combines the scalability of AI with the trust dynamics of human conversation.
We need engineers who get excited about:
The work is hard, the stakes are real, and the mission is healthcare abundance. We’re building the systems that can make it real.
Chamberlain University, & Walden University. (2024). [Micro-learning AI education
program evaluation (preprint).
Hippocratic AI. (2026a). Polaris 5.0. https://hippocraticai.com/polaris/
Hippocratic AI. (2026b). Hippocratic AI research. https://hippocraticai.com/research/
Shah, M. (2026). 2026: The year of healthcare abundance. LinkedIn.
https://www.linkedin.com/pulse/2026-year-healthcare-abundance-munjal-shah-hdx0c/
Systematic review and meta-analysis on AI chatbot empathy. (2025). British Medical
Bulletin, 156(1). https://academic.oup.com/bmb/article/156/1/ldaf017/8293249
Turn-taking timing universals. (2009). Proceedings of the National Academy of Sciences.
https://europepmc.org/articles/PMC2705608
University of Wisconsin–Madison. (n.d.). Instant messages versus human speech:
Hormones and why we still need to hear each another. Waisman Center.
https://childemotion.waisman.wisc.edu/publications/instant-messages-versus-human-speech-hormones-and-why-we-still-need-to-hear-each-another/