Case Study - Scaling Patient Triage with LLMs
To address growing triage volume in patient chat messages, I designed and deployed an AI-driven system using Claude LLM via Amazon Bedrock. It classified message category and severity, responded safely to low-risk inquiries, and escalated urgent messages to providers with real-time SMS alerts.
- Client
- AI Triage System
- Year
- Service
- FastAPI, Amazon Bedrock, Claude LLM
Overview
The company’s chat-based care model relied heavily on human triage to read and route patient messages in real-time. As message volume increased, so did the burden on triage staff — leading to rising costs and the risk of urgent cases getting buried.
Every message had to be manually:
- Classified by category (e.g., behavioral health, physical symptom, insurance, logistics)
- Scored for severity, determining whether it required urgent clinical review
- Routed to the appropriate provider or team
This model was labor-intensive, reactive, and didn’t scale. The team knew AI could help — the challenge was introducing it safely into a regulated, high-stakes workflow.
What we did
As Director of Engineering, I led the design and implementation of a real-time triage system proof of concept powered by OpenAI. After successfully showing the benefits of the system, I worked with a team of contractors to implement the version we would release into production, now powered by a FastAPI backend and integrating Claude LLM via Amazon Bedrock.
The system:
- Automatically analyzed incoming chat messages in real time using prompt-engineered LLM queries
- Classified each message by clinical category and scored severity based on known escalation criteria
- Auto-responded to low-risk, non-clinical messages with pre-approved answers from our training set
- Escalated urgent/severe messages directly to licensed providers with immediate SMS alerts, reducing lag between patient intent and clinician action
We carefully tuned the model’s prompts and outputs to respect healthcare-specific guardrails, including privacy constraints and medical language nuances. By applying role-based response logic, the AI could distinguish between messages needing reassurance, information, or clinical review.
Impact
- 50% reduction in triage workload: Human triage was now only required for edge cases or escalations
- Faster response times for high-severity messages — clinicians were notified immediately without manual filtering
- Greater provider focus: Less time wasted on low-risk messages allowed more attention on patients with clinical needs
- Scalable infrastructure: The system processed messages 24/7, without adding staff or compromising safety
Technologies Used
- FastAPI for core backend and message handling
- Amazon Bedrock to access Claude LLM securely in HIPAA-aware architecture
- Custom prompt engineering for multi-label classification and scoring
- SMS gateway integration for real-time provider notifications
- Role-based logic engine for safe auto-response and escalation
Summary
This project transformed a bottleneck into a strategic advantage. By introducing LLM-driven triage, we enabled the company to scale its care delivery model while protecting response time, patient safety, and provider bandwidth. It was a case of using AI not to replace humans — but to let them focus where they matter most.
The LLM-based triage system Thiago led at Caraway was transformative. It helped us manage increasing message volume without compromising care quality or safety. His thoughtful implementation of AI—grounded in both technical rigor and patient-centered thinking—allowed us to extend our clinical capacity at scale. A true innovation with real-world impact.
Thiago created our home-grown patient triage model atop Amazon/Anthropic tech. He worked closely with business stakeholders to define and implement an impressive piece of technology that dramatically improved our ability to focus clinical operations on patients in greatest need. This work came to life incredibly quickly and soon became one of the most important and valuable tech capabilities at the company.