Production account · Amazon Pharmacy
Three production agent workflows
Three production AI agents
I own three production generative-AI agents across customer assistance, clinical support, and claims and billing.
Senior Product Manager – Technical · Amazon Pharmacy
Three production agents at a national online pharmacy, one standard of accountability.
AI earns a place when it removes meaningful work or improves a decision enough to justify its cost, uncertainty and operating burden. I own three production generative-AI agents, and I hold each to the same standard: define the evidence, evaluate failures, make corrections possible, keep authority with a person for consequential actions, and record the reason for the final decision.
I defined the AI-assisted to AI-primary strategic theme for annual planning, and it was adopted as a planning pillar. The customer assistant resolves 78% of chats without a human. In healthcare, launch is when you have proven safety at scale, not when the code ships.
Production · Customer assistant agent
A customer assistant that resolves 78% of chats without a person
The customer assistant answered common questions using context and retrieval, with a summary for the representative when the issue needed a person. For this illustrative workflow, ordering requires a separate authorized step.
Outcomes: 78% of chats resolved without a human; 18% of voice contacts fully automated; moving from an intent router to ReAct lifted resolution 18%; hot-path optimization cut median inference calls from two to one for 43% P50 and 67% P90 latency improvement and 40–54% lower token cost.
Workflow explanation
- Contact arrives Chat or voice · identity established first
- Reason and act ReAct loop · one inference call on the hot path
- Live data via tools Prescriptions, orders, insurance, pricing
- Bounded actions Illustrative ordering requires separate authorization
- Resolve or hand off Summary to a person · acceptance is illustrative policy
- Measure Illustrative policy · evaluate resolution and handoff
Production · Clinical copilot
A clinical copilot rebuilt when 15-second latency made it unusable
The copilot supported clinical operators with curated procedures, case summaries and next-best actions. The illustrative validation control checks latency and accuracy before expanding the workflow.
Outcomes: P90 15 s to 1.6 s; a model change to a higher per-token cost model with fewer turns lowered net cost; a storage-key change cut memory cost 95%.
Workflow explanation
- Case or call Operator opens a case or takes a call
- Retrieve Curated procedures and knowledge with citations
- Summarize Issue, history and next best action
- Stream Illustrative response delivery · validate latency and accuracy
- Operator decides Clinical authority stays with the person
- Validate Illustrative policy · check latency and accuracy
Production · Claims and billing agent
A claims and billing agent that explains 800+ rejection codes
The billing workflows are in production after pilot and ramp-up. The agent interprets rejection knowledge and can correct a 30-day supply to the allowed 28 days, update insurance and resubmit, or refile an approved 30-day quantity when a 90-day request exceeds the insurance limit. Clinical/DUR edits go to a pharmacist for manual verification and approval; brand/generic prescription and DAW edits go to a technician. Prior authorization goes to a PA bot for Cover My Meds filing, follow-up after four unresolved days and customer notification after approval. Other paths compare coupon and insurance pricing or communicate the next available refill date. I integrated the billing agent into the Amazon Pharmacy platform after prescription typing and launched the prescription flow: a technician or pharmacist refiles after making the required edit, and the agent suggests further changes. The work affected more than 100,000 prescriptions per month at Amazon Pharmacy.
Workflow explanation
- Claim rejects One of 800+ codes · representative asks
- Retrieve Rejection-code knowledge base
- Look up Authorized billing context and rule limits
- Interpret and route Allowed billing actions · clinical edits to pharmacist; prescription edits to technician
- Guardrails Illustrative policy · record and review breaches
- Experiment Illustrative policy · review candidate changes
Illustrative review controls
This illustrative breach-review control records safety exceptions and routes them for review. Candidate prompt, model and guardrail changes are evaluated before reuse. Action permissions remain specific to the authorized workflow.