Investigating a ticket was never simple
For Amazon engineers, investigating a ticket during on-call was high-stakes and time-sensitive. The moment an alarm went off, every minute counted. Yet the investigation itself was slow and manual: engineers had to jump across multiple tools — pulling logs, querying metrics, checking deployments… Then they formed a hypothesis, tried it, and if it didn't hold, started over.
AI proliferation creates new problems
With the rise of GenAI, teams started building AI tools to make ticket investigation easier and faster, and many pushed to integrate into ticketing for direct access. As the designer overseeing the ticketing system, I noticed new problems forming: inconsistent patterns, overlapping coverage, fragmented insights… Each tool helped in isolation — but together, they were competing for attention and fragmenting the very experience they were trying to improve.
From assumption to evidence — and alignment
To find out whether this was a real problem and how big, I interviewed 10 engineers across different teams about how they investigated tickets and how they used AI tools. The findings were clear: fragmentation was creating real problems, and continuing to build tools separately would only make it worse.
To show why this mattered, I translated the findings into three paradoxes — each tied directly to a metric teams were optimizing for:
The efficiency paradox
While tools aimed to accelerate investigations, users actually spent significant time manually synthesizing overwhelming, disconnected data.
The usefulness paradox
While teams worked in silos to improve their products, the capabilities users need already existed—they were just trapped in silos and need to be orchestrated.
The adoption paradox
High barriers to entry led to low adoption. Even onboarded users abandoned the tools, as the cognitive load outweighed the perceived value.
This shifted the conversation and urged ticketing and two supporting teams to commit to a unified, outcome-driven experience.
Diagnosing the present, imagining the future
Then came the new question: what should that unified experience be? I didn't want to just fix what was broken — I wanted to imagine what a great experience could look like, grounded in how users actually think and work.
Looking at how engineers engaged with AI tools today, I first identified where today's AI tools were falling short:
Trust - Lack of reasoning and evidence, so users had no way to evaluate AI’s conclusions.
Human agency - Tools tried to solve everything end-to-end, but "human-in-the-loop" was still a must for high-stakes investigations.
Mapping engineers' investigation workflows and mental models, I identified more opportunities to make it even better:
Workflow partner - Experience is not static, it should evolve with the user journey.
Personalization - What a user needs depends on the type of issue, the experience should adapt accordingly for more tailored support.
Design principles to guide a unified direction
With a clearer picture of what was missing and what was possible, I translated these insights into design principles — concrete enough for teams to anchor on, and a shared foundation that made aligning on design easier.
01
Flexible autonomy
AI is a tool, not a rule.
02
Human-led steering
AI augments, not replaces.
03
Dynamic intelligence
Investigation is a journey, not a one-off task.
04
Relevance as the criteria
In high-pressure situations, less is more.
05
Trust as the foundation
Never let users guess or validate.
Making direction visible
01
Flexible autonomy
Experience is separated into Insights & Facts, supporting needs of different engagement archetypes.
While users default to AI-generated Insights, relevant Facts are sorted and filtered according to context — presented in a digestible manner so users can reach their own conclusions.
02
Human-led steering
A
Multiple ingress points for users to start chatting with the Oncall Assistant.
B
Allowing users to chat with the agent to steer the investigation, pivot directions, or seek clarifications as needed.
C
Critical insights discovered through chat are updated to the Oncall Assistant — shared with all collaborators on the same ticket.
03
Dynamic intelligence
A
Alarms and impact analysis are shown first — matching what engineers need to understand immediately after landing on the ticket.
B
After user performs the mitigation actions, they can easily rerun overview to view the latest status of the alarm, and verify if the issue has been mitigated.
C
When user reruns key findings, results update based on ticket activities and latest investigation status.
04
Relevance as the criteria
Insights and supporting evidences adapt depending on the ticket type and issues identified.
For example, if AI identified it as an recurring issue, it should have a specific response pattern designed to help users quickly compare this ticket to the past ones, and take actions using established procedures.
05
Trust as the foundation
A
Investigation trail to provide traceability of AI’s thinking process
B
Confidence level to help users calibrate their level of trust of AI-generated insights
C
Supporting evidences to help users easily verify and evaluate accuracy of AI-generated insights. Users can click on the references to learn more details about the evidences
1M+ engineering hours reclaimed annually
The north star design and principles unified three teams around a shared direction — informing product requirements, technical architecture, roadmap prioritization, and a migration plan.
The unified experience is projected to reduce ticket investigation time by 40%, reclaiming 1M+ engineering hours annually.