DCR Triage Agent

Smart DCR Triage Assistant for Teams PMs — PM Decides, Agent Assists

🌐 Portal · Web Dashboard 🖥️ CLI · Copilot CLI / Claude

7Step Pipeline

9Judgment Types

4Context Layers

liuhanlin | Mentor: Fei Zuo | FY26 Vendor Student Program | Microsoft Teams CMD

Pain Points — Why We Need Automation

The Teams PM team (CMD SOX) receives a high volume of DCRs (Design Change Requests) via IcM. Each DCR requires safety check → classification → dedup → reply → escalation review — repetitive and time-consuming.

PM Pain Points

⏱️

Repetitive Work

Every DCR follows the same triage flow, consuming significant PM time

🔀

Inconsistent Judgment

Different PMs may judge similar DCRs differently

📋

Inconsistent Reply Format

Reply quality depends on the individual; no shared template or guardrails

Process Pain Points

🔍

Hard to Detect Duplicates

Historical DCRs are scattered; manual comparison is slow and error-prone

🛡️

Safety Boundaries Easily Missed

S500 / compliance / high-severity cases need special handling, easy to overlook

📊

Decisions Not Traceable

Reasoning isn't recorded, blocking retrospection and continuous improvement

Solution Architecture

Core Engine — Copilot CLI

LLM Reasoning · 7-Step Pipeline · Self-Eval

Pulls incident data in real time via the IcM MCP Server
Copilot CLI runs the 7-step pipeline: safety → quality precheck → classify+reply → dedup → escalation → self-eval
Subprocess-per-call isolation: each triage runs in its own process

All triage intelligence is driven by Copilot CLI — Portal calls this engine to fetch results

Frontend Portal — Web Dashboard

FastAPI + Vanilla JS · Dark Theme · Zero Dependencies

One-Click Triage: enter IcM ID → invoke CLI engine → render results
Team Dashboard: global stats + overdue alerts + PM workload
Result Visualization: Judgment Banner + Classification + Escalation scoring
PM Feedback Loop: Agree / Override → record override rate → continuous tuning

PM's daily entry point — review triage results, give feedback, collaborate

📥 Input Layer

IcM Incident ID → fetch DCR data in real time → normalized input format

🧠 Reasoning Layer

4-layer progressive context → CLI 7-step analysis → Self-Eval supervision

📤 Output Layer

Classification + Dedup + Suggested Reply + Escalation → mapped to IcM DCR Decision

♻️ Feedback Layer

PM Agree/Override → record override rate → ongoing prompt tuning

7-Step Triage Pipeline

Safety Boundary Check

Sev0/1, S500/ACE + broad impact, compliance/legal/privacy → forces PM Review

Rule Engine · No LLM

Quality Precheck

Is the description sufficient? Missing customer symptoms? Severity reasonable? → generate clarifying questions

Rule + LLM

Ownership Routing

Based on PM → Feature Area mapping, decide whether the assignee is correct

Keyword Match · No LLM

Documentation Precheck

"Is there existing documentation that already answers this?" → search local / SharePoint

Knowledge Retrieval + LLM

Classification + Suggested Reply

★ Core step — progressive context injection → 9 judgment types + PM-style reply draft

LLM · ContextAssembler

Duplicate / Similar Detection

IcM API + local cache compare against historical DCRs; strict core-ask matching prevents false positives

LLM · Similarity Scoring

Escalation Review

5-dimensional scoring → auto-close / pm-review / escalate / safety-boundary

LLM · Weighted Scoring

Self-Eval Supervision

Actor-Critic: a QA reviewer verifies judgment consistency → PASS / DOWNGRADE / FAIL

LLM · Quality Calibration

9 Judgment Types & Context Engineering

✅ Accepted

accept-backlog
Accepted into backlog, no ETA

accept-existing
Existing feature already addresses this

❌ Rejected

reject-by-design
Conflicts with product direction

reject-technical
Technically infeasible

non-dcr
Bug / regression / misunderstanding

close-no-response
Customer unresponsive

🔄 Re-assign

re-assign
Outside this team's scope
Route to correct owner

⚠️ ≤10% — not a "safe default"

⏳ Pending

need-info
Insufficient info, needs clarification

discuss
Requires PM team discussion

⚠️ discuss ≤5%

4-Layer Progressive Context Injection (ContextAssembler)

🎯

Layer 0 — Role Identity Always injected

DCR triage assistant definition + judgment priority rules + recommending-language requirement

📐

Layer 1 — Decision Rules Always injected

Full decision tree (Step 0-7) + 9 judgment definitions + anti-escalation / anti-evasion rules

👤

Layer 2 — PM Profile On-demand (1-2 matches)

PM historical preferences + routing signals (Forms→Forms PM, recording permissions→ODSP)

⚠️

Layer 3 — Deep Context Conditional (only triggered by safety/compliance)

Safety boundary details + compliance guidance + cross-team decision authority + platform technical limits

Progressive injection saves ~45% tokens — only feed the LLM the knowledge this DCR actually needs

Portal Features at a Glance

FastAPI + single-page frontend, dark theme, zero-dependency vanilla JS

🔍

Smart Search & IcM Sync

Enter IcM Incident ID → fetch data live → auto-start the 7-step triage
Supports search modes: ID / title / assignee

📊

Judgment Summary Banner

Prominent top banner showing DCR Decision + How Fixed + Judgment
Self-Eval status badge: ✅ PASS / ⚠️ DOWNGRADE / ❌ FAIL

💬

Suggested Response Pinned

PM sees the suggested reply first, can copy & forward directly
9 PM-style templates + Support-Safe language guardrails

👥

Live Customer Info

S500 / CritSit / SR count / subscriptions / submitter
IcM API + disk cache, automatically flags high-priority customers

🔗

IcM Direct Links

Each incident links straight to IcM Portal
Classification card auto-bolds keywords for emphasis

👍

PM Feedback Loop

Agree / Override buttons → record override rate
Provides data for prompt tuning & continuous accuracy gains

💾

Result Persistence

localStorage + server disk cache → survives refresh
CLI triage results stay visible after reload

📈

Analytics Dashboard

Bottom observability panel: CLI call count / token usage
Latency distribution / judgment distribution / confidence distribution

🖥️ Product Demo — Core Views

PM daily workflow: from team-wide overview to single DCR triage — click image to view fullscreen

📊 Team Dashboard — Team-wide View

📈 Live Stats Panel

⚠️ Overdue Alerts

📊 Distribution Charts

👥 PM Filter

📈 Stats Panel Total / Active / Mitigated / Triaged · ⚠️ Overdue Alerts Aggregated by PM · 📊 Charts State + PM workload

🔍 One-Click Triage — Judgment + Suggested Response

📋 IcM Metadata

⚡ DCR Decision

✅ Self-Eval

💬 Suggested Reply

⚡ Triage Conclusion Decision + How Fixed · ✅ QA Self-Eval Pass / Downgrade / Fail · 💬 Suggested Reply copy & paste ready

🖥️ Product Demo — Classification & Dedup

AI-driven multi-dim classification, escalation review, and historical dedup — click image to view fullscreen

🏷️ Classification Decision — Classification + Escalation

🏷️ 5-Dim Classification

👍 PM Feedback

🔺 Escalation Score

🏷️ 5-Dim Type / Judgment / Impact / Urgency / Confidence · 👍 Feedback Agree / Override · 🔺 Escalation 1-5 score

🔎 Duplicate Detection — Safety Boundaries + Duplicate Check

🛡️ S500/ACE Detect

🔍 Semantic Match

📝 Final Verdict

🛡️ Safety Boundary S500 / ACE flag · 🔍 Semantic Match similarity % + diff explanation · 📝 Verdict <0.7 → genuinely new request

📋 DCR Decision Flow Overview

9 Judgment types → mapped to 4 DCR Decisions (Accepted / Rejected / Re-assign / Pending) — click image to view fullscreen

📋 DCR Triage Decision Flow — End-to-End Path from Intake to Decision

🤖 System-Led Checks DCR validation → info sufficiency → team ownership → existing solution match 👤 PM-Led Decision Product evaluation (reject-by-design / reject-technical / accept-backlog) ✅ Decision Output Accept → write to IcM | Reject → write to IcM | Pending → awaiting PM discussion

Summary & Outlook

Delivered

🏗️

7-Step Pipeline

Safety → Quality → Classify → Dedup → Reply → Escalate → Self-Eval

📐

Context Engineering

4-layer progressive injection, ~45% token savings, on-demand PM Profile

🔍

Self-Eval Supervision

Actor-Critic architecture with PASS / DOWNGRADE / FAIL three-tier check

🌐

Portal Visualization

One-click triage + PM feedback loop + live customer info

Next Steps

📊

Eval Framework

Build Gold Standard
Systematic accuracy measurement

🎯

Quality Gains

Drive answer quality & accuracy
from real PM feedback

🤖

Teams Bot Integration

PMs receive triage suggestions
directly in Teams

🧩

CLI Skill Integration

Package the DCR knowledge base
as a CLI skill for reuse

Thank You — Questions Welcome

liuhanlin · Mentor: Fei Zuo · CMD SOX · Meeting Intelligence