Johns Hopkins academic project
AI Inbox Triage & Drafting Assistant
A human-reviewed system for turning a crowded inbox into prioritized work, grounded summaries, and safer first-response drafts.
Scope: midterm project for applied generative AI study at Johns Hopkins, submitted under the assignment title AI Email Secretary. It uses a synthetic business scenario and sample email dataset; it is not connected to a live inbox and does not send messages autonomously.
Problem
A manager facing a mixed inbox needs to separate genuine emergencies and same-day deadlines from routine updates, informational messages, and spam. A useful assistant must reduce review time without mistaking urgent language for actual urgency—or making commitments on the manager’s behalf.
Approach
- Filter deterministically. Apply the Yesterbox method with pandas so only the previous day’s messages enter the workflow.
- Classify with guardrails. Sort messages into six defined categories while checking the full body, recipient, context, and actual deadline.
- Generate one item at a time. Produce source-grounded summaries, next steps, and first-response drafts for each critical email.
- Protect consequential decisions. Prevent the model from inventing approvals, commitments, attendance, attachment reviews, or unsupported facts.
- Evaluate against the source. Use structured LLM-as-a-judge scoring for relevance, clarity, and actionability while preserving traceability to each email.
Result
The workflow processed all 51 previous-day messages in the sample dataset. It identified 23 critical messages, generated complete summaries for 13 urgent and 10 deadline-driven emails, produced 23 first-response drafts, and evaluated all 23 drafts against their source messages.
- Relevance: 4.00 / 5 average
- Clarity: 4.70 / 5 average
- Actionability: 3.09 / 5 average
The lower actionability score exposed the project’s most useful next step: make safe follow-up timing, ownership, and requests for missing information more specific without allowing the model to fabricate a decision.
What it demonstrates
The project combines deterministic data processing, constrained prompt design, one-to-one completeness checks, structured evaluation, and explicit human oversight. Its conclusion is intentionally modest: the system can support inbox triage and drafting, but a person must verify classifications, facts, dates, and commitments before any response is sent.