Johns Hopkins academic project
AI Inbox Triage & Drafting Assistant
A human-reviewed system for turning a crowded inbox into prioritized work, grounded summaries, and safer first-response drafts.
Scope: midterm project for Applied Generative AI study at Johns Hopkins, submitted under the assignment title AI Email Secretary. It uses a synthetic business scenario and sample email dataset; it is not connected to a live inbox and does not send messages autonomously.
Workflow
- Sample inbox
- Previous-day filter
- 51 messages reviewed
- 23 critical messages
- Summarize and draft
- Evaluate against source
- Human review
Problem
A manager facing a mixed inbox needs to separate genuine emergencies and same-day deadlines from routine updates, informational messages, and spam. A useful assistant must reduce review time without mistaking urgent language for actual urgency—or making commitments on the manager’s behalf.
Approach
- Filter by fixed rules. Apply the Yesterbox method with pandas so only the previous day’s messages enter the workflow.
- Classify with guardrails. Sort messages into six defined categories while checking the full body, recipient, context, and actual deadline.
- Generate one item at a time. Produce source-grounded summaries, next steps, and first-response drafts for each critical email.
- Protect consequential decisions. Prevent the model from inventing approvals, commitments, attendance, attachment reviews, or unsupported facts.
- Evaluate against the source. Use a second model to score each draft for relevance, clarity, and actionability, with every score traceable to the original email.
Result
The workflow processed all 51 previous-day messages in the sample dataset. It identified 23 critical messages, generated complete summaries for 13 urgent and 10 deadline-driven emails, produced 23 first-response drafts, and evaluated all 23 drafts against their source messages.
- Relevance: 4.00 / 5 average
- Clarity: 4.70 / 5 average
- Actionability: 3.09 / 5 average
The lower actionability score exposed the project’s most useful next step: make safe follow-up timing, ownership, and requests for missing information more specific without allowing the model to fabricate a decision.
Limits
The assistant supports triage and drafting; it does not make or send decisions. A person must verify classifications, facts, dates, and commitments before any response is sent.