AI for Research Proposal Automation

A research-to-proposal workflow—and a close look at what happens when fluent drafts and confident evaluations outrun their evidence.

Scope: final project for Designing Advanced Generative AI Workflows, submitted as part of my Johns Hopkins Applied Generative AI studies. This is an executed academic notebook using supplied research documents and a funding call, not a grant submission, clinical trial, or deployed product.

The problem

How can a researcher connect a large publication archive to a specific funding opportunity without mistaking a related topic for evidence that actually supports a proposal? The assignment used a mental-health funding call to explore retrieval, research screening, ideation, drafting, and review.

  1. Extract funding topic
  2. Retrieve candidate papers
  3. Screen evidence
  4. Generate and select ideas
  5. Draft proposal
  6. Evaluate and refine

What ran

These are observed run counts, not retrieval-accuracy or productivity metrics. Only 12 of 112 papers received detailed screening.

Follow one evidence trail

Funding topic extracted
“Digital mental health intervention optimization and effectiveness testing.”
Source and screening
The supplied observational social-media/mental-health paper was retained as BACKGROUND, with PDF pages 1–5 recorded as its evidence locator. That is not evidence of treatment efficacy.
Generated idea
“Optimizing Video Content in Digital Mental Health Interventions.” The draft proposed video-versus-text delivery and six-week anxiety and depression measures.
Where it failed
The draft introduced two references unsupported by the supplied evidence and treated an observational motivation as stronger support than it was. A plausible proposal structure did not establish a valid intervention.
My revision
Removed the unsupported references; made the platform partner, evidence base, and 1,000-active-user requirement unresolved dependencies; specified a proposed primary six-week depression endpoint and active treatment comparator; omitted heart-rate variability without a validation argument.

Those are revisions to a research concept, not completed clinical work. No partnership, approval, recruitment, efficacy, or adequate statistical power is claimed.

The judge needed review, too

The original draft received model scores of 3/5 Innovation, 3/5 Significance, 2/5 Approach, and 3/5 Investigator Expertise—a 2.75/5 mean across four criteria for one draft. The assignment defined 1 as poor and 5 as excellent. The gpt-4o-mini judge used the draft, selected source, and funding call at temperature 0.

One critique said the draft claimed a completed power analysis. It did not. My review identified wording in the evaluation prompt that named that anticipated fault; the model repeated it as a finding while missing the unsupported references. This run suggests a prompt-induced failure, not a controlled causal test.

The accurate criticism was missing sample-size justification and planning assumptions. The useful lesson: check the evaluator against the source, not just the draft against the evaluator.

These scores belong only to the original model-generated draft—not the revised concept, my course grade, or an independently validated quality assessment. The revision was not rerun or rescored.

Limits worth keeping visible

What I would improve next

Deduplicate before screening; verify citations against exact passages; justify inclusion against the funding criteria; test alternative retrieval queries; and separately check whether the judge’s criticisms describe anything the draft actually says. Clinical and statistical experts would need to review any real research protocol.

Source: my submitted “JHU AI for Research Proposal Automation Final Project” HTML notebook, including the preserved executed outputs and final written review. This public case study summarizes that evidence; raw notebooks, credentials, and source files are not published.

← Back to AI & Media Operations