Research Forge: Multi-Agent Research Pipeline with Citations

01 / Overview

Research with language models becomes difficult to trust once the process grows beyond a handful of searches. Sources disappear into context, claims become detached from their evidence, and a failed run can mean repeating large parts of the work.

I built Research Forge around a different idea:

the research process itself should have structured state.

Instead of asking one agent to search, reason, and write a report in a single pass, Research Forge breaks the process into distinct stages for planning, collection, claim extraction, relationship analysis, synthesis, formatting, and publishing.

Every run keeps its evidence, claims, citations, gaps, decisions, and outputs as inspectable artifacts.

The goal

Turn:

research question

into:

evidence → verified claims → structured synthesis → citation-backed report

without losing the connection between the final writing and the sources that support it.

02 / My Contribution

I designed and built the project as an end-to-end research workflow rather than a single prompt or agent.

What I built

Designed the multi-stage research architecture and orchestration flow
Built a seven-phase pipeline separating planning, collection, synthesis, formatting, and publishing
Added resumable runs using persistent manifests and artifact validation
Designed a provenance system connecting collected evidence to canonical claims
Implemented citation validation and research coverage auditing
Built gap detection to identify weak or uncovered areas before final report generation
Added human review checkpoints for scope, sources, claims, and the finished report
Integrated web crawling and document ingestion through Crawl4AI and Docling
Added graph-based relationship analysis between claims and entities
Built optional report publishing through Quarto

The main engineering work was not simply connecting an LLM to a crawler. It was designing the state and boundaries between each stage so the system could remain inspectable, recoverable, and steerable.

03 / Architecture

Research Forge runs as a collection of native Claude Code skills and agents coordinated by a central research orchestrator.

Pipeline flow

Phase	Responsibility	Output
01 · Planning	Define scope and break the question into an investigation tree	Scope + research plan
02 · Collection	Crawl web sources and parse documents	Provenance-tagged evidence
03 · Claims	Convert evidence into canonical research claims	Claim bank
04 · Relationships	Connect claims and entities	Graph metadata
05 · Synthesis	Build compact section-level research state	Section briefs + gap analysis
06 · Formatting	Assemble the approved research into a report	`report.md`
07 · Publishing	Render optional presentation formats	HTML / PDF

The important boundary is that collection does not write the report.

Evidence first becomes structured research state. The formatter only receives that validated state when composing the final document.

04 / Engineering Decisions

Claims as the primary research state

One of the most important decisions was making a structured claim bank the center of the synthesis process.

Instead of repeatedly feeding all raw research back into later agents, evidence is converted into canonical claims with stable IDs and source relationships.

CODE

source
  ↓
evidence
  ↓
claim
  ↓
section brief
  ↓
report

This keeps the path from a sentence in the final report back to its supporting evidence explicit.

It also makes other operations possible:

detecting unsupported claims
identifying evidence used across multiple sections
checking citation coverage
inspecting weak areas of the research
changing report structure without recollecting everything

Human checkpoints instead of full autonomy

Research Forge intentionally stops at four points in the pipeline.

Gate	Human decision
Scope	Is the system investigating the right question?
Sources	Are the collected sources acceptable?
Claims	Is the evidence strong and complete enough?
Report	Is the final document ready to publish?

The system can automate a large amount of work, but decisions that substantially change the research direction remain visible to the user.

This was a deliberate tradeoff: more interaction in exchange for greater control over the final result.

Persistent artifacts instead of hidden agent memory

Every research run produces a self-contained directory rather than depending on conversation history.

CODE

research/run-*/
│
├── scope/
│   ├── scope.md
│   ├── plan.json
│   └── question_tree.json
│
├── collect/
│   ├── inventory.json
│   ├── evidence/
│   └── quarantine/
│
├── synthesis/
│   ├── claim_bank.json
│   ├── entity_index.json
│   ├── section_briefs/
│   ├── citation_audit.md
│   └── gap_analysis.md
│
├── output/
│   ├── report.md
│   ├── formatter_audit.json
│   └── report.pdf
│
└── manifest.json

The filesystem effectively becomes the pipeline's memory.

That makes individual stages easier to inspect, validate, retry, and debug.

05 / The Challenge

Making a multi-agent workflow recoverable

Long-running research workflows have an awkward failure mode.

If stage six fails after stages one through five completed successfully, restarting the entire research process wastes work and can produce a different result.

Research Forge therefore treats progress as persistent state.

Each run has a manifest.json containing the status of every phase:

CODE

pending → running → complete
             ↓
           failed

When an interrupted run is resumed, the orchestrator does not simply trust that a previous phase finished.

It checks that the artifacts required by that phase still exist and pass validation before continuing.

This changed the architecture from a sequence of prompts into something closer to a small workflow engine.

Keeping research grounded

Another challenge is that collecting more information does not automatically produce better research.

The system needs to know:

which claims have strong support
which sections have weak evidence
which branches of the original investigation remain unanswered
whether the final report actually cites the evidence that was collected

Research Forge therefore performs gap analysis before report generation.

If important research branches remain uncovered, the pipeline can return to collection and search specifically for the missing evidence.

CODE

question tree
     ↓
evidence coverage
     ↓
claim coverage
     ↓
gap detected?
   ↙       ↘
 yes       no
  ↓         ↓
collect    format
 more      report

That feedback loop is one of the core differences between Research Forge and a single-pass research agent.

06 / Safety & Provenance

Research pipelines have an additional problem: the content they retrieve is not trustworthy simply because it appears on a webpage.

Collected web pages and documents are therefore treated as data, not instructions.

Research Forge separates external content from agent control logic and can quarantine sources before they enter synthesis.

At the same time, provenance metadata follows evidence through the pipeline:

CODE

URL / Document
      ↓
Evidence ID
      ↓
Claim ID
      ↓
Section
      ↓
Inline Citation

This makes the final report easier to audit because a reader can trace important statements back through the system instead of relying on hidden model context.

07 / Current State

Research Forge currently supports the complete research lifecycle from question to report.

Working now

A research session can be started directly inside Claude Code with:

/research <research question>

and resumed later from its persisted state if the workflow is interrupted.

08 / What This Project Demonstrates

Research Forge started as an experiment in AI-assisted research, but the more interesting problem became system design around the model.

The project pushed me to think about:

orchestration instead of individual prompts
explicit state instead of hidden context
provenance instead of generated confidence
validation instead of assuming a stage succeeded
human review instead of unconditional autonomy
recoverability instead of restarting failed workflows
structured intermediate representations instead of passing raw text between agents

The LLM performs important reasoning inside the system, but much of the reliability comes from the architecture surrounding it.

09 / Next

The project is still evolving.

Areas I want to explore next include:

stronger quantitative evaluation of citation accuracy
automated tests against known research questions
better visualization of claim/evidence relationships
configurable source-quality policies
more targeted gap-filling strategies
comparison of research quality across different pipeline configurations
richer reporting and interactive exploration of research state

The larger goal is to keep pushing Research Forge toward a research system where the reasoning process is not only useful, but inspectable, reproducible, and defensible.