Detailed Introduction to the STORM Project

Project Overview

STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking) is an open-source AI knowledge curation system developed by the Stanford Open Virtual Assistant Lab (OVAL) at Stanford University. This system can write Wikipedia-like articles from scratch based on internet searches and automatically generate complete citations.

To date, over 70,000 people have tried STORM's online research preview, demonstrating the system's practical value and widespread attention.

Core Features

1. Two-Stage Article Generation Process

STORM breaks down long article generation into two key stages:

• Pre-writing Stage: The system conducts internet-based research, gathers reference materials, and generates an article outline.
• Writing Stage: The system uses the outline and reference materials to generate a complete article with citations.

2. Multi-Perspective Question Asking Mechanism

STORM employs two strategies to enhance the depth and breadth of questions:

• Perspective-Oriented Question Asking: Discovers different perspectives by researching existing articles on similar topics, which are then used to control the question-asking process.
• Simulated Conversation: Simulates a dialogue between a Wikipedia editor and a subject matter expert, discussing based on internet resources.

3. Co-STORM Collaborative Enhanced Version

Co-STORM is a collaborative enhanced version of STORM, supporting human-AI collaborative knowledge curation:

• Multi-type LLM Agents: Includes Co-STORM expert agents and a moderator.
• Dynamic Mind Map: Maintains a dynamically updated mind map, organizing collected information into a hierarchical conceptual structure.
• Human-AI Collaboration Protocol: Implements transition management strategies to support seamless collaboration between humans and AI systems.

Technical Architecture

Supported Components

Language Models:

• Supports all language models supported by litellm.
• Different models can be configured for different task components.

Retrieval Module: Supports various search engines and retrievers: YouRM, BingSearch, VectorRM, SerperRM, BraveRM, SearXNG, DuckDuckGoSearchRM, TavilySearchRM, GoogleSearch, and AzureAISearch.

Embedding Models:

• Supports all embedding models supported by litellm.

Modular Design

STORM adopts a highly modular design, implemented based on the dspy framework, and comprises four main modules:

1. Knowledge Curation Module: Gathers extensive information on a given topic.
2. Outline Generation Module: Organizes the collected information by generating a hierarchical outline.
3. Article Generation Module: Populates and generates the article based on the outline and collected information.
4. Article Polishing Module: Optimizes and enhances the presentation of the written article.

Installation and Usage

Quick Installation

# Install using pip
pip install knowledge-storm

# Or install from source
git clone https://github.com/stanford-oval/storm.git
cd storm
conda create -n storm python=3.11
conda activate storm
pip install -r requirements.txt

Basic Usage Example

import os
from knowledge_storm import STORMWikiRunnerArguments, STORMWikiRunner, STORMWikiLMConfigs
from knowledge_storm.lm import LitellmModel
from knowledge_storm.rm import YouRM

# Configure language models
lm_configs = STORMWikiLMConfigs()
openai_kwargs = {
    'api_key': os.getenv("OPENAI_API_KEY"),
    'temperature': 1.0,
    'top_p': 0.9,
}

# Set models for different components
gpt_35 = LitellmModel(model='gpt-3.5-turbo', max_tokens=500, **openai_kwargs)
gpt_4 = LitellmModel(model='gpt-4o', max_tokens=3000, **openai_kwargs)

lm_configs.set_conv_simulator_lm(gpt_35)
lm_configs.set_question_asker_lm(gpt_35)
lm_configs.set_outline_gen_lm(gpt_4)
lm_configs.set_article_gen_lm(gpt_4)
lm_configs.set_article_polish_lm(gpt_4)

# Configure retrieval module
engine_args = STORMWikiRunnerArguments(...)
rm = YouRM(ydc_api_key=os.getenv('YDC_API_KEY'), k=engine_args.search_top_k)
runner = STORMWikiRunner(engine_args, lm_configs, rm)

# Run generation
topic = input('Topic: ')
runner.run(
    topic=topic,
    do_research=True,
    do_generate_outline=True,
    do_generate_article=True,
    do_polish_article=True,
)

Co-STORM Usage Example

from knowledge_storm.collaborative_storm.engine import CollaborativeStormLMConfigs, RunnerArgument, CoStormRunner

# Configure Co-STORM
lm_config = CollaborativeStormLMConfigs()
# ... Configure various language models ...

topic = input('Topic: ')
runner_argument = RunnerArgument(topic=topic, ...)
costorm_runner = CoStormRunner(lm_config=lm_config, ...)

# Warm-start the system
costorm_runner.warm_start()

# Conduct collaborative dialogue
conv_turn = costorm_runner.step()
# Or inject user utterance
costorm_runner.step(user_utterance="YOUR UTTERANCE HERE")

# Generate report
costorm_runner.knowledge_base.reorganize()
article = costorm_runner.generate_report()

Academic Research and Datasets

Research Papers

STORM's research findings were published at NAACL 2024, with the paper titled "Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models." The Co-STORM paper was accepted to EMNLP 2024 Main Conference.

Dataset Contributions

FreshWiki Dataset: A collection of 100 high-quality Wikipedia articles, focusing on the most edited pages between February 2022 and September 2023.

WildSeek Dataset: Used to study user interests in complex information-seeking tasks, where each data point includes a topic and the user's goal for in-depth search.

Evaluation and Feedback

Automatic Evaluation Results

STORM outperforms strong retrieval-augmented generation baselines on all automatic metrics, including LM evaluation and comparison metrics against human-written articles.

Expert Evaluation

In human evaluations with experienced Wikipedia editors, all participants agreed that the system was helpful for their pre-writing stage. Compared to articles generated by outline-based retrieval-augmented baselines, more STORM articles were perceived as organized (25% absolute increase) and comprehensive (10% increase).

Application Scenarios

Applicable User Groups

• Students: For creating research papers and reports with citations.
• Researchers: For compiling comprehensive literature reviews.
• Content Creators: For generating structured, in-depth articles.
• Wikipedia Editors: As an auxiliary tool for the pre-writing stage.

Usage Limitations

While the system cannot produce publish-ready articles (which typically require extensive editing), experienced Wikipedia editors found it helpful during the pre-writing stage.

Project Development

Latest Progress

• January 2025: Added litellm integration for language models and embedding models.
• September 2024: Co-STORM codebase released and integrated into the knowledge-storm python package v1.0.0.
• July 2024: Available for installation via pip install knowledge-storm.

Future Directions

The team is actively developing:

• Human-AI Interaction Features: To support user participation in the knowledge curation process.
• Information Abstraction: To develop abstractions of curated information to support presentation formats beyond Wikipedia-style reports.

Open Source Contributions

This project is fully open source, and community contributions are welcome. PRs integrating more search engines/retrievers into knowledge_storm/rm.py are especially encouraged.

Project Address: https://github.com/stanford-oval/storm Online Demo: https://storm.genie.stanford.edu/ Project Website: https://storm-project.stanford.edu/

Citation Information

If you use STORM in your research, please cite the relevant paper:

@inproceedings{shao-etal-2024-assisting,
    title = "Assisting in Writing {W}ikipedia-like Articles From Scratch with Large Language Models",
    author = "Shao, Yijia and Jiang, Yucheng and Kanell, Theodore and Xu, Peter and Khattab, Omar and Lam, Monica",
    booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    pages = "6252--6278",
}

The STORM project represents a significant breakthrough in AI-assisted knowledge curation, providing powerful tools and methods for automated research and writing.