How It Works
1. Idea Generation
The system brainstorms novel research directions:
Input: "Improve training efficiency of small language models"
Generated Ideas:
1. Adaptive learning rate scheduling based on gradient noise
2. Curriculum learning with dynamic difficulty assessment
3. Sparse attention patterns for resource-constrained training
...2. Experiment Design
For each idea, it designs and implements experiments:
# AI generates experiment code automatically
# Modifies the template codebase
# Adds metrics, logging, and ablation studies3. Experiment Execution
Runs experiments with full logging:
Running experiment: adaptive_lr_scheduling
Epoch 1/50: loss=2.34, lr=0.001
Epoch 2/50: loss=1.89, lr=0.0008
...
Results: 15% improvement over baseline4. Paper Writing
Generates a complete LaTeX paper:
├── paper.tex # Full manuscript
├── figures/ # Generated plots
├── references.bib # Cited works
└── review.txt # Self-review and scoring5. Self-Review
The system reviews its own paper and provides a score:
Novelty: 7/10
Significance: 6/10
Clarity: 8/10
Overall: Accept with minor revisionsResearch Templates Available
| Template | Domain | Description |
|---|---|---|
nanoGPT |
NLP | Small language model training |
grokking |
ML Theory | Generalization phenomena |
2d_diffusion |
Generative AI | 2D diffusion models |
Key Stats
- 12,000+ GitHub stars
- By Sakana AI
- End-to-end automated research
- LaTeX paper generation
- Self-review and scoring
- Multiple research templates
FAQ
Q: What is AI Scientist? A: AI Scientist is an automated research system that generates ideas, runs experiments, and writes complete scientific papers autonomously using LLMs.
Q: Is AI Scientist free? A: Yes, open-source under Apache 2.0. You need LLM API keys and compute for experiments.
Q: Can AI Scientist produce publishable papers? A: It generates papers at workshop-quality level. The system includes self-review, but human review is recommended before submission.