PRAXIST — Autonomous Research System for Executable Experiments

Introduction

PRAXIST is an autonomous research system that turns natural-language hypotheses into fully executable experiments. Given a research question, it designs the experimental protocol, writes the code, runs the experiment, collects results, and produces a structured report — all without manual intervention. It bridges the gap between asking a research question and getting a reproducible, quantitative answer.

What PRAXIST Does

• Accepts research hypotheses in natural language and generates executable experiment plans
• Writes experiment code automatically, including data loading, training loops, and evaluation
• Runs experiments in isolated environments and captures all metrics and artifacts
• Produces structured reports with tables, charts, and statistical significance tests
• Maintains an experiment log for reproducibility and audit trails

Architecture Overview

PRAXIST uses a multi-agent pipeline where a planner agent decomposes the hypothesis into sub-experiments, a coder agent generates Python scripts for each, and an executor agent runs them in sandboxed containers. A reviewer agent validates results against the original hypothesis and flags anomalies. The entire pipeline is orchestrated by a coordinator that manages dependencies between experiment stages. Results are stored in a structured database with full provenance tracking.

Self-Hosting & Configuration

• Install from source with pip and configure the LLM backend (supports multiple providers)
• Set compute resource limits for experiment sandboxes via the configuration file
• Configure artifact storage to local disk or S3-compatible object storage
• Enable GPU access for experiments that require training by setting device flags
• Customize the report template to match your team's publication format

Key Features

• End-to-end automation from hypothesis to reproducible results
• Sandboxed experiment execution prevents side effects between runs
• Full provenance tracking: every generated file, command, and metric is logged
• Statistical validation layer that checks for significance and common pitfalls
• Extensible agent pipeline: add custom agents for domain-specific analysis

Comparison with Similar Tools

• AutoResearch (Karpathy) — focuses on neural-net training experiments; PRAXIST handles broader research types
• MLflow — tracks experiments but does not design or run them; PRAXIST automates the full cycle
• Weights & Biases — experiment tracking and visualization; PRAXIST starts from hypothesis, not code
• DeerFlow — general-purpose research agent; PRAXIST specializes in measurable, executable experiments

FAQ

Q: What kinds of experiments can PRAXIST run? A: Any experiment expressible as Python code — ML training, data analysis, simulations, benchmarks.

Q: Does it require a specific LLM? A: No. PRAXIST supports OpenAI, Anthropic, and open-source models via a pluggable backend.

Q: How does it ensure reproducibility? A: Every run records the full environment, random seeds, code, and data versions in a manifest.

Q: Can I review the generated code before execution? A: Yes. A dry-run mode outputs the experiment plan and code for human review before running.

Sources

• https://github.com/sapientinc/PRAXIST