# OpenAI Gym — Reinforcement Learning Environment Toolkit

> The foundational Python toolkit for developing and comparing reinforcement learning algorithms, providing a standard API for hundreds of simulation environments.

## Install

Save in your project root:

# OpenAI Gym — Reinforcement Learning Environment Toolkit

## Quick Use
```bash
pip install gym
python -c "
import gym
env = gym.make('CartPole-v1', render_mode='human')
obs, info = env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()
env.close()
"
```

## Introduction
OpenAI Gym is the toolkit that established the standard API for reinforcement learning environments. It provides a uniform interface for agents to interact with diverse simulated tasks, from classic control problems to Atari games, enabling researchers to benchmark algorithms on a common set of challenges. Its successor, Gymnasium (maintained by the Farama Foundation), continues active development.

## What OpenAI Gym Does
- Defines a universal environment API with reset(), step(), and render() methods
- Ships classic control environments like CartPole, MountainCar, and Pendulum
- Provides wrappers for Atari 2600 games via the Arcade Learning Environment
- Supports environment registration so third parties can add custom environments
- Enables reproducible benchmarking with seeded randomness and standardized reward structures

## Architecture Overview
Gym centers on the Env base class, which defines the observation_space, action_space, and the step/reset lifecycle. Environments are registered in a global registry and instantiated via gym.make(). Wrappers follow the decorator pattern, stacking transformations like frame skipping, reward clipping, or observation normalization on top of base environments without modifying them.

## Self-Hosting & Configuration
- Install via pip: pip install gym with optional extras like gym[atari] or gym[box2d]
- Requires Python 3.7+ with NumPy as the only hard dependency
- Atari environments need ROMs installed separately via AutoROM
- Custom environments are registered using gym.register() with an entry point
- For continued development, migrate to Gymnasium (pip install gymnasium) which maintains API compatibility

## Key Features
- Minimal, elegant API that became the de facto standard for RL research
- Over 100 built-in environments spanning classic control, toy text, and Atari games
- Wrapper system for composable environment modifications
- Space classes (Box, Discrete, MultiDiscrete, Dict) for describing observation and action domains
- Extensive third-party ecosystem with thousands of compatible environments

## Comparison with Similar Tools
- **Gymnasium** — the actively maintained successor to Gym by the Farama Foundation; use Gymnasium for new projects
- **PettingZoo** — extends the Gym API to multi-agent settings with turn-based and parallel interfaces
- **DeepMind Control Suite** — MuJoCo-based continuous control benchmarks; more physics-focused than Gym
- **Stable-Baselines3** — builds on Gym/Gymnasium to provide ready-made RL algorithm implementations

## FAQ
**Q: Should I use Gym or Gymnasium for new projects?**
A: Use Gymnasium. OpenAI Gym is archived and no longer receives updates. Gymnasium is API-compatible and actively maintained.

**Q: How do I create a custom environment?**
A: Subclass gym.Env, implement reset() and step(), define observation_space and action_space, then register it with gym.register().

**Q: Can Gym environments run in parallel?**
A: Gym provides VectorEnv wrappers that run multiple environment instances concurrently for faster data collection during training.

**Q: What is the difference between terminated and truncated?**
A: Terminated means the episode ended naturally (e.g., pole fell). Truncated means it was cut short by a time limit. This distinction was formalized in Gym v0.26+.

## Sources
- https://github.com/openai/gym
- https://www.gymlibrary.dev/

---
Source: https://tokrepo.com/en/workflows/asset-95c94a96
Author: AI Open Source