Gymnasium — Standard API for Reinforcement Learning Environments

Introduction

Gymnasium is the maintained successor to OpenAI Gym, managed by the Farama Foundation. It provides a standard Python API that every reinforcement learning framework depends on for defining, running, and benchmarking single-agent environments.

What Gymnasium Does

• Defines a universal Env interface (reset, step, render, close) adopted by the entire RL ecosystem
• Ships dozens of built-in environments: classic control, toy text, Box2D, MuJoCo, and Atari via plugins
• Provides wrappers for observation/reward/action transformations without editing environment code
• Supports vectorized environments for parallel rollout collection
• Offers a registration system so third-party environments plug in with a single gym.make() call

Architecture Overview

Every environment subclasses gymnasium.Env and implements reset() and step(action). Spaces (Box, Discrete, Dict, etc.) describe observation and action shapes. Wrappers stack transparently around any env, and gymnasium.vector provides SyncVectorEnv and AsyncVectorEnv for batched simulation. The registry maps string IDs to entry points, allowing lazy imports of heavy dependencies.

Self-Hosting & Configuration

• Install from PyPI: pip install gymnasium or with extras like gymnasium[atari], gymnasium[mujoco]
• No server or daemon needed; it is a pure Python library
• Set render_mode="human" for on-screen visualization or "rgb_array" for headless rendering
• Configure wrappers via gymnasium.wrappers (TimeLimit, RecordVideo, FlattenObservation, etc.)
• Register custom environments with gymnasium.register(id="MyEnv-v0", entry_point="my_module:MyEnv")

Key Features

• Drop-in replacement for OpenAI Gym with a cleaner, fully typed API
• Active maintenance with regular releases and Python 3.8+ support
• Wrappers ecosystem for reward shaping, frame stacking, and auto-reset
• Compatibility shims (gymnasium.utils.env_checker) to validate custom envs
• First-class support in Stable Baselines3, CleanRL, RLlib, and most RL libraries

Comparison with Similar Tools

• OpenAI Gym — original project, now deprecated; Gymnasium is the official continuation
• PettingZoo — multi-agent environments with a similar API, also by Farama Foundation
• DeepMind Control Suite — MuJoCo-based continuous control, narrower scope
• Brax — JAX-accelerated physics environments for fast parallel simulation
• Unity ML-Agents — 3D game-engine environments, heavier setup

FAQ

Q: Is Gymnasium backward-compatible with OpenAI Gym code? A: Mostly yes. The API added explicit truncation handling and typing improvements, but a compatibility wrapper exists.

Q: How do I use Atari environments? A: Install gymnasium[atari] and gymnasium[accept-rom-license], then gym.make("ALE/Pong-v5").

Q: Can I use Gymnasium with JAX or PyTorch directly? A: Gymnasium itself is NumPy-based, but gymnasium.experimental.wrappers.JaxToNumpyV0 and community bridges exist.

Q: Where did the name come from? A: It keeps the Gym metaphor while indicating it is a separate, actively maintained project under the Farama Foundation.

Sources

• https://github.com/Farama-Foundation/Gymnasium
• https://gymnasium.farama.org/