HumanAgent Interface

atari_env

Atari Environment

Integration of Atari 2600 games with the OpenEnv framework via the Arcade Learning Environment (ALE). ALE provides access to 100+ classic Atari games for RL research.

Supported Games

ALE supports 100+ Atari 2600 games including:

Popular Games

Pong - Classic two-player tennis

Breakout - Break bricks with a ball

Space Invaders - Shoot descending aliens

Pac-Man / Ms. Pac-Man - Navigate mazes and eat pellets

Asteroids - Destroy asteroids in space

Defender - Side-scrolling space shooter

Centipede - Shoot segmented centipede

Donkey Kong - Jump over barrels to save princess

Frogger - Cross road and river safely

Q*bert - Jump on pyramid cubes

Architecture

┌────────────────────────────────────┐
│ RL Training Code (Client)          │
│   AtariEnv.step(action)            │
└──────────────┬─────────────────────┘
               │ HTTP
┌──────────────▼─────────────────────┐
│ FastAPI Server (Docker)            │
│   AtariEnvironment                 │
│     ├─ Wraps ALEInterface          │
│     ├─ Handles observations        │
│     └─ Action execution            │
└────────────────────────────────────┘

Installation & Usage

Option 1: Local Development (without Docker)

Requirements:

Python 3.11+

ale-py installed: pip install ale-py

from envs.atari_env import AtariEnv, AtariAction

Start local server manually

python -m envs.atari_env.server.app


Connect to local server

env = AtariEnv(base_url="http://localhost:8000")

Reset environment

result = env.reset()
print(f"Screen shape: {result.observation.screen_shape}")
print(f"Legal actions: {result.observation.legal_actions}")
print(f"Lives: {result.observation.lives}")

Take actions

for _ in range(10):
    action_id = 2  # UP action
    result = env.step(AtariAction(action_id=action_id, game_name="pong"))
    print(f"Reward: {result.reward}, Done: {result.done}")
    if result.done:
        break

Cleanup

env.close()

Option 2: Docker (Recommended)

Build Atari image:

cd OpenEnv

Build the image

docker build \
  -f src/envs/atari_env/server/Dockerfile \
  -t atari-env:latest \
  .

Run specific games:

Pong (default)

docker run -p 8000:8000 atari-env:latest

Breakout

docker run -p 8000:8000 -e ATARI_GAME=breakout atari-env:latest

Space Invaders with grayscale observation

docker run -p 8000:8000 \
  -e ATARI_GAME=space_invaders \
  -e ATARI_OBS_TYPE=grayscale \
  atari-env:latest

Ms. Pac-Man with full action space

docker run -p 8000:8000 \
  -e ATARI_GAME=ms_pacman \
  -e ATARI_FULL_ACTION_SPACE=true \
  atari-env:latest

Use with from_docker_image():

from envs.atari_env import AtariEnv, AtariAction
import numpy as np

Automatically starts container

env = AtariEnv.from_docker_image("atari-env:latest")

result = env.reset()
result = env.step(AtariAction(action_id=2))  # UP

Reshape screen for visualization

screen = np.array(result.observation.screen).reshape(result.observation.screen_shape)
print(f"Screen shape: {screen.shape}")  # (210, 160, 3) for RGB

env.close()  # Stops container

Observation Types

1. RGB (Default)

Shape: [210, 160, 3]

Description: Full-color screen observation

Usage: Most realistic, good for vision-based learning

docker run -p 8000:8000 -e ATARI_OBS_TYPE=rgb atari-env:latest

2. Grayscale

Shape: [210, 160]

Description: Grayscale screen observation

Usage: Reduced dimensionality, faster processing

docker run -p 8000:8000 -e ATARI_OBS_TYPE=grayscale atari-env:latest

3. RAM

Shape: [128]

Description: Raw 128-byte Atari 2600 RAM contents

Usage: Compact representation, useful for specific research

docker run -p 8000:8000 -e ATARI_OBS_TYPE=ram atari-env:latest

Action Spaces

Minimal Action Set (Default)

Pong: 6 actions (NOOP, FIRE, UP, DOWN, etc.)

Breakout: 4 actions (NOOP, FIRE, LEFT, RIGHT)

docker run -p 8000:8000 -e ATARI_FULL_ACTION_SPACE=false atari-env:latest

Full Action Set

docker run -p 8000:8000 -e ATARI_FULL_ACTION_SPACE=true atari-env:latest

Configuration

Environment Variables

ATARI_GAME: Game name (default: "pong")

ATARI_OBS_TYPE: Observation type - "rgb", "grayscale", "ram" (default: "rgb")

ATARI_FULL_ACTION_SPACE: Use full action space - "true"/"false" (default: "false")

ATARI_MODE: Game mode (optional, game-specific)

ATARI_DIFFICULTY: Game difficulty (optional, game-specific)

ATARI_REPEAT_ACTION_PROB: Sticky action probability 0.0-1.0 (default: "0.0")

ATARI_FRAMESKIP: Frames to skip per action (default: "4")

Example: Breakout with Custom Settings

docker run -p 8000:8000 \
  -e ATARI_GAME=breakout \
  -e ATARI_OBS_TYPE=grayscale \
  -e ATARI_FULL_ACTION_SPACE=true \
  -e ATARI_REPEAT_ACTION_PROB=0.25 \
  -e ATARI_FRAMESKIP=4 \
  atari-env:latest

API Reference

AtariAction

@dataclass
class AtariAction(Action):
    action_id: int                  # Action index to execute
    game_name: str = "pong"         # Game name
    obs_type: str = "rgb"           # Observation type
    full_action_space: bool = False # Full or minimal action space

AtariObservation

@dataclass
class AtariObservation(Observation):
    screen: List[int]               # Flattened screen pixels
    screen_shape: List[int]         # Original screen shape
    legal_actions: List[int]        # Legal action indices
    lives: int                      # Lives remaining
    episode_frame_number: int       # Frame # in episode
    frame_number: int               # Total frame #
    done: bool                      # Episode finished
    reward: Optional[float]         # Reward from last action

AtariState

@dataclass
class AtariState(State):
    episode_id: str                      # Unique episode ID
    step_count: int                      # Number of steps
    game_name: str                       # Game name
    obs_type: str                        # Observation type
    full_action_space: bool              # Action space type
    mode: Optional[int]                  # Game mode
    difficulty: Optional[int]            # Game difficulty
    repeat_action_probability: float     # Sticky action prob
    frameskip: int                       # Frameskip setting

Example Script

#!/usr/bin/env python3
"""Example training loop with Atari environment."""

import numpy as np
from envs.atari_env import AtariEnv, AtariAction

Start environment

env = AtariEnv.from_docker_image("atari-env:latest")

Training loop

for episode in range(10):
    result = env.reset()
    episode_reward = 0
    steps = 0

    while not result.done:
        # Random policy (replace with your RL agent)
        action_id = np.random.choice(result.observation.legal_actions)

        # Take action
        result = env.step(AtariAction(action_id=action_id))

        episode_reward += result.reward or 0
        steps += 1

        # Reshape screen for processing
        screen = np.array(result.observation.screen).reshape(
            result.observation.screen_shape
        )

        # Your RL training code here
        # ...

    print(f"Episode {episode}: reward={episode_reward:.2f}, steps={steps}")

env.close()

Testing

Local Testing

Install dependencies

pip install ale-py fastapi uvicorn requests

Start server

cd /Users/sanyambhutani/OpenEnv/OpenEnv
export PYTHONPATH=/Users/sanyambhutani/OpenEnv/OpenEnv/src
python -m envs.atari_env.server.app

Test from another terminal

python -c "
from envs.atari_env import AtariEnv, AtariAction
env = AtariEnv(base_url='http://localhost:8000')
result = env.reset()
print(f'Initial obs: {result.observation.screen_shape}')
result = env.step(AtariAction(action_id=2))
print(f'After step: reward={result.reward}, done={result.done}')
env.close()
"

Docker Testing

Build and run

docker build -f src/envs/atari_env/server/Dockerfile -t atari-env:latest .
docker run -p 8000:8000 atari-env:latest

Test in another terminal

curl http://localhost:8000/health
curl -X POST http://localhost:8000/reset

Popular Games and Their Characteristics

Limitations & Notes

Frame perfect timing: Some games require precise timing

Exploration: Games like Montezuma's Revenge are notoriously difficult

Observation delay: HTTP adds minimal latency vs local gym

Determinism: Set ATARI_REPEAT_ACTION_PROB=0.0 for deterministic behavior

ROMs: All ROMs are bundled with ale-py package

References

[Arcade Learning Environment Paper (2013)](https://jair.org/index.php/jair/article/view/10819)

[ALE GitHub](https://github.com/Farama-Foundation/Arcade-Learning-Environment)

[ALE Documentation](https://ale.farama.org/)

[Gymnasium Atari Environments](https://gymnasium.farama.org/environments/atari/)

Citation

If you use ALE in your research, please cite:

@Article{bellemare13arcade,
    author = {{Bellemare}, M.~G. and {Naddaf}, Y. and {Veness}, J. and {Bowling}, M.},
    title = {The Arcade Learning Environment: An Evaluation Platform for General Agents},
    journal = {Journal of Artificial Intelligence Research},
    year = "2013",
    month = "jun",
    volume = "47",
    pages = "253--279",
}

Take Action

Current State

Status: Not initialized

Episode ID: -

Step Count: 0

atari_env

Atari Environment

Supported Games

Popular Games

Architecture

Installation & Usage

Option 1: Local Development (without Docker)

Start local server manually

python -m envs.atari_env.server.app

Connect to local server

Reset environment

Take actions

Cleanup

Option 2: Docker (Recommended)

Build the image

Pong (default)

Breakout

Space Invaders with grayscale observation

Ms. Pac-Man with full action space

Automatically starts container

Reshape screen for visualization