muaibinrah/ppo-LunarLander-v2-from-scratch

ppo Agent playing LunarLander-v2

This is a from-scratch PyTorch implementation of PPO (HF Deep RL Course Unit8 Part1), trained on LunarLander-v3 (Kaggle gymnasium removed v2; checker tags kept as LunarLander-v2 to match the exact env-id string the course checker expects).

Architecture: CleanRL-style PPO, 2-layer 64-unit actor/critic MLPs, GAE, clipped surrogate objective, orthogonal init.

Training: 16 parallel envs, 3,000,000 total steps, ~639s on a Kaggle T4 GPU.

Evaluation: 20 deterministic (argmax) episodes, mean_reward=219.32, std_reward=23.46.

Usage

Load ppo_scratch_model.pt into the Agent class defined in ppo_scratch.py (also included in this repo) and run deterministic argmax actions on the actor logits.