PufferLib is the reinforcement learning library I wish existed during my PhD. It started as a compatibility layer to make working with complex environments a breeze. Now, it's a high-performance toolkit for research and industry with optimized parallel simulation, environments that run and train at 1M+ steps/second, and tons of quality of life improvements for practitioners. All our tools are free and open source. We also offer priority service for companies, startups, and labs!

The demo below is running live 100% client side in your browser. Hold shift to take control!

Pong

A classic reimagined: Play against our reinforcement learned agent or watch AI vs AI matches. Running at 1M+ steps per second directly in your browser.

Citation

@misc{suarez2024pufferlibmakingreinforcementlearning,
      title={PufferLib: Making Reinforcement Learning
          Libraries and Environments Play Nice},
      author={Joseph Suarez},
      year={2024},
      eprint={2406.12905},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2406.12905},
}

Contributors

Joseph Suarez Founder & Head Puffer. Writes a lot of code.
thatguy Several performance improvements w/ torch compilation, major pokerl contributor.
Kyoung Whan Choe (최경환) Mujoco bindings, Testing and bug fixes.
Spencer Cheng Go, Connect4, TripleTriad, RWare
Nathan Lichtlé Tactics
Daniel Addis Enduro, testing, bug fixes, outreach, recruitment; major pokerl contributor.
Leanke Major pokerl contributor.
Xander Trash Pickup
Noah Farr Breakout
Jake Forsey Connect4 rewrite with fast minmax AI opponent
David (dmoore101) Improved breakout physics
haterade Website design for demo page
arb8020 Website improvements for environment demos
David Bloomin CARBS integration improvements, 0.4 policy pool/store/selector
Black Ink South Character art for MOBA
Nick Jenkins Layout for the system architecture diagram. Adversary.design
Andranik Tigranyan Streamline and animate the pufferfish. Hire him on UpWork if you like what you see here.
Sara Earle Original pufferfish model. Hire her on UpWork if you like what you see here.