PufferLib is a fast and sane reinforcement learning library that can train tiny, super-human models in seconds. The included learning algorithm, hyperparameter tuning, and simulation methods are the product of our own research. All our tools are free and open source, but you can purchase priority service to get our eyes on your problem from $10k/month. We also offer fixed-deliverables terms for larger problems. Contact jsuarez🐡puffer🐡ai.

The demo below is running live 100% client side in your browser. Hold shift to take control!

Pong

A classic reimagined: Play against our reinforcement learned agent or watch AI vs AI matches. Running at 1M+ steps per second directly in your browser.

Citation

@misc{suarez2024pufferlibmakingreinforcementlearning,
      title={PufferLib: Making Reinforcement Learning
          Libraries and Environments Play Nice},
      author={Joseph Suarez},
      year={2024},
      eprint={2406.12905},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2406.12905},
}

Contributors

Joseph Suarez Founder & Head Puffer. Writes a lot of code.

David Rubinstein Several performance improvements w/ torch compilation, lead pokerl contributor.

Kyoung Whan Choe (최경환) Mujoco bindings, Testing and bug fixes.

Spencer Cheng Go, Connect4, TripleTriad, RWare, Tower Climb

Andrew LeFevre Impulse Wars

Nathan Lichtlé Tactics

Daniel Addis Enduro, testing, bug fixes, outreach, recruitment; major pokerl contributor.

Hadrien Crassous Tetris, freeway

Finlay Sanders Drone

Sam Turner Drone

Keelan Donovan Major pokerl contributor.

Gabe Pacman

Xander Trash Pickup

Noah Farr Breakout

Jake Forsey Connect4 rewrite with fast minmax AI opponent

David (dmoore101) Improved breakout physics

haterade Website design for demo page

arb8020 Website improvements for environment demos

David Bloomin CARBS integration improvements, 0.4 policy pool/store/selector

Black Ink South Character art for MOBA

Nick Jenkins Layout for the system architecture diagram. Adversary.design

Andranik Tigranyan Streamline and animate the pufferfish. Hire him on UpWork if you like what you see here.

Sara Earle Original pufferfish model. Hire her on UpWork if you like what you see here.