Wow this looks clean (no pun intended). Great speedups as well!

Clean RL is a great library if you're looking to get started doing some deep reinforcement learning! That plus gymnasium are pretty standard.

It's good for the world if we keep publishing improvements and optimizations to understandable primitives.

I am curious why not contribute back upstream, though.

Very cool! How does the optimized Pytoch code compare to the Jax implementation?

This looks awesome. CleanRL has been incredibly useful for some of my students starting out in RL. Adding Pytorch's compilation capabilities is a fantastic addition.