Applications
Note: 2026 update still under construction
Note: 2026 update still under construction
You can choose to center your project around the application of reinforcement learning to a real-world problem. Below are some suggestions.
You are allowed to come up with your own application, but it should have a public Gym repository, and it cannot be a game (since there is a separate Game AI master course, and we don't want to repeat the same course twice.)
If you focus on an application, we advise to stick to an estabilished (model-free) RL algorithm from the CleanRL library, such as PPO (Code).
Explanation:
A key question in (computational) chemistry is: what path(s) will a chemical reaction follow? This question has broad applicability, for example in sustainability.
Out of all possible reaction paths, the most likely one is named the 'minimal energy path' (MEP). With the correct (physics-inspired) reward formulation, we can use an RL agent to discover (explore) the most likely reaction path.
Possible papers:
Minimal Energy Path (MEP) discovery with RL. Thesis (to follow). Code.
This is a relatively novel direction, based on first steps taken by a thesis student last year (linked above). You could for example try to scale up those results to higher-dimensional problems.
As an extension, you could also try to integrate the above idea in the Jax-MD simulator, and 1) try different energy functions or 2) apply (entropy-regularized) RL to full molecular dynamics (MD) scenarios (where we simulate the distribution of reaction pathways).
Explanation:
A major solution to the global energy problem would be the development of nuclear fusion reactors, which (as opposed to nuclear fission) generated no harmful side products. However, controlling the fusion process is a challenging control problem.
Possible papers:
This is the Gym environment of the tokamak machine, which uses magnetic force to move plasma into a fusion state.
Note: the repository only contains the environment simulator. You would need to integrate an RL agent, for example from CleanRL, into it.
Magnetic control of tokamak plasmas through deep reinforcement learning. Paper.
This is the landmark paper that explains the use of RL for tokamak plasma control. It does not come with public code, so purely for explanation.
Explanation:
A major challenge in physics is the simulation of rigid-body dynamics: how do objects in the world move and bounce against eachother? This general understanding of the physics of our world is for example crucial for robotics applications.
Possible papers:
Kinetix is a fast (Jax-based) simulator of rigid-body physical interaction.
You could try to replicate the 'S' results (Fig 3, left).
Note that the standard set-up is very generic: you train on procedurally generated (random) tasks, to learn how the 'physics' of the world work. The agent is then tested on a set of specific test tasks. If this set-up is too computationally heavy, you could also first train and test on the same set of specific tasks.