Applications

You can choose to center your project around the application of reinforcement learning to a real-world problem. There are some suggestions below.

If you focus on an application, we advise to stick to an estabilished (model-free) RL algorithm from the CleanRL library, such as PPO (Code).    

Chemistry:
Reaction Path Discovery

Explanation: 

• A key question in (computational) chemistry is: what path(s) will a chemical reaction follow? This question has broad applicability, for example in sustainability. 

• Out of all possible reaction paths, the most likely one is named the 'minimal energy path' (MEP). With the correct (physics-inspired) reward formulation, we can use an RL agent to discover (explore) the most likely reaction path.

Possible papers: 

• Minimal Energy Path (MEP) discovery with RL. Thesis (to follow). Code. 

• • This is a relatively novel direction, based on first steps taken by a thesis student last year (linked above). You could for example try to scale up those results to higher-dimensional problems. 

Physics:
Quantum Error Correction

Explanation: 

• Quantum computing is inherently sensitive to noise. Therefore, in practice, quantum computers repeat the same information over multiple physical qubits (a kind of 'repetition code') -- a form of redundancy. 

• During a quantum computation, we then need/want to continuously correct errors in the quantum information state, for which we may use reinforcement learning.  

Possible papers: 

• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation. Paper. Code. 

• Quantum error correction for the toric code using deep reinforcement learning. Paper. Code. 

• Internally developed QEC environment. Thesis (to follow). Code. 

Energy:
EV Charging

Explanation: 

• The challenge is to schedule arriving cars in the optimal way to charging sites. The best strategies depends on complex patterns in weather, car arrival distributions, customers rest time, battery charge, market prices, etc. 

Possible papers: 

• Chargax: A JAX Accelerated EV Charging Simulator. Paper. Github. 

• • Chargax is an internally developed, high-speed simulator for EV charging. 

  • You could for example try to learn a smart strategy for an EV-site battery under flexible grid pricing (when should this battery load from the grid & when should it discharge into cars).