A nice direction for the course is to apply reinforcement learning to a new application. This year we will primarily focus on reinforcement learning for sustainable energy.
If you focus on an application, we advise to pick a well-known RL algorithm from CleanRL and use that implementation.
Note: always make sure your algorithm matches the application (i.e., you cannot do multi-agent RL in a single-agent RL problem).
The transition to sustainable energy also involves the electrification of transport. However, car batteries take time to charge, and therefore the operation of a charging site poses a new decision-making problem: how much charge does each car get delivered? Optimal decisions depend on complex patterns in weather, car arrival distributions, customers rest time, battery charge, market prices, etc. Machine learning, especially reinforcement learning, could be a promising solution to solve this challenge.
Chargax (the linked paper) is an internally developed, high-speed simulator for EV charging. The course teachers can also easily provide additional support for this tool (since it's internally developed).
Production of sustainable energy is highly variable, since - for example - sun and wind are not always available. Therefore, the transition to sustainable energy sources also requires much storage capacity, such as batteries connected to the grid, which can absorb and deliver back energy. However, appropriately scheduling these batteries (when to absorb and deliver energy) is a major decision-making challenge, which is crucial to ensure the stability of our electricity grids.
RL-ADN (the linked paper) is a battery dispatch environment to test reinforcement learning solutions, developed at the TU Delft. We also have internal support from teachers actively working on RL for battery dispatch.