Applications

A nice direction for the course is to apply the algorithm in your paper to a new application. We will focus on two types of applications with real-world relevance: 1) energy management, and 2) quantum problems. 

• Gymnasium environment: Make sure that your application has a readily available Gymnasium environment. If the codebase of your paper also uses Gym, then you can simply swap out the environments. 

• Match paper and application: If you want to study a particular application, make sure the paper you choose matches with the application (e.g., you cannot do multi-agent RL in a single-agent problem). We indicate these matches below. 

Below you find relevant Gymnasium packages!

RL for Sustainable Energy

Building energy management

• Beobench: Codebase. Documentation. (white paper). This package integrates several packages for building energy management simulation: 

  • • BOPTEST: Codebase. Documentation. 

    • SinerGym: Codebase. Documentation. (white paper)

    • Energym: Codebase. Documentation. 

Possible algorithms: model-free RL, model-based RL, (transformers).

Electricity distribution networks

• • CityLearn: Codebase. Documentation. (white paper) ----- Multi-agent RL paper with code

  • Gym-ANM: Codebase. Documentation. 

  • PowerGridworld: Codebase. Documentation. (white paper) 

Possible algorithms: multi-agent RL (with multiple buildings), model-free RL. 

RL for Quantum

Quantum games

• Quantum Tiq-Taq-Toe: Example play. Code (you need to write a Gymnasium wrapper around this code)

Possible algorithms: model-free RL, (model-based RL, transformers).

RL for Astronomy

Telescope Imaging

• Adaptive Optics. Code. Documentation/explanation. Paper.

Possible algorithms: model-free RL, (model-based RL, transformers).