Battery energy storage systems (BESS) in microgrids require dynamic control to handle uncertainty and variability in load, photovoltaic (PV), and wind generation. Assigning grid-support functions to the BESS further increases the control complexity. With large-scale renewable integration, electricity market price volatility has intensified, underscoring the need for operational strategies that adapt to real-time microgrid conditions while ensuring long-term profitability. Under such uncertainty, reinforcement learning (RL) can outperform conventional deterministic optimization. This study applies an RL-based controller to the optimal operation of a microgrid BESS.

We develop a Proximal Policy Optimization (PPO) controller that jointly optimizes local load supply and bidding in both the Real-Time Market and balancing markets. Compared with PPO without forecasts, augmenting the state with LightGBM-based PV generation forecasts increases total revenue.

We implement Python-based simulations of a microgrid comprising load and PV, where the BESS performs supply–demand balancing (same-time, same-quantity control) while participating in Real-Time Market and balancing markets. Market-based revenue is formulated as the reward under constraints such as state-of-charge (SOC) limits and power/energy capacity, and provided to PPO for policy learning. A PV forecasting model using Light GBM (Light Gradient Boosting Machine) leverages temperature and weather features; its forecasts are incorporated into the PPO state. We evaluate the effect of including PV forecasts versus PPO alone from a revenue perspective.

Prior applications of machine learning to BESS operation—especially in microgrids—are limited. Related EV studies often rely solely on real-time inputs; integrating forecasting with RL offers both novelty and practical value.

PPO‑based control is feasible for revenue‑maximizing operation of microgrid BESS. By incorporating PV output forecasts, cases with and without forecasts were compared, and the influence of adding PV predictive information on the RL‑based market‑participating microgrid control was confirmed.