Thermal PCM buffers with reinforcement learning: A framework for scalable smart building energy management

Buildings experience a sharp midday peak in cooling demand, straining the grid, increasing operational costs, and raising carbon emissions. This study introduces a phase change material thermal buffer (PCMTB) integrated into heating, ventilation, and air conditioning (HVAC) systems to exploit cooler nighttime ambient conditions, thereby enhancing latent heat thermal energy storage (LHTES) and improving coefficient of performance (COP). Typically, the HVAC system runs in idle mode overnight, missing the opportunity to utilize cooler ambient temperatures. To exploit this, the PCMTB can significantly activate the HVAC system during idle periods, thereby boosting LHTES and improving COP. Unlike conventional approaches, it combines unsupervised cluster-guided reinforcement learning (UCGRL) with rule-based control, refined through deep clustering of Lagrangian trajectories, enabling adaptive, real-time load shifting. The results showed over 23% energy savings, reduced peak loads, and improved system responsiveness, highlighting the potential of PCMTB and intelligent control strategies for scalable, sustainable building energy management. These results are attributed to the increased COP achieved by the integrated PCMTB, which is designed to operate in tandem with a UCGRL for the HVAC system.