Quantized FedPD (QFedPD): Beyond Conventional Wisdom - The Energy Benefits of Frequent Communication

Federated Averaging (FedAvg) is a well-recognized framework for distributed learning that efficiently manages communication. Several algorithms have emerged to enhance the communication efficiency of FedAvg and its variations. Some of these algorithms focus on reducing the number of communication rounds by allowing clients to skip frequent interactions with the parameter server. In this work, our primary concern is the overall energy consumption during model training in federated learning. We challenge the conventional notion that reducing the frequency of communication leads to energy savings and present evidence that for non-IID data distribution, increasing the frequency of communication can, in fact, result in greater energy conservation. Our contribution comprises two key aspects: Firstly, we introduce a quantized version of the recently proposed algorithm called Federated Primal-Dual (FedPD) 1, which we refer to as Quantized FedPD (QFedPD). Importantly, we substantiate the convergence guarantees for QFedPD. Secondly, we explore the trade-off between quantization and communication skipping in the proposed approach. Our analysis demonstrates that applying quantization without skipping communication, using QFedPD, yields the most significant energy-saving benefits for non-IID data distribution. Intriguingly, when dealing with non-IID data distribution, the preferred strategy is to maximize energy efficiency, allowing all clients to transmit at every iteration while quantizing their updates.