Federated Learning with Differential Privacy: Gaussian Mechanism or Laplacian Mechanism?

Differential privacy (DP) has been widely used in communication systems, especially those using federated learning or distributed computing. DP comes in the data link layer before line coding and transmission. In this paper, we consider two DP mechanisms; namely, the Gaussian Mechanism (GM) and the Laplacian Mechanism (LM). We start by explaining why we have ?-DP if the LM is used, while we must have (?, d)-DP if the GM is used. Furthermore, we derive a new lower bound on the perturbation noise required for the GM to guarantee (?, d)- DP. Although no closed form is obtained for the new lower bound, a very simple one dimensional search algorithm can be used to achieve the lowest possible noise variance. Since the perturbation noise is known to negatively affect the performance of federated learning such as the convergence and the average loss, the new lower bound on the perturbation noise is expected to improve the performance over the classical GM. Moreover, we analytically derive the border between the region where GM is better to use and the region where LM is better to use.