Effect of Prior Probability Knowledge on Differential Privacy in Federated Learning

Differential privacy (DP) has been widely used in federated learning (FL) and distributed computing. DP is usually added in the data preparation stage before line coding and transmission. In this paper, we consider two DP mechanisms; namely, the Gaussian Mechanism (GM) and the Laplacian Mechanism (LM), and we derive the perturbation noise required for both mechanisms in case of the adversary having knowledge about the prior probabilities of the possible datasets. We show that the availability of the prior probabilities at the adversary is equivalent to reducing the tolerable privacy leakage, and hence it requires more perturbation noise. This, in turn, deteriorates the performance of FL. Furthermore, we derive a new lower bound on the perturbation noise required for the GM to guarantee (ϵ, δ)-DP. Although no-closed form is obtained for the new lower bound, a very simple one dimensional search algorithm can be used to achieve the lowest possible noise variance. Since the perturbation noise is known to negatively affect FL performance, including convergence and average loss, the new lower bound on the perturbation noise is expected to improve the performance over the classical GM.