Epsilon: Privacy Metric for Machine Learning Models

• Main Authors: Negoescu, Diana M, Gonzalez, Humberto, Orjany, Saad Eddin Al, Yang, Jilei, Lut, Yuliia, Tandra, Rahul, Zhang, Xiaowen, Zheng, Xinyi, Douglas, Zach, Nolkha, Vidita, Ahammad, Parvez, Samorodnitsky, Gennady
• Format: Journal Article
• Language: English
• Published: 20-07-2023
• Subjects: Computer Science - Cryptography and Security; Computer Science - Data Structures and Algorithms; Computer Science - Learning
• DOI: 10.48550/arxiv.2307.11280

We introduce Epsilon*, a new privacy metric for measuring the privacy risk of
a single model instance prior to, during, or after deployment of privacy
mitigation strategies. The metric requires only black-box access to model
predictions, does not require training data re-sampling or model re-training,
and can be used to measure the privacy risk of models not trained with
differential privacy. Epsilon* is a function of true positive and false
positive rates in a hypothesis test used by an adversary in a membership
inference attack. We distinguish between quantifying the privacy loss of a
trained model instance, which we refer to as empirical privacy, and quantifying
the privacy loss of the training mechanism which produces this model instance.
Existing approaches in the privacy auditing literature provide lower bounds for
the latter, while our metric provides an empirical lower bound for the former
by relying on an (${\epsilon}$, ${\delta}$)-type of quantification of the
privacy of the trained model instance. We establish a relationship between
these lower bounds and show how to implement Epsilon* to avoid numerical and
noise amplification instability. We further show in experiments on benchmark
public data sets that Epsilon* is sensitive to privacy risk mitigation by
training with differential privacy (DP), where the value of Epsilon* is reduced
by up to 800% compared to the Epsilon* values of non-DP trained baseline
models. This metric allows privacy auditors to be independent of model owners,
and enables visualizing the privacy-utility landscape to make informed
decisions regarding the trade-offs between model privacy and utility.