and privacy enhancing technologies or PETs for short. You'll gain some intuition. Learn how to think about PETs in the context of federated learning and dive into differential privacy as an example. Let's get to it. When we look at federated learning through the lens of privacy, we can say that federated learning itself serves as a data minimization solution. By preventing direct access to data. However, the model updates exchange between client and server can still potentially lead to privacy leaks. There are different possible attacks that need to be considered depending on the attack model and the adversaries. Role in federated learning. Various privacy attacks can be constructed. The adversary can be the client, it can be the server, or it can be a third party. Let's look at three examples. A membership inference attack aims to infer the participation of data samples. An attribute inference attack aims at inferring unseen attributes of the training data, and a reconstruction attack aims to infer specific training data samples. For example, researchers have shown in a paper how in one particular setting, a malicious server was able to reconstruct training data samples of a specific client in federated learning. You can see that the reconstructed images are not the exact same images, but also that the quality of reconstructed images is surprisingly close to the original data. Differential privacy is a prominent solution to enhance privacy for individuals during a data analysis. It obscures individual data by adding calibrated noise to query results, which ensures that the presence or absence of any single data point does not significantly impact the outcome of the analysis. This ensures accurate analysis without compromising sensitive information. So let's say you have two datasets, D and Dr that differ in only one data point. Alice. In this example, differential privacy guarantees that any analysis m like calculating the average income, for example, will produce nearly identical results for both datasets. O and O Prime would be similar in this case. Differential privacy in machine learning would provide us with the guarantee that if we train a model M1 on dataset D and then add or remove a data point such as Alice's data in this example, and we train a second model, M2, the resulting models M1 and M2 will be indistinguishable, capable to a certain degree. This degree of indistinguishable ability is quantified by the level of privacy protection we aim to achieve. When we look at differential privacy, or DP for short, in the context of federated learning, we see that it can be applied at various stages of the process, including during model training. Aggregation of model updates and communication between clients and the server. Depending on its application, DP provides us with different levels of privacy. In this lesson, you will learn about two variants of differential privacy. Central DP and local DP. There are two important topics regarding DP one is clipping, which bounds the sensitivity and mitigate the impact of outliers. Sensitivity in this context means the maximum amount that the output can change when a single data point is added or removed from the dataset. And the other topic is noisy, which adds calibrated noise to make the output statistically indistinguishable. In central differential privacy, the central server is responsible for adding noise to the globally aggregated parameters. It should be noted that trust in the server is required. The overall approach is to clip the model updates sent by the clients, and then add some amount of noise to the aggregated model and local differential privacy. Each client is responsible for performing DP. Local DP avoids the need for a fully trusted aggregator. Each client is responsible for performing clipping and noise locally before sending the updated model to the server. Let's jump into the lab. As usual, we start by importing utility functions and classes. We also import both a server side differential privacy strategy called differential privacy, client side adaptive clipping, and a client side adaptive clipping mode called adaptive clipping underscore mode. The go into what these components do in a minute. Like in the previous lesson, we load the MNIST dataset and partition it into ten parts using Flower datasets. We also defined a Flower client. Like in previous lessons, we defined a client underscore fn function that initializes train loader, test loader, and Flower client. When defining the client app, we use a new feature called Flower mods. Mods, also known as modifiers, allow you to perform operations before and after a task is processed. In the client app, you can use built in mods or even define your own custom mods. Here, you use the Adaptive Clipping mod that performs adaptive clipping of model updates before sending them back to the server. On the server side, you start by creating the federated averaging strategy as usual. This time we'll give it a different name. We'll call it federating without DP. Instead of passing our federated averaging strategy object directly to server app wrapped a strategy in a wrapper strategy called differential privacy. Client side adaptive clipping. To do that, you create an instance of differential privacy, client side adaptive clipping, and you pass it the previously created strategy object, along with two DP specific arguments the noise multiplier and the number of sampled clients. The differential privacy client side adaptive clipping strategy wrapper is a strategy itself. It wraps other strategies and is responsible for applying DP on the server side. This means it receives model updates, it forwards them to the inner strategy for aggregation, and then adds noise to the aggregated model. The last step is to create server app as usual, and this time you pass You now have a matching server app and client app that can perform central DP with client side adaptive clipping. Let's run it. You now have a matching server app and client app that together can perform central DP with client side adaptive clipping with both our server side DP strategy and our client side clipping mode configured, we can see how the federate training runs with DP. After local training, a client side mode clips the parameters and send the clip model updates to the server. The inner strategy then aggregates those model updates and the DP wrapper strategy adds noise to the aggregate model. You can run this experiment as usual. While run simulation. This time you run the simulation with ten simulated clients, and you select six of those clients in every single round. This is what fraction fit equals 0.6 results in. The logs are a bit more noisy now, but what you can see is that on the client side the mode clips the parameters. You can see some logs starting with adaptive clipping underscore mods, and it says parameters are clipped by a certain value. adds noise with a certain standard deviation. You can see that in some of the logs that start with aggregate underscore fit central DP noise with a certain standard deviation is added because of the clipping a noisy DP often leads to slower convergence. This is why for this experiment, we use a small noise multiplier which results in less privacy, and we run the federated training for 50 rounds and preferably more with regard to the desired privacy and utility. Let's review lesson 4. Federated learning by itself does not guarantee data privacy. Privacy enhancing technologies are often referred to as PETs for short, like differential privacy and secure aggregation can help. Differential privacy either central or local works by clipping gradients and adding noise. Such privacy enhancements often come with costs like reduced utility and computational overhead.