Federe Makine Öğrenmesi: Kavramlar, Zorluklar, Yöntemler
Özet
The Federated Learning (FL) research, led by MC Mahan and his team, was first introduced at Google in 2016. Since then, this field has been increasing interest and ongoing research. In the client-server model used, the clients are devices that hold data and are referred to as participants. The goal is to train machine learning models using the data on the participating clients. The trained model is then used again on the clients. One approach is to have the participants send their data to a central server, where the training is performed using the entire data, and then share the trained model with the participants. This approach follows the traditional centralized machine learning paradigm. In federated learning, however, the participants do not send their raw data to the central server. Instead, the server and participants collaborate to ensure the data remains on the devices. With advancing technology, applications where data resides on edge participants rather than a central server, are increasing, and the size of data on edge participants is also growing. The cost of transferring large amounts of data from the edge to the center and the privacy concerns associated with sharing personal data highlight the importance of federated learning for future systems and make it a widely studied research area. In federated learning, the clients train the received models from the server using their data on their own devices. The newly generated models are then aggregated on the server and redistributed to the devices.
This thesis examines and explains the fundamentals of federated machine learning. The critical challenges of federated learning, such as system heterogeneity, statistical heterogeneity, and data sparsity, are addressed, and their impacts are studied. Developing new aggregation methods to overcome these challenges is a current topic in the literature. In this thesis, three of these methods, FEDSGD, FEDAVGM, and FEDAVG, are investigated.
Experiments are conducted to examine federated learning and the effects of parameter changes on the system. In these experiments, the impact of system heterogeneity, statistical heterogeneity, and data sparsity on accuracy, loss, and computation time performance is explicitly analyzed for the FEDAVGM method. Different methods exist to control the rate of parameter changes in each iteration. For this purpose, two different methods, FEDAVGM and FEDAVG, are compared in the experiments.
The results show that selecting learning parameters can significantly affect the system's performance. Overfitting tendencies can arise when the parameter values are not chosen appropriately for the situation. It is observed through the experiments that system heterogeneity prolongs the learning time, data sparsity introduces variations across devices and increases computation time, and statistical heterogeneity reduces learning and increases oscillations. In specific cases, FEDAVGM has shown better performance, relying on past gradient calculations with momentum.
