RT Journal Article
T1 A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints
A1 Morell, José Ángel
A1 Abdelmoiz Dahi, Zakaria
A1 Chicano-García, José-Francisco
A1 Luque-Polo, Gabriel Jesús
A1 Alba-Torres, Enrique
K1 Informática
K1 Algoritmos genéticos
K1 Redes neuronales (Informática)
AB Federated learning is a paradigm that proposes protecting data privacy by sharing local models instead of raw data during each iteration of model training. However, these models can be large, with many parameters, provoking a substantial communication cost and having a notable environmental impact. Reducing communication overhead is paramount but conflictual to maintaining the model’s accuracy. Most research has dealt with the different factors influencing communication reduction separately without addressing their correlations. Moreover, most of them do not consider the heterogeneity of clients’ hardware. Finding the optimal configuration to fulfil all these training aspects can become intractable for classical techniques. This work explores the add-in that multi-objective evolutionary algorithms can provide for solving the communication overhead problem while achieving high accuracy. We do this by 1) realistically modelling and formulating this task as a multi-objective problem by considering the devices’ heterogeneity, 2) including all the communication-triggering aspects, and 3) applying a multi-objective evolutionary algorithm with an intensification operator to solve the problem. A simulated client–server architecture of four devices with four different processing speeds is studied. Both fully connected and convolutional neural network models are investigated with 33,400 and 887,530 weights, respectively. The experiments are performed using the MNIST and Fashion-MNIST datasets. A comparison is made between three approaches using an extensive set of metrics. Results prove that our approach obtains solutions with better accuracy than the full-communication setting and other methods while getting reductions in communications by around 1,000 times in most cases and up to 10,000 times in some cases compared to the maximum communication setting.
PB Elsevier
YR 2024
FD 2024-02-24
LK https://hdl.handle.net/10630/30831
UL https://hdl.handle.net/10630/30831
LA eng
NO José Ángel Morell, Zakaria Abdelmoiz Dahi, Francisco Chicano, Gabriel Luque, Enrique Alba, A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints, Future Generation Computer Systems, Volume 155, 2024, Pages 367-383, ISSN 0167-739X, https://doi.org/10.1016/j.future.2024.02.022
NO Funding for open access charge: Universidad de Málaga / CBUA. This research is funded by PID 2020-116727RB-I00 (HUmove) funded by MCIN/AEI/ 10.13039/501100011033, Spain; and TAILOR ICT-48 Network (No 952215) funded by EU Horizon 2020 research and innovation programme, Spain. The authors thank the Supercomputing and Bioinnovation Center (SCBI) for their provision of computational resources and technical support.
DS RIUMA. Repositorio Institucional de la Universidad de Málaga
RD 23 ene 2026