Key Takeaways & Executive Findings
- •• Addresses the dual challenge of generalization and personalization in cross-silo federated learning (GPFL) under domain shift. • Proposes FFT-GP, which integrates fairness-aware aggregation (FAA) to minimize generalization gap variance among clients. • Employs a meta-learning strategy to align local training with the global model’s feature distribution, balancing generalization and personalization. • Demonstrates superior efficacy over existing methods, enhancing FL systems in practical cross-silo scenarios.
Abstract
Abstract: Cross-silo federated learning (FL), which benefits from relatively abundant data and rich computing power, is drawing increasing focus due to the significant transformations that foundation models (FMs) are instigating in the artificial intelligence field. The intensified data heterogeneity issue of this area, unlike that in cross-device FL, is caused mainly by substantial data volumes and distribution shifts across clients, which requires algorithms to comprehensively consider the personalization and generalization balance. In this paper, we aim to address the objective of generalized and personalized federated learning (GPFL) by enhancing the global model’s cross-domain generalization capabilities and simultaneously improving the personalization performance of local training clients. By investigating the fairness of performance distribution within the federation system, we explore a new connection between generalization gap and aggregation weights established in previous studies, culminating in the fairness-guided federated training for generalization and personalization (FFT-GP) approach. FFT-GP integrates a fairness-aware aggregation (FAA) approach to minimize the generalization gap variance among training clients and a meta-learning strategy that aligns local training with the global model’s feature distribution, thereby balancing generalization and personalization. Our extensive experimental results demonstrate FFT-GP’s superior efficacy compared to existing models, showcasing its potential to enhance FL systems across a variety of practical scenarios.
1. Introduction
Federated learning (FL) has recently emerged as a prominent privacy-preserving paradigm for collaborative learning on distributed data (McMahan et al., 2017) in data-sensitive domains such as health care (Haque et al., 2020; Rieke et al., 2020; Xu A et al., 2022) and finance (Kairouz et al., 2021; Zhu et al., 2021). However, existing FL (Zhao et al., 2018; Karimireddy et al., 2020; Li T et al., 2020c; Li X et al., 2020; Wang et al., 2020) predominantly focuses on the cross-device scenario, characterized by numerous clients each possessing limited data, computing power, and communication capabilities. Consequently, data heterogeneity issues are often observed in data across all clients whose distributions follow a uniform meta simplex (Zhao et al., 2018; Li X et al., 2020). Algorithms targeting data heterogeneity have solely concentrated on convergence on the global distribution (Karimireddy et al., 2020; Li T et al., 2020c) or personalization on the training clients (Smith et al., 2017; Arivazhagan et al., 2019; Oh et al., 2022), not both.
In recent years, the cross-silo scenario (Huang YT et al., 2021; Huang C et al., 2022) has garnered increased attention, with notable applications in medical image analysis (Xu A et al., 2022) and autonomous driving (Chu et al., 2021; Liu KZ et al., 2022), among others (du Terrail et al., 2022). Unlike the cross-device scenario, the cross-silo framework involves significant data volumes at each client, with each one possessing independent data distributions and sufficient computational and communication resources, as illustrated in Fig. 1. Here, the primary form of data heterogeneity transitions to domain or distribution shifts (Khosla et al., 2012; Cohen et al., 2020; Zhang HR et al., 2021) among clients, necessitating a dual focus on enhancing the generalization of the global model and the personalization of local models. This paradigm is thus also called generalized and personalized federated learning (GPFL) (Jiang et al., 2023; Lu et al., 2023; Zhang RP et al., 2023b). Meanwhile, the substantial computational power and data abundance in the cross-silo setting pave the way for adapting foundation models (FMs) within the FL framework. FMs like GPT-4 (Achiam et al., 2023) and contrastive language-image pre-training (CLIP) (Radford et al., 2021) have marked significant advancements, revolutionizing artificial intelligence (AI) research and applications. However, FMs are trained solely with large-scale open-source data available on the Internet...
Loading authentic research manuscript (Pages 1–5)...
Ruipeng ZHANG, Ziqing FAN, Jiangchao YAO, Ya ZHANG, Yanfeng WANG (2025). Fairness-guided federated training for generalization and personalization in cross-silo federated learning. Frontiers of Information Technology & Electronic Engineering. https://doi.org/10.1631/FITEE_2400279
Research & Educational Purpose Only:The translations, structured abstracts, analytical annotations, and data reports provided by SinoTechIntel are intended exclusively for academic research, internal corporate R&D, and educational benchmarking. They do not constitute formal engineering, chemical safety, legal, or professional advice.
Copyright & Intellectual Property Notice: Original copyright of the underlying source articles and experimental data remains with the respective authors, institutions, and original publishing journals. SinoTechIntel claims intellectual property only over its proprietary translations, analytical syntheses, and AEO structured enhancements in accordance with international fair use and academic citation principles.
Frequently Asked Questions
What is cross-silo federated learning?
Cross-silo federated learning is a federated learning paradigm where a small number of clients (e.g., organizations) each possess large amounts of data and ample computational resources, enabling collaborative training while preserving privacy.
How does FFT-GP improve generalization and personalization?
FFT-GP uses fairness-aware aggregation to minimize variance in the generalization gap across clients, and a meta-learning strategy to align local training with the global model's feature distribution, thereby balancing cross-domain generalization and local personalization.
What is fairness-aware aggregation (FAA)?
Fairness-aware aggregation is a method that adjusts aggregation weights to reduce the variance of generalization gaps among clients, promoting a fairer performance distribution across the federation.
Why is meta-learning used in FFT-GP?
Meta-learning helps local models adapt efficiently to new tasks by aligning their training process with the global model's feature distribution, improving both generalization and personalization under domain shifts.
What are the main experimental findings of the paper?
Extensive experiments show that FFT-GP achieves superior performance compared to existing federated learning models, demonstrating its potential to enhance FL systems across various practical cross-silo scenarios.
Related Technical Papers & Translations
Design and optimization of a high-efficiency distillation process for cellulosic fuel ethanol integrated with thermal coupling and molecular sieve adsorption
To address the challenges of high energy consumption and prominent costs in the traditional three-columns distillation process for cellulosic fuel ethanol, a distillation—molecular sieve coupling separation process is proposed. This process integrates a three-column (crude distillation column, first distillation column, second distillation column) system with a 3A molecular sieve adsorption deep dehydration unit. A thermal coupling network is constructed via differential pressure design (steam from medium/high-pressure columns as mutual heat sources, reboiler liquid waste heat for feed preheating), and molecular sieve adsorption conditions are optimized. The study first performs a thermodynamic consistency test on the ethanol—water system, determines optimal non-random two-liquid (NRTL) model binary interaction parameters via experimental data regression for Aspen Plus simulation. Aiming at minimum total annual cost (TAC), Aspen Plus is used to optimize process parameters (theoretical tray number, feed location, reflux ratio, side-draw position, etc.). Economic analysis shows this process reduces CO2 emission costs by 27.56%, TAC by 15.58% (to 5.123 × 106 USD·a-1), and increases ethanol purity to ＞99.6%, providing an effective solution for green, efficient separation.
A cohesion loss model for determining residual strength of deep bedded sandstone
Rock residual strength, as an important input parameter, plays an indispensable role in proposing the reasonable and scientific scheme about stope design, underground tunnel excavation and stability evaluation of deep chambers. Therefore, previous residual strength models of rocks established were reviewed. And corresponding related problems were stated. Subsequently, starting from the effects of bedding and whole life-cycle evolution process, series of triaxial mechanical tests of deep bedded s
Federated model with contrastive learning and adaptive control variates for human activity recognition
Recent attention to privacy issues demands a communication-safe method for training human activity recognition (HAR) models on client activity data. Federated learning (FL) has become a compelling technique to facilitate model training between the server and clients while preserving data privacy. However, classical FL methods often assume independent and identically distributed (IID) data among clients. This assumption does not hold true in practical scenarios. Human activity in real-world scena