Key Takeaways & Executive Findings
- •• Provides the first systematic survey integrating knowledge distillation with financial large language models, addressing a critical research gap. • Introduces a structured taxonomy for distillation strategies and an upstream–midstream–downstream framework for financial applications. • Constructs a multi-dimensional evaluation framework for distilled FinLLMs, emphasizing financial accuracy, reasoning fidelity, and robustness. • Demonstrates KD's role in reducing deployment costs and inference latency, enabling practical FinLLM deployment in resource-constrained environments.
Abstract
Financial large language models (FinLLMs) offer immense potential for financial applications. While excessive deployment expenditures and considerable inference latency constitute major obstacles, as a prominent compression methodology, knowledge distillation (KD) offers an effective solution to these difficulties. A comprehensive survey is conducted in this work on how KD interacts with FinLLMs, covering three core aspects: strategy, application, and evaluation. At the strategy level, this review introduces a structured taxonomy to comparatively analyze existing distillation pathways. At the application level, this review puts forward a logical upstream–midstream–downstream framework to systematically explain the practical value of distilled models in the financial field. At the evaluation level, to tackle the absence of standards in the financial field, this review constructs a comprehensive evaluation framework that proceeds from multiple dimensions such as financial accuracy, reasoning fidelity, and robustness. In summary, this research aims to provide a clear roadmap for this interdisciplinary field, to accelerate the development of distilled FinLLMs.
1. Introduction
Over the past few years, substantial progress in natural language processing (NLP) has been achieved through the development of large language models (LLMs), with their powerful capabilities in contextual understanding, text generation (TG), and reasoning indicating broad application prospects across various industries (Zhao ZH et al., 2024). The financial industry, being highly dependent on information processing, analysis, and decision-making (DM), naturally becomes an important application scenario for LLM technology (Li YH et al., 2023). Financial large language models (FinLLMs) are trained or fine-tuned on general LLMs using specialized data from the financial domain, aiming to better understand financial terminology, capture market dynamics, and execute finance-specific tasks (Lee et al., 2025). The emergence of these models marks the rise of FinLLM research (Liu XY et al., 2023; Wu SJ et al., 2023; Xie et al., 2023; Zhang and Yang, 2023; Bhatia et al., 2024). These models have demonstrated application potential in various aspects, such as financial sentiment analysis, market prediction, quantitative trading, risk management (RM), report generation and summarization, and intelligent customer service (Raza et al., 2025).
However, the deployment of FinLLMs faces several major challenges. First, FinLLMs require high costs (Nie et al., 2024), and training a model such as BloombergGPT is estimated to cost millions of dollars. FinLLMs’ large size makes them difficult to deploy on standard hardware such as mobile devices or regular servers. Additionally, FinLLMs’ latency is a critical issue for tasks that need instant decisions, such as algorithmic trading executed at millisecond speeds. These challenges are the main roadblocks preventing FinLLMs from turning their potential into practical value.
Knowledge distillation (KD), recognized for its effectiveness in model compression and knowledge transmission, constitutes a fundamental strategy for resolving the difficulties that FinLLMs encounter in real-world applications (Acharya et al., 2024). Through a teacher–student paradigm (Li LJ et al., 2023), it enables smaller, simpler student models with fewer parameters to learn and inherit the key capabilities of large teacher models, thereby significantly reducing computational resource requirements, shortening inference time, and supporting deployment in resource-constrained environments. Consequently, KD technology can effectively bridge the gap between the powerful potential of FinLLMs and the practical implementation needs of the financial industry.
FinLLMs and KD are key research areas in artificial intelligence (AI). Li YH et al. (2023) and Nie et al. (2024) have published comprehensive surveys on FinLLMs, detailing their applications and challenges in financial tasks. Similarly, some researchers have provided thorough reviews on KD methods and applications (Xu XH et al., 2024; Yang CP et al., 2024). However, there is currently no systematic survey focusing on the integration of KD with FinLLMs. This paper addresses this gap by systematically investigating the synergy between KD and FinLLMs, offering a comprehensive survey to guide future research.
Loading authentic research manuscript (Pages 1–5)...
Jiaqi SHI, Xulong ZHANG, Xiaoyang QU, Junfei XIE, Jianzong WANG (2025). Knowledge distillation for financial large language models: a systematic review of strategies, applications, and evaluation. Frontiers of Information Technology & Electronic Engineering. https://doi.org/10.1631/FITEE_2500282
Research & Educational Purpose Only:The translations, structured abstracts, analytical annotations, and data reports provided by SinoTechIntel are intended exclusively for academic research, internal corporate R&D, and educational benchmarking. They do not constitute formal engineering, chemical safety, legal, or professional advice.
Copyright & Intellectual Property Notice: Original copyright of the underlying source articles and experimental data remains with the respective authors, institutions, and original publishing journals. SinoTechIntel claims intellectual property only over its proprietary translations, analytical syntheses, and AEO structured enhancements in accordance with international fair use and academic citation principles.
Frequently Asked Questions
What is the focus of this systematic review?
The review systematically investigates how knowledge distillation (KD) interacts with financial large language models (FinLLMs), covering strategies, applications, and evaluation.
Why is knowledge distillation important for FinLLMs?
KD reduces deployment costs and inference latency while preserving key capabilities, enabling efficient deployment in resource-constrained financial environments.
What framework is proposed for financial applications?
The paper introduces an upstream–midstream–downstream framework to explain the practical value of distilled models across the financial domain.
How are distilled FinLLMs evaluated?
A comprehensive evaluation framework is constructed, focusing on financial accuracy, reasoning fidelity, and robustness.
What gap does this paper address?
It is the first systematic survey integrating KD with FinLLMs, filling the absence of a focused review in this interdisciplinary field.
Related Technical Papers & Translations
Design and optimization of a high-efficiency distillation process for cellulosic fuel ethanol integrated with thermal coupling and molecular sieve adsorption
To address the challenges of high energy consumption and prominent costs in the traditional three-columns distillation process for cellulosic fuel ethanol, a distillation—molecular sieve coupling separation process is proposed. This process integrates a three-column (crude distillation column, first distillation column, second distillation column) system with a 3A molecular sieve adsorption deep dehydration unit. A thermal coupling network is constructed via differential pressure design (steam from medium/high-pressure columns as mutual heat sources, reboiler liquid waste heat for feed preheating), and molecular sieve adsorption conditions are optimized. The study first performs a thermodynamic consistency test on the ethanol—water system, determines optimal non-random two-liquid (NRTL) model binary interaction parameters via experimental data regression for Aspen Plus simulation. Aiming at minimum total annual cost (TAC), Aspen Plus is used to optimize process parameters (theoretical tray number, feed location, reflux ratio, side-draw position, etc.). Economic analysis shows this process reduces CO2 emission costs by 27.56%, TAC by 15.58% (to 5.123 × 106 USD·a-1), and increases ethanol purity to ＞99.6%, providing an effective solution for green, efficient separation.
A cohesion loss model for determining residual strength of deep bedded sandstone
Rock residual strength, as an important input parameter, plays an indispensable role in proposing the reasonable and scientific scheme about stope design, underground tunnel excavation and stability evaluation of deep chambers. Therefore, previous residual strength models of rocks established were reviewed. And corresponding related problems were stated. Subsequently, starting from the effects of bedding and whole life-cycle evolution process, series of triaxial mechanical tests of deep bedded s
Federated model with contrastive learning and adaptive control variates for human activity recognition
Recent attention to privacy issues demands a communication-safe method for training human activity recognition (HAR) models on client activity data. Federated learning (FL) has become a compelling technique to facilitate model training between the server and clients while preserving data privacy. However, classical FL methods often assume independent and identically distributed (IID) data among clients. This assumption does not hold true in practical scenarios. Human activity in real-world scena