Key Takeaways & Executive Findings
- •• A novel energy management strategy based on Double Deep Q-Learning (DDQL) is developed for fuel cell hybrid electric buses, incorporating fuel cell power change rate as action variables to respect power response characteristics. • The proposed DDQL-based EMS achieves a 15.4% improvement in fuel economy over a rule-based strategy under training scenarios, and 13.3% under testing scenarios, demonstrating strong generalization. • Comparative analysis with dynamic programming (DP) and rule-based strategies validates the effectiveness of the DDQL approach in balancing fuel economy and real-time applicability. • The study addresses the high operational costs of fuel cell systems by optimizing energy management, contributing to the economic viability of fuel cell hybrid electric vehicles.
Abstract
With increasingly serious environmental pollution and the energy crisis, fuel cell hybrid electric vehicles have been considered as an ideal alternative to traditional hybrid electric vehicles. Nevertheless, the total costs of fuel cell systems are still too high, thus limiting the further development of fuel cell hybrid electric vehicles. This paper presents an energy management strategy (EMS) based on deep reinforcement learning for the energy management of fuel cell hybrid electric vehicles. The energy management model of a fuel cell hybrid electric bus and its main components are established. Considering the power response characteristics of the fuel cell system, the power change rate of the fuel cell system is reasonably limited and introduced as action variables into the network of Double Deep Q-Learning (DDQL), and a novel DDQL-based EMS is developed for the fuel cell hybrid electric bus. Subsequently, a comparative test is conducted with the DP-based and the Rule-based EMS to analyze the performance of the DDQL-based EMS. The results indicate that the proposed EMS achieves good fuel economy performance, with an improvement of 15.4% compared to the Rule-based EMS under the training scenarios. In terms of generalization performance, the proposed EMS also achieves good fuel economy performance, which improves by 13.3% compared to the Rule-based energy management strategy under the testing scenario.
1. Introduction
With increasingly serious environmental pollution and the energy crisis, fuel cells as an environmentally friendly energy source have become an important direction for development in various countries. Fuel cell hybrid electric vehicles (FCHEVs) have also been considered as an ideal alternative to traditional hybrid electric vehicles [1]. Nevertheless, the total costs of fuel cell systems (FCSs) are still too high, thus limiting the further development of FCHEVs [2, 3]. Specifically, due to the relatively high price of hydrogen, the costs of hydrogen consumption in vehicular operation account for a significant share of total FCHEVs’ ownership costs [4]. Improving the economics of FCSs have become a focus of the research on FCHEVs. Since FCHEVs contain two or more power sources, energy management is the core component of FCHEVs to achieve energy conservation [5].
The existing energy management strategies (EMSs) for hybrid propulsion systems can be divided into three categories: rule-based, optimization-based, and machine learning-based strategies [6, 7]. Rule-based EMSs have been widely used in practical engineering due to their reliability, but they all lack a mathematical guarantee for control optimality and are highly dependent on engineering experience [8]. Huang et al. proposed an EMS combining a logic threshold approach and an instantaneous optimization algorithm for a parallel hybrid electrical urban bus, and the results indicated that the proposed control strategy improved the fuel economy significantly [9]. Yuan et al. developed a practical rule-based EMS with multi-objective optimization capability in an FCHEV system, realizing a real-time optimization for similar driving pattern [10]. The optimization-based strategies adjust the control variables by minimizing the predefined cost function under feasible constraints. Typical optimization-based EMSs include Equivalent Consumption Minimization Strategy, Model Predictive Control, Dynamic Programming (DP) algorithm [11, 12]. DP is a general optimization solver providing a global optimal solution for a variety of control problems [13]. Vahidi et al. first applied Model Predictive Control strategy for FCHEVs to prevent insufficient or saturation of hydrogen supply [14, 15]. Ma et al. proposed a multi-objective predictive EMS based on Model Predictive Control, combined with velocity forecast and driving pattern recognition, and the results illustrate that could reduce fuel consumption by 6.67%, and avoid FCS degradation [16]. Sun et al. proposed a type of power-balancing instantaneous optimization energy management strategy based on Equivalent Consumption Minimization Strategy for a series-parall
Loading authentic research manuscript (Pages 1–5)...
Ruoyan Han, Hongwen He, Yaxiong Wang, Yong Wang (2025). Reinforcement Learning Based Energy Management Strategy for Fuel Cell Hybrid Electric Vehicles. Chinese Journal of Mechanical Engineering. https://doi.org/10.1186/s10033-024-01143-0
Research & Educational Purpose Only:The translations, structured abstracts, analytical annotations, and data reports provided by SinoTechIntel are intended exclusively for academic research, internal corporate R&D, and educational benchmarking. They do not constitute formal engineering, chemical safety, legal, or professional advice.
Copyright & Intellectual Property Notice: Original copyright of the underlying source articles and experimental data remains with the respective authors, institutions, and original publishing journals. SinoTechIntel claims intellectual property only over its proprietary translations, analytical syntheses, and AEO structured enhancements in accordance with international fair use and academic citation principles.
Frequently Asked Questions
What is the main contribution of this paper?
The paper proposes a novel energy management strategy based on Double Deep Q-Learning (DDQL) for fuel cell hybrid electric vehicles, which incorporates the fuel cell power change rate as action variables to respect power response characteristics, achieving significant fuel economy improvements over rule-based strategies.
How does the proposed DDQL-based EMS compare to other strategies?
The proposed DDQL-based EMS outperforms rule-based strategies by 15.4% in training scenarios and 13.3% in testing scenarios, and shows competitive performance compared to dynamic programming (DP) while being more suitable for real-time implementation.
What are the key components of the energy management model?
The model includes a fuel cell hybrid electric bus and its main components, such as the fuel cell system, battery, and electric motor, with the power change rate of the fuel cell system limited and introduced as action variables in the DDQL network.
Why is energy management important for fuel cell hybrid electric vehicles?
Energy management is crucial because it directly affects fuel economy and operational costs, which are major barriers to the widespread adoption of FCHEVs due to high hydrogen prices. Effective EMS can reduce hydrogen consumption and improve overall efficiency.
What are the future research directions suggested by this study?
The study suggests that further improvements could be made by exploring other deep reinforcement learning algorithms, incorporating more complex driving scenarios, and considering degradation of fuel cell systems to extend their lifespan.
Related Technical Papers & Translations
Direct Repair of the Crystal Structure and Coating Surface of Spent LiFePO4 Materials Enables Superfast Li-Ion Migration
The rapid accumulation of spent LiFePO4 (LFP) cathodes from retired lithium-ion batteries necessitates the development of effective and environmental-friendly recycling strategies. In this context, direct regeneration has emerged as a promising approach for reclaiming LFP cathode materials, offering a streamlined pathway to restore their electrochemical functionality. We report an integrated regeneration protocol that simultaneously repairs the degraded crystal structure and reconstructs the damaged carbon coating in spent LFP. The regenerated cathode material had superfast lithium-ion diffusion kinetics and a stable cathode–electrolyte interface, giving a remarkable rate capability with specific capacities of 122 mAh g−1 at 5C and 106 mAh g−1 at 10C (1C = 170 mA g−1). It also maintained capacities of 110.7 mAh g−1 (5C) and 84.1 mAh g−1 (10C) after 400 cycles. It could be used in harsh environments and could be stably cycled at subzero temperatures (−10 and −20 °C) and in solid-state electrolyte batteries. Life cycle assessment combined with economic evaluation using the EverBatt model reveals that this direct regeneration approach has high economic and environmental benefits.
Oxide Semiconductor for Advanced Memory Architectures: Atomic Layer Deposition, Key Requirement and Challenges
Oxide semiconductors (OSs), introduced by the Hosono group in the early 2000s, have evolved from display backplane materials to promising candidates for advanced memory and logic devices. The exceptionally low leakage current of OSs and compatibility with three-dimensional (3D) architectures have recently sparked renewed interest in their use in semiconductor applications. This review begins by exploring the unique material properties of OSs, which fundamentally originate from their distinct electronic band structure. Subsequently, we focus on atomic layer deposition (ALD), a core technique for growing excellent OS films, covering both basic and advanced processes compatible with 3D scaling. The basic surface reaction mechanisms—adsorption and reaction—and their roles in film growth are introduced. Furthermore, material design strategies, such as cation selection, crystallinity control, anion doping, and heterostructure engineering, are discussed. We also highlight challenges in memory applications, including contact resistance, hydrogen instability, and lack of p-type materials, and discuss the feasibility of ALD-grown OSs as potential solutions. Lastly, we provide an outlook on the role of ALD-grown OSs in memory technologies. This review bridges material fundamentals and device-level requirements, offering a comprehensive perspective on the potential of ALD-driven OSs for next-generation semiconductor memory devices.
Laser powder bed fusion of biodegradable Zn-4Cu alloy: Processing, microstructure and properties
Zn's natural degradability and biocompatibility make it a promising candidate for implants, however, its mechanical properties remain insufficient for bone applications. In this study, the performance of Zn was enhanced by developing Zn-Cu alloys via laser powder bed fusion (LPBF). Optimal LPBF parameters for forming stable tracks were achieved by adjusting laser power and scanning speed. Under optimized conditions of 100 W and 100 mm/s, high-density (99.58%) Zn-Cu alloys with improved hardness (68.2HV) and yield strength (160 MPa) were achieved. These improvements are attributed to solid solution strengthening, segregation strengthening, and grain refinement. The Zn-Cu alloys also demonstrated favorable degradation behavior, with a rate of 0.16 mm/year. This degradation is primarily driven by micro-galvanic corrosion between the CuZn5 phase and Zn matrix, along with refined grains and increased grain boundary density. This work demonstrates a viable strategy for fabricating Zn-based implants with enhanced structural integrity and mechanical performance via LPBF.