Key Takeaways & Executive Findings
- •• A hybrid framework combining deep reinforcement learning with rule-based decision-making enhances safety and training efficiency for lane change maneuvers. • The proposed Risk-fused DDQN algorithm improves convergence reward by up to 7.6% and reduces training episodes by up to 66.8% compared to conventional DDQN. • Real vehicle tests show a 57.3% improvement in lane change success rate and at least 16.5% increase in time headway, confirming enhanced safety and adaptability. • The risk assessment model acts as a safety filter, correcting dangerous actions and improving sampling efficiency through a separate experience buffer for dangerous trials.
Abstract
To solve problems of poor security guarantee and insufficient training efficiency in the conventional reinforcement learning methods for decision-making, this study proposes a hybrid framework to combine deep reinforcement learning with rule-based decision-making methods. A risk assessment model for lane-change maneuvers considering uncertain predictions of surrounding vehicles is established as a safety filter to improve learning efficiency while correcting dangerous actions for safety enhancement. On this basis, a Risk-fused DDQN is constructed utilizing the model-based risk assessment and supervision mechanism. The proposed reinforcement learning algorithm sets up a separate experience buffer for dangerous trials and punishes such actions, which is shown to improve the sampling efficiency and training outcomes. Compared with conventional DDQN methods, the proposed algorithm improves the convergence value of cumulated reward by 7.6% and 2.2% in the two constructed scenarios in the simulation study and reduces the number of training episodes by 52.2% and 66.8% respectively. The success rate of lane change is improved by 57.3% while the time headway is increased at least by 16.5% in real vehicle tests, which confirms the higher training efficiency, scenario adaptability, and security of the proposed Risk-fused DDQN.
1. Introduction
Autonomous vehicle, which has great potential to reduce traffic accidents and jams, is a future trend in automobiles [1]. Decision-making is a central component of an autonomous driving system since the decision-making module determines the behavior of the vehicle. More specifically, it outputs specific target points, target poses, vehicle speed and other boundary constraints based on behavioral patterns, which are later utilized in the planning module to generate trajectories [2, 3]. The lane change maneuver is an important part of behavioral decision-making [4, 5]. The current widely adopted technical routes for lane change decision-making can be divided into rule-based approaches and learning-based approaches [6, 7].
1.1 Rule-Based Decision-making: The rule-based lane change decision-making system determines the behavior of vehicles based on an established rule base. The rule-based method has the following advantages: simple to apply, highly interpretable, safe and stable [8]. Nilsson et al. [9] made simple and clear logical rules to recognize the behavioral intentions of surrounding vehicles by observing the lateral and longitudinal movement law of traffic participants, and accordingly proposed an appropriate scheme to be applied to highway lane change decision-making [10]. Constantin et al. [11] constructed a decision tree by enumerating all the possible resulting navigation decisions associated with each obstacle. The vehicle will consider the optimal decision behavior for each lane from left to right each time it approaches an obstacle, but the decision tree-based method faces the problem of difficult state classification for complex working conditions.
Brechtel et al. [12] combined dynamic Bayesian network based continuous space prediction with discrete-space Markov Decision Process (discrete-space MDP), so that the decision-making system can cope with the uncertainty in the evolution of the lane change state. Ref. [13] estimated the distribution of potential driving intentions of surrounding vehicles based on their historical trajectories. They used the Partial Observable Markov Decision Process (POMDP) solution framework to take the coupling effect between multiple traffic participants into account, and verified the effectiveness of the decision-making algorithm in lane change and intersection scenarios. Bahram et al. [14] used game theory to find a sequence of actions in the planning time domain to balance environmental risk and vehicle intent, solving for an optimal strategy considering the interaction.
The risk assessment-based decision-making method can model and evaluate the risk degree of the driving process for autonomous vehicles. The concept of Artificial Potential Field (APF) in the field of robot path planning is a risk assessment method that has subsequently been widely used in the field of autonomous driving and assisted driving.
Loading authentic research manuscript (Pages 1–5)...
Lu Xiong, Zhuoren Li, Danyang Zhong, Puhang Xu, Chen Tang (2025). Rule-Guidance Reinforcement Learning for Lane Change Decision-making: A Risk Assessment Approach. Chinese Journal of Mechanical Engineering. https://doi.org/10.1186/s10033-024-01160-z
Research & Educational Purpose Only:The translations, structured abstracts, analytical annotations, and data reports provided by SinoTechIntel are intended exclusively for academic research, internal corporate R&D, and educational benchmarking. They do not constitute formal engineering, chemical safety, legal, or professional advice.
Copyright & Intellectual Property Notice: Original copyright of the underlying source articles and experimental data remains with the respective authors, institutions, and original publishing journals. SinoTechIntel claims intellectual property only over its proprietary translations, analytical syntheses, and AEO structured enhancements in accordance with international fair use and academic citation principles.
Frequently Asked Questions
What is the main contribution of this paper?
The paper proposes a hybrid framework that integrates deep reinforcement learning with rule-based decision-making, incorporating a risk assessment model as a safety filter to improve training efficiency and safety for lane change maneuvers.
How does the Risk-fused DDQN algorithm improve training efficiency?
The algorithm uses a separate experience buffer for dangerous trials and applies punishment to such actions, which enhances sampling efficiency and reduces the number of training episodes by up to 66.8% compared to conventional DDQN.
What are the key results from real vehicle tests?
Real vehicle tests show a 57.3% improvement in lane change success rate and at least a 16.5% increase in time headway, confirming higher safety and adaptability of the proposed method.
What is the role of the risk assessment model?
The risk assessment model evaluates the risk of lane-change maneuvers considering uncertain predictions of surrounding vehicles, acting as a safety filter to correct dangerous actions and improve learning efficiency.
How does the proposed method compare to conventional DDQN?
The proposed method improves the convergence value of cumulated reward by 7.6% and 2.2% in two simulation scenarios, and reduces training episodes by 52.2% and 66.8%, respectively, demonstrating superior performance.
Related Technical Papers & Translations
Direct Repair of the Crystal Structure and Coating Surface of Spent LiFePO4 Materials Enables Superfast Li-Ion Migration
The rapid accumulation of spent LiFePO4 (LFP) cathodes from retired lithium-ion batteries necessitates the development of effective and environmental-friendly recycling strategies. In this context, direct regeneration has emerged as a promising approach for reclaiming LFP cathode materials, offering a streamlined pathway to restore their electrochemical functionality. We report an integrated regeneration protocol that simultaneously repairs the degraded crystal structure and reconstructs the damaged carbon coating in spent LFP. The regenerated cathode material had superfast lithium-ion diffusion kinetics and a stable cathode–electrolyte interface, giving a remarkable rate capability with specific capacities of 122 mAh g−1 at 5C and 106 mAh g−1 at 10C (1C = 170 mA g−1). It also maintained capacities of 110.7 mAh g−1 (5C) and 84.1 mAh g−1 (10C) after 400 cycles. It could be used in harsh environments and could be stably cycled at subzero temperatures (−10 and −20 °C) and in solid-state electrolyte batteries. Life cycle assessment combined with economic evaluation using the EverBatt model reveals that this direct regeneration approach has high economic and environmental benefits.
Oxide Semiconductor for Advanced Memory Architectures: Atomic Layer Deposition, Key Requirement and Challenges
Oxide semiconductors (OSs), introduced by the Hosono group in the early 2000s, have evolved from display backplane materials to promising candidates for advanced memory and logic devices. The exceptionally low leakage current of OSs and compatibility with three-dimensional (3D) architectures have recently sparked renewed interest in their use in semiconductor applications. This review begins by exploring the unique material properties of OSs, which fundamentally originate from their distinct electronic band structure. Subsequently, we focus on atomic layer deposition (ALD), a core technique for growing excellent OS films, covering both basic and advanced processes compatible with 3D scaling. The basic surface reaction mechanisms—adsorption and reaction—and their roles in film growth are introduced. Furthermore, material design strategies, such as cation selection, crystallinity control, anion doping, and heterostructure engineering, are discussed. We also highlight challenges in memory applications, including contact resistance, hydrogen instability, and lack of p-type materials, and discuss the feasibility of ALD-grown OSs as potential solutions. Lastly, we provide an outlook on the role of ALD-grown OSs in memory technologies. This review bridges material fundamentals and device-level requirements, offering a comprehensive perspective on the potential of ALD-driven OSs for next-generation semiconductor memory devices.
Laser powder bed fusion of biodegradable Zn-4Cu alloy: Processing, microstructure and properties
Zn's natural degradability and biocompatibility make it a promising candidate for implants, however, its mechanical properties remain insufficient for bone applications. In this study, the performance of Zn was enhanced by developing Zn-Cu alloys via laser powder bed fusion (LPBF). Optimal LPBF parameters for forming stable tracks were achieved by adjusting laser power and scanning speed. Under optimized conditions of 100 W and 100 mm/s, high-density (99.58%) Zn-Cu alloys with improved hardness (68.2HV) and yield strength (160 MPa) were achieved. These improvements are attributed to solid solution strengthening, segregation strengthening, and grain refinement. The Zn-Cu alloys also demonstrated favorable degradation behavior, with a rate of 0.16 mm/year. This degradation is primarily driven by micro-galvanic corrosion between the CuZn5 phase and Zn matrix, along with refined grains and increased grain boundary density. This work demonstrates a viable strategy for fabricating Zn-based implants with enhanced structural integrity and mechanical performance via LPBF.