Key Takeaways & Executive Findings
- •• The vision chip integrates a 2D/3D image sensor with a programmable visual processor using organic substrate 3D packaging, enabling compact end-to-end visual SoC. • A novel on-chip processing flow with die-to-die image transmission and low-latency fixed-point processing supports real-time CNNs and conventional algorithms. • The end-to-end 2D/3D vision system achieves real-time applications: face detection with 10.2 ms delay and depth map reconstruction with 4.1 ms delay. • The system maintains a frame rate exceeding 30 fps for image acquisition, processing, and display, demonstrating practical viability.
Abstract
This paper describes a 2D/3D vision chip with integrated sensing and processing capabilities. The 2D/3D vision chip architecture includes a 2D/3D image sensor and a programmable visual processor. In this architecture, we design a novel on-chip processing flow with die-to-die image transmission and low-latency fixed-point image processing. The vision chip achieves real-time end-to-end processing of convolutional neural networks (CNNs) and conventional image processing algorithms. Furthermore, an end-to-end 2D/3D vision system is built to exhibit the capacity of the vision chip. The vision system achieves real-timing applications under 2D and 3D scenes, such as human face detection (processing delay 10.2 ms) and depth map reconstruction (processing delay 4.1 ms). The frame rate of image acquisition, image process, and result display is larger than 30 fps.
1. Introduction
A vision chip integrates an image sensor and parallel processing element (PE) array to form a compact end-to-end visual system-on-chip (SoC)[1]. Vision chips of the first generation combine PEs and pixels to achieve maximum parallelism with ultra-low latency. This type of integration causes passive interaction of both the sensing and processing parts. The fill factor and pixel area degrade because of PE insertion, and the functionality of PE is forced to be simplified because of the limited area for computing circuits in pixels.
Recently, researchers have applied 3D stack technology on vision chips[2] to integrate sensors and visual processors. 3D stack imposes a design constraint on sensors and processors, and these chips need to be designed to be the same size for die-to-die vias to interconnect.
Nowadays, novel image sensors have expanded from single-modal 2D data to multimodal data (2D/3D). By introducing the concept of depth maps, the processing of 2D/3D images is widespread in both industrial and recreational applications such as automotive[3], robotics[4], and a...
Loading authentic research manuscript (Pages 1–5)...
Siyuan Wei, Quanmin Chen, Jingyi Yu, Xuanzhe Xu, Yuxiao Wen, Runjiang Dou, Shuangming Yu, Guike Li, Kaiming Nie, Jie Cheng, Jiangtao Xu, Liyuan Liu, Nanjian Wu (2025). A 2D/3D vision chip based on organic substrate 3D package. SinoTechIntel Verified Research. https://doi.org/10.1088/1674-4926/25010030
Research & Educational Purpose Only:The translations, structured abstracts, analytical annotations, and data reports provided by SinoTechIntel are intended exclusively for academic research, internal corporate R&D, and educational benchmarking. They do not constitute formal engineering, chemical safety, legal, or professional advice.
Copyright & Intellectual Property Notice: Original copyright of the underlying source articles and experimental data remains with the respective authors, institutions, and original publishing journals. SinoTechIntel claims intellectual property only over its proprietary translations, analytical syntheses, and AEO structured enhancements in accordance with international fair use and academic citation principles.
Frequently Asked Questions
What is the main contribution of this paper?
The paper presents a 2D/3D vision chip that integrates sensing and processing using organic substrate 3D packaging, enabling real-time end-to-end processing of CNNs and conventional algorithms with low latency.
How does the vision chip achieve low-latency processing?
The chip uses a novel on-chip processing flow with die-to-die image transmission and low-latency fixed-point image processing, which reduces delays in data transfer and computation.
What are the demonstrated applications of the vision system?
The vision system demonstrates real-time human face detection with a processing delay of 10.2 ms and depth map reconstruction with a delay of 4.1 ms, both operating at over 30 fps.
What is the significance of using organic substrate 3D packaging?
Organic substrate 3D packaging allows for compact integration of the sensor and processor without the size-matching constraints of traditional 3D stacking, potentially improving design flexibility and performance.
What types of algorithms can the vision chip process?
The vision chip can process both convolutional neural networks (CNNs) and conventional image processing algorithms in real time, making it versatile for various vision tasks.
Related Technical Papers & Translations
A Novel Approach for Enhanced Brain Tumor Segmentation Using Multimodal MRI and Deep Learning
Brain tumor segmentation from multimodal MRI is crucial for diagnosis and treatment planning. In this study, we propose a novel deep learning framework that integrates structural and functional imaging modalities to improve segmentation accuracy. Our method employs a multi-scale attention mechanism and a hybrid loss function to handle class imbalance and boundary ambiguity. Evaluated on the BraTS benchmark, our approach achieves state-of-the-art performance, with Dice scores of 0.91, 0.87, and 0.84 for whole tumor, core, and enhancing tumor, respectively. Furthermore, we demonstrate the generalizability of our model across different scanners and protocols. Our findings suggest that the proposed method can significantly aid clinical decision-making and surgical planning.
Investigation of coupled acoustic and electrical responses and early warning approaches during re-loading of damaged coal
Initial damage from engineering disturbances in deep coal mining degrades mechanical properties and heightens dynamic-hazard risks, challenging conventional monitoring. This study probes the coupled acoustic-electrical responses of initially damaged coal under reloading and develops a multi-parameter, multi-level dynamic integrated early-warning model. Using a true-triaxial Split Hopkinson Pressure Bar (SHPB) system, we prepared specimens with graded damage by varying static deviatoric stresses and dynamic impacts. Uniaxial compression reloading was conducted with synchronous acoustic emission (AE) and resistivity monitoring. Joint time-domain responses of force, acoustics, and electricity delineated distinct loading stages. Time-frequency features were extracted via Fourier and wavelet transforms; crack architecture was quantified by 3D AE localization and fractal-dimension analysis. Initial damage markedly reduced load-bearing capacity. Resistivity decreased sharply with increasing deviatoric stress, while cumulative AE counts increased strongly. The AE spectrum evolved from bimodal to broadband with low- and high-frequency enhancement. The resistivity spectrum showed progressive bandwidth broadening, energy amplification, and high-frequency advancement. The AE spatial fractal dimension rose significantly during compaction. An integrated warning system combining multiscale entropy fusion, Temporal Convolutional Network (TCN)-Transformer forecasting, recurrence-network analysis, and a Bayesian framework yielded a 28.4 s lead time, offering a theoretical basis and technical pathway for intelligent prevention of dynamic hazards.
Influence of aggregate particle size on fracture behavior and energy evolution of cemented rockfill in the post-peak stage
Cemented rockfill (CRF) combines structural support with sustainable reuse of coal-derived solid waste. This study integrates digital image correlation, acoustic emission monitoring, and finite–discrete element simulations to investigate mechanical behavior, fracture development, and energy evolution of CRF containing 54% aggregate content with three grain-size distributions (5–10, 10–20, and 20–30 mm). Results indicate finer aggregates raise compressive strength and elastic modulus, and increase post-peak softening and residual stiffness. Fracture patterns transition from dominantly unidirectional failure in coarse specimens to pronounced X-shaped conjugate shear in fine specimens, with cracks initiating at boundaries and propagating inward. The proportion of failed joints at comparable strains decreases markedly with finer gradation, reflecting a more homogeneous crack network that enhances post-peak load retention and produces frequent minor stress fluctuations. Energy analyses reveal a coarse > medium > fine ordering in cumulative dissipation; however, finer aggregates delay rapid kinetic and dissipative energy release, promoting slower energy redistribution and improved load resistance. These findings quantify how aggregate gradation controls deformational mechanisms, crack topology, and energy partitioning, and provide design guidance for optimizing aggregate size and cementitious composition to enhance ductility, energy absorption, and structural reliability of CRF in underground engineering.