A 502-GOPS and 0.984-mW Dual-Mode Intelligent ADAS SoC With Real-Time Semiglobal Matching and Intention Prediction for Smart Automotive Black Box System

Lee, Kyuho Jason,Bong, Kyeongryeol,Kim, Changhyeon,Jang, Jaeeun,Lee, Kyoung-Rog,Lee, Jihee,Kim, Gyeonghoon,Yoo, Hoi-Jun IEEE 2017 IEEE journal of solid-state circuits Vol.52 No.1

<P>The advanced driver assistance system (ADAS) for adaptive cruise control and collision avoidance is strongly dependent upon the robust image recognition technology such as lane detection, vehicle/pedestrian detection, and traffic sign recognition. However, the conventional ADAS cannot realize more advanced collision evasion in real environments due to the absence of intelligent vehicle/pedestrian behavior analysis. Moreover, accurate distance estimation is essential in ADAS applications and semiglobal matching (SGM) is most widely adopted for high accuracy, but its system-on-chip (SoC) implementation is difficult due to the massive external memory bandwidth. In this paper, an ADAS SoC with behavior analysis with Artificial Intelligence functions and hardware implementation of SGM is proposed. The proposed SoC has dual-mode operations of highperformance operation for intelligent ADAS with real-time SGM in D-Mode (d-mode) and ultralow-power operation for black box system in parking-mode. It features: 1) task-level pipelined SGM processor to reduce external memory bandwidth by 85.8%; 2) region-of-interest generation processor to reduce 86.2% of computation; 3) mixed-mode intention prediction engine for dualmode intelligence; and 4) dynamic voltage and frequency scaling control to save 36.2% of power in d-mode. The proposed ADAS processor achieves 862 GOPS/W energy efficiency and 31.4GOPS/ mm(2) area efficiency, which are 1.53x and 1.75x improvements than the state of the art, with 30 frames/s throughput under 720p stereo inputs.</P>

A Vocabulary Forest Object Matching Processor With 2.07 M-Vector/s Throughput and 13.3 nJ/Vector Per-Vector Energy for Full-HD 60 fps Video Object Recognition

Lee, Kyuho Jason,Gyeonghoon Kim,Junyoung Park,Hoi-Jun Yoo IEEE 2015 IEEE journal of solid-state circuits Vol.50 No.4

<P>Approximate nearest neighbor searching has been studied as the keypoint matching algorithm for object recognition systems, and its hardware realization has reduced the external memory access which is the main bottleneck in object recognition process. However, external memory access reduction alone cannot satisfy the ever-increasing memory bandwidth requirement due to the rapid increase of the image resolution and frame rate of many recent applications such as advanced driver assistance system. In this paper, vocabulary forest (VF) processor is proposed that achieves both high accuracy and high speed by integrating on-chip database (DB) to remove external memory access. The area-efficient reusable-vocabulary tree architecture is proposed to reduce area, and the propagate-and-compute-array architecture is proposed to enhance the processing speed of the VF. The proposed VF processor can speed up the object matching stage by 16.4x compared with the state-of-the-art matching processor [Hong et al., Symp. VLSIC, 2013] for high resolution (Full-HD) and real-time (60 fps) video object recognition. It is fabricated using 65 nm CMOS technology and integrated into an object recognition SoC. The proposed VF chip achieves 2.07 M-vector/s throughput and 13.3 nJ/vector per-vector energy with 95.7% matching accuracy for 100 objects.</P>

A 1.4-m <tex> $\Omega$</tex> -Sensitivity 94-dB Dynamic-Range Electrical Impedance Tomography SoC and 48-Channel Hub-SoC for 3-D Lung Ventilation Monitoring System

Kim, Minseo,Jang, Jaeeun,Kim, Hyunki,Lee, Jihee,Lee, Jaehyuck,Lee, Jiwon,Lee, Kyoung-Rog,Kim, Kwantae,Lee, Yongsu,Lee, Kyuho Jason,Yoo, Hoi-Jun IEEE 2017 IEEE journal of solid-state circuits Vol.52 No.11

<P>A wearable electrical impedance tomography (EIT) system is proposed for the portable real-time 3-D lung ventilation monitoring. It consists of two types of SoCs, active electrode (AE)-SoC and Hub-SoC, mounted on wearable belts. The 48-channel AE-SoCs are integrated on flexible printed circuit board belt, and Hub-SoC is integrated in the hub module which performs data gathering and wireless communication between an external imaging device. To get high accuracy under the variation of conductivity, the dual-mode current stimulator provides the optimal frequency for time difference-EIT and frequency difference-EIT with simultaneous 4 k-128 kHz impedance sensing. A wide dynamic range instruments amplifier is proposed to provide 94 dB of wide dynamic range impedance sensing. In addition, the 48-channel AE system with the dedicated communication and calibration is implemented to achieve 1.4-m Omega sensitivity of impedance difference in the in vivo environment. The AE-/Hub-SoCs occupy 3.2 and 1.3 mm2in 65-nm CMOS technology and consume 124 mu W and 1.1 mW with 1.2 V supply, respectively. As a result, EIT images are reconstructed with 90% of accuracy, and up to 10 frames/s real-time 3-D lung images are successfully displayed.</P>

A 9.52 ms Latency, and Low-power Streaming Depth-estimation Processor with Shifter-based Pipelined Architecture for Smart Mobile Devices

Sungpill Choi,Kyuho Jason Lee,Youngwoo Kim,Hoi-Jun Yoo 대한전자공학회 2020 Journal of semiconductor technology and science Vol.20 No.3

The 3D hand gesture interface (HGI) for virtual reality and mixed reality on smart mobile devices is strongly dependent upon the robust depth-estimation with low latency and power consumption. However, the conventional depth-estimation hardware such as active depth sensors and stereo matching accelerators cannot realize the always-on and natural 3D HGI on mobile platform due to their large power consumption from active depth sensors and computations as well as the massive external memory bandwidth, respectively. To resolve the limit, we propose a depth-estimation processor that realizes the always-on and natural 3D HGI with algorithm and hardware co-optimization. The processor features: 1) shifter-based adaptive support weight aggregation that replaces complex floating-point operations with integer operations to reduce power and bandwidth by 92.2% and 69.1%; 2) line streaming 7-stage pipeline architecture with aggregation pipeline reordering optimization to realize 94% utilization and 43.9% memory reduction; and 3) shifting register-based pipeline buffer optimization to reduce 29.8% area. The proposed depth-estimation processor realizes a real-time 3D HGI with 9.52 ms of latency under QVGA stereo inputs. It achieves external memory bandwidth reduction to 18.93 MB/s with 15.56 mW power and 2.8 mm2 area, which are 4.1x and 6.9x more efficient than state-of-the-arts [9, 10], respectively.

A 82-nW Chaotic Map True Random Number Generator Based on a Sub-Ranging SAR ADC

Minseo Kim,Unsoo Ha,Kyuho Jason Lee,Yongsu Lee,Hoi-Jun Yoo IEEE 2017 IEEE journal of solid-state circuits Vol.52 No.7

<P>An ultra-low power true random number generator (TRNG) based on a sub-ranging SAR analog-to-digital converter (ADC) is proposed. The proposed TRNG is composed of a coarse-SAR ADC with a low-power adaptive-reset comparator and a low-power dynamic amplifier. The coarse-ADC part is shared with a sub-ranging SAR ADC for area reduction. The shared coarse-ADC not only plays the role of discrete-time chaotic circuit but also reduces the overall SAR ADC energy consumption by selectively activating the fine-SAR ADC. Also, the proposed dynamic residue amplifier consumes only 48 nW and the adaptive-reset comparator generates a chaotic map with only 6-nW consumption. The proposed TRNG core occupies 0.0045 mm<SUP>2</SUP> in 0.18-μm CMOS technology and consumes 82 nW at 270-kbps throughput with 0.6-V supply. It successfully passes all of National Institute of Standards and Technology (NIST) tests, and it achieves the state-of-the-art figure-of-merit of 0.3 pJ/bit.</P>

A CMOS Image Sensor-Based Stereo Matching Accelerator With Focal-Plane Sparse Rectification and Analog Census Transform

Kim, Changhyeon,Bong, Kyeongryeol,Choi, Sungpill,Lee, Kyuho Jason,Yoo, Hoi-Jun IEEE 2016 IEEE transactions on circuits and systems. a publi Vol.63 No.12

<P>A low-latency and low-power stereo matching accelerator is monolithically integrated with a CMOS image sensor (CIS) for mobile applications. To reduce the overall latency, focal-plane processing is adopted by using the proposed analog census transform circuit (ACTC), and the image readout is pipelined with the following stereo matching process. In addition, a novel focal-plane rectification pixel array (FRPA) merges the rectification with the image readout without any additional processing latency. For area-efficient pixel design, sparse rectification is proposed, and the image rectification is implemented with only two additional switches in each pixel. A stereo matching digital processor (SMDP) is integrated with the CIS for cost aggregation. We present the full design including the layout with a 65 nm CMOS process, and the FRPA, the ACTC, and the SMDP achieve 11.0 ms latency with complete stereo matching stages, which is suitable for a smooth user interface. As a result, the 2-chip stereo matching system dissipates 573.9 mu J/frame and achieves 17% energy reduction compared to a previous stereo matching SoC.</P>

A 2.71 nJ/Pixel Gaze-Activated Object Recognition System for Low-Power Mobile Smart Glasses

Injoon Hong,Kyeongryeol Bong,Dongjoo Shin,Seongwook Park,Kyuho Jason Lee,Youchang Kim,Hoi-Jun Yoo IEEE 2016 IEEE journal of solid-state circuits Vol.51 No.1

<P>A low-power object recognition (OR) system with intuitive gaze user interface (UI) is proposed for battery-powered smart glasses. For low-power gaze UI, we propose a low-power single-chip gaze estimation sensor, called gaze image sensor (GIS). In GIS, a novel column-parallel pupil edge detection circuit (PEDC) with new pupil edge detection algorithm XY pupil detection (XY-PD) is proposed which results in 2.9x power reduction with 16x larger resolution compared to previous work. Also, a logarithmic SIMD processor is proposed for robust pupil center estimation, <1 pixel error, with low-power floating-point implementation. For OR, low-power multicore OR processor (ORP) is implemented. In ORP, task-level pipeline with keypoint-level scoring is proposed to reduce the number of cores as well as the operating frequency of keypoint-matching processor (KMP) for low-power consumption. Also, dual-mode convolutional neural network processor (CNNP) is designed for fast tile selection without external memory accesses. In addition, a pipelined descriptor generation processor (DGP) with LUT-based nonlinear operation is newly proposed for low-power OR. Lastly, dynamic voltage and frequency scaling (DVFS) for dynamic power reduction in ORP is applied. Combining both of the GIS and ORP fabricated in 65 nm CMOS logic process, only 75 mW average power consumption is achieved with real-time OR performance, which is 1.2x and 4.4x lower power than the previously published work.</P>