Abstract
An image sensor that uses stereo vision has been developed for forward obstacle and lane marking detection. Detection is performed by the following functions.
(1) Stereo camera input interface
(2) Warp correction of image distortion and parallelization of stereo images
(3) Generation of 3D depth map from the parallelized stereo images
(4) Detection of obstacles from the 3D depth map
(5) Hough transform
(6) Application to road model
Obstacle detection is achieved by functions (1) to (4), and lane marking detection is achieved by functions (1), (2), (5) and (6). The hardware achieving these functions comprises a geometry transform LSI (Pollux), a microprocessor for image processing (IMAP/Castor), an original synchronous static RAM (SSRAM), a LSI for digital video interface (DVI) and a supervisor micro-controller (SV). Pollux performs functions (1) and (2), while IMAP/Castor performs (3) to (6). The SSRAM is used for the image transform table for Pollux, and for the calculation memory for IMAP/Castor. The DVI has the function of receiving digital data from the cameras, and the SV performs ECU control. The developed Pollux, IMAP/Castor and SSRAM units can be described in more detail as follows. Pollux has geometry transform circuitry that uses a bi-linear interpolation method based on four neighboring pixels. It also has circuitry for detecting digital input interface error. Its size is approximately 7 mm × 7 mm, and it was developed using a 0.15 µm CMOS process. It operates at 60 MHz. IMAP/Castor is a one-dimensional processor array-type on-chip parallel processor that uses a SIMD method. It has 128 8-bit processor elements that operate using a 4-way VLIW method, and a RISC processor for chip control. Its size is approximately 10 mm × 10 mm, and it was developed using a 0.13 µm CMOS process. It operates at 100 MHz. The SSRAM has a memory capacity of 1 MB, and improves the reliability of each item of data through the addition of ECC logic or parity bits. Its size is approximately 9 mm × 9 mm, and it was developed using a 0.13 µm CMOS process. It operates at 100 MHz. This LSI composition achieves a processing time of 100 ms or less in a cycle of 100 ms for the input image data of the cameras in order to perform obstacle and lane marking detection.