Introduction: On April 11, 2018, the Ministry of Industry and Information Technology, the Ministry of Public Security and the Ministry of Transport jointly issued the 'Notice on Printing and Distributing the Intelligent Network Vehicle Road Test Management Regulations (Trial)' for China’s Intelligent Network Road Network. The test provided the relevant legal basis. The three ministries also put forward several strict conditions while granting the qualifications of the intelligent network.
Among them, in the seventh article 'fourth' point of 'test subject, test driver and test vehicle', the three ministries require: Having the vehicle status record, storage and online monitoring functions, and the following first , 2, 3 items of information, and automatically record and store the following information at least 90 seconds before the vehicle accident or failure occurs, data storage time is not less than 3 years:
1. Vehicle control mode; 2. Vehicle position; 3. Vehicle speed, acceleration and other movement status; 4. Environmental perception and response status; 5. Vehicle lighting, signal real-time status; 6. Vehicle external 360-degree video monitoring; Reflect the in-vehicle video and voice monitoring of the test driver and human-machine interaction status; 8. Remote control instructions (if any) received by the vehicle; 9. Vehicle fault conditions (if any).
It can be seen from the above that in addition to the functional requirements for intelligent networked vehicles that perform road tests, the above conditions will also promote the development of several new generation technologies such as communications, monitoring, control, and storage. The development of automotive SoCs provides new market opportunities.
Embedded FPGA (eFPGA) will play an important role in this type of chip. For example, in order to meet the acquisition and processing of 360-degree video surveillance data outside the vehicle mentioned in Article 6, the use of eFPGA to design related functional chips has obvious advantages. As a company that provides both stand-alone FPGA chips and eFPGA IP products, Achronix can help smart car SoC designers develop and debug related functions on the FPGA chip first, and without the need to significantly modify the design after the market enters into batch applications, immediately transplant the design. To the SoC equipped with eFPGA.
For how eFPGA supports the design of embedded 360-degree viewing vision systems, read the article by Alok Sanghavi, Senior Product Marketing Manager at Achronix Semiconductor, under the heading 'Use eFPGA in embedded 360-degree viewing vision systems'.
Application of eFPGA in Embedded 360-degree Vision Vision System
Author: Alok Sanghavi, Achronix Semiconductor senior product marketing manager
Embedded 360° vision vision systems with multiple high-resolution cameras have entered various applications such as automotive sensor fusion, video surveillance, target detection, motion analysis, etc. In such systems, multiple real-time cameras The video streams (up to 6) are aggregated together frame-by-frame, processed for distortion and other image artefacts, adjusted for exposure and white balance, and then dynamically tiled into a 360° panoramic view with 4K resolution and 60 fps frame rate Output, finally projected onto a spherical coordinate space.
Current high-resolution fish-eye camera lenses used in such applications typically have a wide-angle field of view (FOV). One of the biggest bottlenecks in ring-vision camera systems is: Real-time access to or storage of multi-channel camera inputs from external memory The data is then processed as a single frame. The hardware needs to be within one frame delay to complete the processing run between inputting the raw sensor data passed from the camera and splicing the output video.
High-performance computing platforms have been moving toward the use of FPGAs in conjunction with CPUs to provide dedicated hardware acceleration for real-time image processing tasks. This configuration allows the CPU to focus on particularly complex algorithms where they can quickly switch threads and contexts. , and assign repetitive tasks to an FPGA to act as a configurable hardware accelerator/coprocessor/offload engine. Even if FPGAs and CPUs are used as discrete devices, the system can improve overall efficiency because these technologies do not collide , but like putting gloves on your hands.
For example, an image obtained from a fisheye lens suffers from severe distortion, so a splicing operation generated based on multiple camera videos is a highly computationally intensive task because it is a point-pixel operation. This splicing requires a large amount of real-time imagery. Processing and highly parallel architectures. However, this next-generation application exceeds the performance that FPGAs can continue to achieve, mainly due to the latency of the chip's throughput data. This in turn affects the overall system's overall latency, throughput speed, and performance. .
Add an eFPGA semiconductor intellectual property (IP) that can be embedded with the CPU in an SoC. Compared with a stand-alone FPGA chip plus CPU solution, the embedded FPGA array structure has unique advantages, the main advantage is the stronger performance. The eFPGA can be connected directly to an ASIC (without I/O buffers) through a wide parallel interface, providing significantly improved throughput and latency counted in single-digit clock cycles. Low latency is a complex image real-time processing process The key, such as correcting the distortion of the fisheye lens is such a process.
With Speedcore eFPGA IP, customers can define their logic, memory and DSP resource requirements, and then Achronix can configure their IP to meet their needs. Look-up tables (LUTs), RAM unit blocks, and DSP64 unit blocks can be combined just like building blocks. Create the best programmable structure for any given application.
In addition to standard logic, embedded memory, and DSP unit modules, customers can define their own function blocks within the Speedcore eFPGA structure. By adding these custom function blocks together with traditional building blocks into the logic array structure, you can add optimizations. The features to reduce the area and improve the performance of the target application can greatly improve the performance of eFPGA, especially for embedded vision and image processing algorithms.
Using custom cell blocks to successfully solve high-performance image processing is a good example. You can only look once (YOLO). This kind of state-of-the-art real-time object detection algorithm that uses neural networks can be excellent. The early method greatly improved performance. The algorithm relies on a large number of matrix multipliers, and when implemented in an FPGA, these matrix multipliers need to be built using DSP and RAM modules; the YOLO requires the most of the DSP and RAM modules Good configuration, there is a problem with the mismatch found in a typical FPGA array structure. For example, the FPGA array structure may provide 18×27 multiply/accumulate unit blocks and 32×128 RAM DSP block, The best solution for this may be a 16×8 DSP block with 48×1024 RAM. By creating a custom block that implements the best DSP and RAM block configuration, the resulting Speedcore array structure will use less chip area. 40% to achieve the same functionality and achieve a higher level of system performance.
Embedding an FPGA array structure in an SoC provides two additional system-level benefits:
Lower power consumption - Programmable I / O circuits make up half of the total FPGA chip's total power consumption, while an eFPGA can be directly connected to other modules in the host SoC, eliminating the need for large programmable I / O buffers .
Lower system cost - Since the eFPGA only needs to implement specific functions, the die size of the eFPGA is much smaller than that of the equivalent stand-alone FPGA chip. This is because the eFPGA no longer requires a programmable I/O buffer and unnecessary interface logic.
With the ultra-low latency and real-time processing capabilities, vision systems based on 360° views can be effectively implemented. Speedcore eFPGA with custom blocks is used in conjunction with a CPU in the same host SoC, making it ideal for implementing specialized functions such as target detection. And image recognition, distortion and distortion correction, and finally splicing together the final image. Embedded in the SoC FPGA array structure is a natural development process of ultra-deep submicron era system integration.