Detailed OPPO TOF structure light and unlimited possibilities for future applications

On August 6th, OPPO held a TOF technical communication meeting in Beijing. Zhang Xueyong, OPPO's 3D visual technology research and development project manager, gave a comprehensive technical explanation of TOF technology, and answered questions about TOF technology with OPPO product manager Zhao Yuqi.

The continuous development of mobile phones has enabled him to acquire and process information. However, for a long time in the past, mobile phones have seen a flat world. The addition of 3D vision technology can solve this problem well. The vision is from plane to stereo. There are three mainstream 3D vision solutions in the industry: binocular stereo vision scheme, 3D structured light scheme, TOF scheme. Binocular stereo vision scheme belongs to passive acquisition scheme, 3D structured light and TOF belong to active acquisition. Program.

What are the differences between 3D structured light and TOF? 3D structured light is mainly suitable for close-range 3D information collection, special application scenarios are used for face recognition, and then face recognition is applied to unlocking, payment, etc.; TOF scheme is mainly applied to Relative to long-distance 3D information collection, the TOF solution has a larger application range and imagination space. The front-end use scene and the rear-use scene of the mobile phone can be well adapted to 3D structured light and TOF technology.

TOF is an abbreviation of time of flight. This technology mainly transmits optical signals continuously through the measured target, receives the optical signal returned by the measured target at the sensor end, and calculates the round-trip flight time of transmitting and receiving optical signals. To obtain the distance of the target to be measured. TOF technology is based on the traditional 2D XY axis imaging, adding depth information from the Z-axis direction, and finally generating 3D image information. For the accuracy of 3D image information generated by TOF, add Longitudinal accuracy is also the key to 3D imaging technology.

The principle of 3D structured light is to emit a diffractive spot onto an object. After receiving the spot that has been deformed, the sensor determines the depth information according to the amount of spot deformation, but the diffracted spot emitted by the 3D structured light has an energy density outside a certain distance. It will be reduced, so it is not suitable for long-distance depth information acquisition; TOF technology is to emit a surface light source instead of speckle, so the optical information of TOF will not be attenuated as much as 3D structured light, and the pixels of TOF photosensitive element will reach 10μm, there is sufficient guarantee for light collection.

TOF's five core hardware units Infrared emission unit Including Vcsel transmitter, Diffuser (diffuser). Vcsel emits a pulsed square wave with a wavelength of 940nm. The infrared light of this wavelength is non-visible, and the amount in the spectrum is the least, which can avoid the interference of ambient light. It is emitted by Vcsel. The light source will also modulate the light into a uniform surface source through Diffuser and then emit it. 2. Optical lens It is used to concentrate the reflected light and image it on the optical sensor. However, unlike ordinary optical lenses, a narrow-band filter is needed to ensure that only the same wavelength (ie 940 nm) of light as the emitted optical signal can enter. The purpose of this is to suppress the background noise caused by the incoherent light source while preventing the sensor from being overexposed due to external light interference. 3. Imaging sensor Similar to the sensor of a general camera, it is used to receive the reflected light and perform photoelectric conversion on the sensor. However, due to the principle of TOF, the sensor's sensing time is very short, reaching the nanosecond level, so the single pixel size is larger than the average camera. A lot bigger, for example: At present, the pixel size of RGB is 1μm, and the pixel size of our TOF sensor is 10μm. 4. Control unit The control unit is the driving IC of the laser emitter, which can drive the high-frequency pulse driving with the upper limit of 100MHz. At the same time, it eliminates all kinds of interference and ensures that the driving waveform is a perfect square wave. The rising and falling edges are around 0.2ns. Thereby effectively guaranteeing the extraction of high precision depth accuracy. 5. Core algorithm calculation unit The core algorithm calculation unit is the AP of our mobile phone. We transplant the deep extracted core algorithm library to the AP. The AP reads the single module calibration data from the module, drives the deep extraction algorithm library, and converts the RAW image into Depth map; then use the depth map for each application.

The advantage of the OPPO TOF solution lies in the Z-axis precision development, and it has reached the industry leading level in terms of accuracy, energy consumption, and application scene range:

1. Accurate Z-direction accuracy OPPO's TOF technology uses dual-frequency driving, sampling is 240 frames fps, that is: each single frequency corresponds to four phases, each phase corresponds to one laser pulse, thus obtaining 8 frames of RAW image, and finally combining one depth information every 8 frames Other solutions on the market generally have 30 frames per second. Each depth information uses only one frame of information, so the accuracy in the Z direction is higher, the absolute accuracy is 1%, and the relative accuracy is 0.5%. 2. Strict accuracy calibration In order to ensure the accuracy of the TOF, we have customized the process of precision calibration, which is not done in the industry, but for TOF, its extremely precise construction, each link will affect the accuracy of the final 3D imaging. In particular, the accuracy of the Z direction, so in the development process only for the accuracy calibration has been carried out three rounds of calibration equipment update, each time invested in high research and development costs, and we have developed a set of four distances An innovative combination of multiple calibrations at four angles to escort the experience of the final product. 3. Low power consumption OPPO's TOF solution uses BSI CMOS, which is 3 to 5 times lower in power consumption than other CCD solutions. 4. Strong anti-interference OPPO's TOF scheme uses an optical signal with a wavelength of 940 nm, which is the least in the spectrum. At the same time, because it is actively emitting light and is less affected by ambient light, OPPO's TOF scheme can work in all weather without dim light.

Application of TOF technology

Compared with 3D structured light, it focuses on close-up face unlocking and face recognition. TOF technology is more suitable for long-distance 3D information collection. It also makes TOF more widely used and more imaginative. As long as it is within reasonable range. The objects can be 3D modeled by TOF. After modeling, the 3D information can be displayed by AR, holographic projection, etc. For example, 3D fitting, AR decoration, AR game, somatosensory game, holographic image interaction, etc. It can be realized by TOF technology. The high bandwidth brought by the future 5G network provides information transmission guarantee for 3D vision technology, future 3D video call, virtual scene + virtual scene remote VR, virtual scene + real scene remote AR, Real-life + real-time remote JR and other ubiquitous real-life scenarios will usher in explosive applications.

Whether TOF will be applied to the front scene

Zhang Xueyong, OPPO 3D vision technology research and development manager, said: 3D structure light is close range, used in front scene applications; TOF is long distance mainly for post scene applications. So will TOF be applied to the front scene? OPPO 3D vision technology Zhang Xueyong, R&D manager, believes that the accuracy of 3D structured light is higher than TOF within 1.2 meters. The front face 3D information requires more accurate information, so the higher the precision, the better the effect, so the current TOF technology will not be added. In the TOF pre-scenario, after all, 3D structured light is a better choice than TOF.

OPPO product manager Zhao Yuqi also said: From the perspective of product planning, OPPO will choose the two technologies of front-end best effect and post-best effect. The reason why the 3D structured light is selected in the front is because of the accuracy of TOF. It is not a substitute for structured light, especially in beauty, face payment, etc. It is clear that these technologies can be seen and used in the second half of 2018.

OPPO will integrate TOF and 3D structured light into a mobile phone. OPPO product manager Zhao Yuqi said: The front will choose 3D structured light technology, but whether 3D structured light is combined with TOF design on the mobile phone, this can be expected.

Advantages of OPPO TOF

Compared with the technology released by other manufacturers, OPPO TOF is different in scheme. OPPO's TOF adopts CMOS scheme, other manufacturers may use CCD scheme. From a technical point of view, CCD adopts straight out 30 Frames, but CMOS logic can be more complex, you can synthesize 30 frames with 240 frames, which can achieve higher precision. Another reason is that CCD power consumption is 3-5 times higher than COS.

OPPO 3D vision technology development manager Zhang Xueyong seems: The structure of the future industry is the world of CMOS, CMOS has replaced the CCD with the camera industry as evidence. Unlike the traditional RGB Sensor's 300,000 points, the most important thing about 3D imaging technology is to acquire Z. The accuracy of the direction, in order to establish the model very accurately, so the most important thing at this stage is to ensure the accuracy of the Z direction. In the case of ensuring the accuracy of the Z direction, the higher the XY precision, the better. OPPO pays more attention to the Z axis at this stage. The accuracy of the direction, the accuracy of Z can achieve absolute accuracy of 1%, relative accuracy of 0.5%. The most important thing is to ensure that the accuracy of each TOF can reach the standard. OPPO TOF is 240 by 180 resolution, can meet the present Requirements for phase-related applications.

The accuracy of TOF sensing at close range is not better than that of structural light. It can also achieve femtosecond imaging. OPPO does not use femtosecond imaging system. The key is the calibration of TOF sensor accuracy. It is possible to improve the precision of structured light in short distance. It can be calibrated every five centimeters or two centimeters apart, but the workload is very large. According to the current production efficiency, it can not meet the mass production demand, which is not in line with the development of the entire industrial chain.

3D structured light security and privacy

When using 3D structured light, the mobile phone obtains the information of the human face, whether it will involve user privacy. Zhang Xueyong, OPPO visual technology research and development manager, said: In terms of privacy definition and protection, it is necessary to form a consensus in the industry and establish a The standards of compliance are now in their infancy.