Samsung's latest smart phone, the Galaxy S9, is ready to go on the market, and Huawei is about to announce the new product P20 in Paris next week. Imaging technology experts say that 3D sensing technology has become a The mobile phone industry's latest competition focus. But what is not clear yet is whether Samsung's or Huawei's smart phone 3D sensing capability is qualified to compete with Apple's iPhone X?
Pierre Cambou, director of MEMS and imaging technology research at Yole Developpement, a market research organization in France, said that because Apple’s TrueDepth camera technology sets a high threshold, he predicts that other competitors may take more than a year to provide iPhone 3 with comparable 3D sensing. technology.
Unlike the “megapixel wars” of the past, most smart phone makers will work harder on the new frontier of 3D sensing technology; this is because 3D cameras contain many components that need to work in unison, and they need to be adequate. Supply Chain Management.
Cambou described the 3D camera as "sub-devices." He pointed out: "Remember the Apple iPhone X? Participating suppliers include STMicroelectronics, LG Innotek, Foxconn. , Austrian Microelectronics (ams) and Lumentum. ”
The market has already commented that Samsung’s S9 forward sensing technology is “disappointing.” There was a CNET report commentary released earlier this month: “In-depth observation of Samsung’s latest biometric system – and the system and iPhone X’s Face ID comparison - results show that it is not safe on mobile payments. ”
The report pointed out that "this is because Samsung's facial recognition system uses a standard camera to create a 2D map of the user's face, and Apple's Face ID is a complex 3D scan that will create a facial image." People have already launched at Samsung last year. The Galaxy S8 confirmed that it can use photos to trick Samsung's face recognition technology. Obviously, S9 will be deceived by the same method.
As for the forthcoming Huawei P20, many media reports have considered the "three shots" of the phone as a key function - why do we need three camera lenses. Can three be better than two?
To this end, we asked Eran Briman, vice president of marketing and business development at Corephotonics, a supplier of dual-lens technology leadership. He said: “Three shots are a brand new technology area that has only just begun and has not proved its worth;” he pointed out Three cameras may have many types and configurations. These lenses may have their own purpose, including support for "low light shooting, zoom, depth of field, high resolution, and overall image quality."
According to Briman, the combination of three cameras may include: 1. A color (RGB) lens, a black and white lens, and a third camera that provides a super-wide field of view (FoV) or twice as far Narrow FoV; 2. A fisheye lens, a standard wide-angle lens, and a double telephoto lens; or 3. A standard wide-angle lens, a double telescope, and a five-time telescope to support super zoom ( Super-zoom) function.
He pointed out: "There may also be other configurations, including different image sensors;" but the main point is: "Compared to the dual lens design, all three lens configurations can provide more accurate depth information."
Briman said: "The main reason is that in fact its baseline (the distance between the shots) will become larger, especially the first and third shots; the greater the baseline value, the more accurate the depth information will be- ─ But the software is very challenging. It needs to deal with occlusion, synchronization, etc. "He added: "In addition, by integrating information from three cameras, theoretically better depth information can be obtained."
In other words, Huawei's three-lens solution is clearly intended to demonstrate the company's efforts in strengthening depth sensing technology. Although unconfirmed, the industry suspects that Qualcomm is Huawei's 3D sensing technology partner. Last year In the summer, Qualcomm announced that it has collaborated with Himax Technologies to create a fully integrated structured light module (SliM). Qualcomm said at the time that its 3D depth sensing camera module will be produced in the first quarter of this year.
What are the important vendors in the 3D sensing technology field?
So what are the key suppliers in the smart phone 3D sensing technology? Before we discuss this topic, first review the operating principle of Apple’s 3D camera:
Apple TrueDepth camera structure
(Source: Yole Developpement)
As explained by Yole previously, the iPhone X can recognize the user's face and unlock the phone. It combines a ToF range sensor with an infrared "structured light" camera, so it can use uniform "flood" or "bit pattern" lighting. The iPhone X combines an infrared camera with a floodlight sensor to project a uniform infrared light in front of the phone. Then it shoots the image and triggers the face recognition algorithm.
However, this facial recognition feature does not continue to function. An infrared camera connected to a time of flight (ToF) range sensor sends out a signal to instruct the camera to take a picture when it detects a face. iPhone X then launches its dot matrix projector Shoot the image. Then send the general image and bit pattern image to the application processing unit (APU) for neural network training to identify the mobile phone user and unlock the mobile phone.
Yole's Cambou pointed out that 3D image computing has not yet begun. The 3D information is contained in the bit pattern image. "In order to perform 3D applications, the same APU can use another algorithm for calculating image depth maps;" he added: " Due to the computationally intensive structured light approach, the iPhone X makes full use of the powerful processing capabilities of the A11 chip. The use of neural networks is a key technology enabling this design."
There are basically three types of technologies that can achieve 3D sensing, including stereo vision, structured light, and ToF sensing; Yole provides the following table to see the advantages, disadvantages, and characteristics of these three technologies:
Comparison of three major 3D sensing technologies
(Source: Yole Developpement)
Because each 3D camera contains a bunch of sub-devices, Yole provides some component supplier combinations for the above three types of technologies.
First, stereo vision or active stereo can be achieved by combining a global shutter near-infrared (NIR) camera provided by Omnivision or ST, and NIR lighting by Osram, Lumentum, Finisar (optional). There are 3D hardware accelerators (ie visual processors) provided by Iniutive, Intel (Intel)/Movidius.
Second, in the structural light section, the necessary sub-devices include a global shutter NIR camera, as well as suppliers such as Omnivision or ST; there is also a structural NIR lighting assembly (optional), supplier is Austrian Microelectronics, Wonder Photo, South Korea's Namuga and Goertek; also a 3D reconstruction software provider for Apple Primesense, Mantis, Namuga and Orbbec.
The third ToF solution requires suppliers including PMD, Sony/Softkinetic's ToF lens, and NIR lighting components from OSRAM, Lumentum, Finisar and austriamicrosystems (optional). Cambou points out that Austrian microelectronics and The law provides ToF proximity detectors, but only supports a few pixels. The company has not yet developed ToF cameras.
Apple's high competition threshold
Assuming that iPhone X's TrueDepth is far better than Yole's competitors, what are the other 3D sensing solutions?
Cambou believes that the "neural engine" is a key and an obstacle that Apple's competitors can't overcome: "Qualcomm clearly has a "me too" solution ... but has not yet seen the results;" In addition, he said: "You need a powerful 2D-3D camera system that is suitable for smart phones. The cost is less than $15." This is not impossible, but it is really difficult to do.
iPhone X's Optical Hub System Cost
(Source: Yole Developpement and System Plus Consulting)
In addition, Cambou added that the system also needs to achieve a certain degree of biometric effect. If a photo can fool the system, then the solution is useless; he thinks this is the problem: "Apple knew from the beginning to use a 3D camera. What to do is a user interface that includes biometric identification solutions," while other competitors are simply trying to sell the same features of the cheaper version.
Cambou pointed out: "The level of performance required by Apple's identification system has set a standard that exceeds the technical capabilities of all competitors; the dilemma of Qijing Optoelectronics means that the mobile phone industry has not yet fully understood its application focus."
In addition to the cooperation between Qualcomm and Wonder Photo, are there other cases? Cambou pointed out that Mediatek also intends to join the 3D sensing battlefield as an APU supplier, intending to use a convolutional neural network (CNN)--similar In the nerve engine of Apple - to support biometric recognition; and he said that MediaTek clearly will provide Xiaomi in the future CNN accelerator, will be combined with the 3D camera designed by Ophir Zhongguang.
3D sensing technology market forecast
(Source: Yole Developpement)
Cambou also stated that if the mobile phone's facial recognition system will be used for mobile payment, we can always debate which 3D camera module is more secure, but there is one thing that cannot be ignored. It is the Chinese factor. For example, as long as Alipay determines Which kind of 3D sensing technology is good enough will be recognized by the Chinese market – a large part of China’s population already relies on mobile payment solutions such as WeChat payment, Alipay, etc. This will be the biggest determinant of the winners of 3D sensing technology.
Compilation: Judith Cheng